Semantically consistent double contrast learning long-tail image recognition method
By employing a dual-contrast learning and dynamic semantic alignment strategy, combined with a dual-branch decoupled classifier, the feature representation is optimized and label noise is corrected, thus solving the class imbalance problem in long-tail recognition and improving the model's recognition accuracy and generalization ability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2025-11-21
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies struggle to effectively balance the influence of head and tail categories in long-tail recognition, resulting in insufficient generalization ability of the model under unknown test distributions. Furthermore, traditional methods increase training complexity and computational overhead.
A dual contrastive learning module is adopted to integrate supervised contrastive learning and self-supervised contrastive learning. Combined with dynamic semantic alignment strategy and dual-branch decoupled classifier, feature representation is optimized and label noise is corrected. The classifier weights are dynamically adjusted to suppress the head class dominance effect.
It significantly improves the model's recognition accuracy and generalization ability on long-tail datasets, solves the problems of feature space distortion and low label credibility caused by class imbalance, and achieves higher tail class recognition performance.
Smart Images
Figure CN121883907A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically a semantically consistent dual-contrast learning long-tail recognition method. Background Technology
[0002] Long-tail recognition is an important research area in machine learning, especially in visual recognition tasks. The long-tail distribution problem is a long-standing structural challenge, characterized by extreme imbalance in data distribution: a small number of head classes occupy the majority of samples, while the far more numerous tail classes have only a small number of samples. This distribution characteristic makes traditional deep learning models prone to class bias: models trained under a long-tail distribution are more likely to favor head classes with abundant training data, resulting in poor performance on tail classes with limited data. In high-risk scenarios such as medical imaging diagnosis, industrial defect detection, and biodiversity monitoring, misclassification of tail classes can lead to serious consequences (such as missed diagnoses of rare diseases and product safety hazards).
[0003] Currently, research on long-tail identification focuses on learning high-performance models from imbalanced data. In recent years, a significant amount of research has been conducted on deep long-tail learning. Recent studies have achieved remarkable results using data augmentation techniques to fill in the losses of tail classes and contrastive learning techniques to optimize intra-class aggregation and inter-class separation. These methods aim to adapt to a uniform class distribution by rebalancing the performance of different classes. However, learning directly from long-tail data can easily lead to over-biased predictions towards the head classes. Directly rebalancing training on tail classes would disrupt the original distribution characteristics of the long-tail data, exacerbating the overfitting risk of tail classes and sacrificing the discriminative ability of the head classes. Therefore, existing methods typically employ a two-stage decoupled training paradigm: the first stage trains the feature extractor on the original distribution, and the second stage fixes the feature representations and fine-tunes the balanced classifier. While this paradigm significantly improves performance, it inevitably increases training complexity and computational overhead.
[0004] Meanwhile, most studies assume a uniform distribution of test categories (i.e., equal sample size per category), but actual test data may follow any distribution (uniform, long-tailed, or opposite to the training distribution). In this context, a single classifier model is easily dominated by the head category, which occupies a large amount of data. Weight updates tend to favor the feature distribution of high-frequency categories, leading to degradation of the classification boundary of tail categories. Therefore, balancing the mutual influence between head and tail categories under unknown test distributions has become a core challenge in the field of long-tail recognition.
[0005] After searching, the application publication number CN118411555A, a long-tail learning method based on Fourier transform and contrastive learning, attempts to "alleviate" the symptoms of uneven data distribution by increasing the diversity of tail samples. Its main drawback is the superficiality and passivity of its solution: it fails to address the three interrelated deep-seated causes in long-tail recognition: feature space distortion, low label credibility, and classifier decision bias.
[0006] Conversely, this invention provides a proactive, in-depth, and systematic diagnostic and treatment plan:
[0007] The dual contrastive learning module corresponding to claims 1 and 3: This module actively optimizes feature representations by fusing supervised contrastive learning (BalSCL) and self-supervised contrastive learning (SimCL). Its direct technical effect is to systematically improve the intra-class aggregation and inter-class discriminativity of the feature space, and enhance the model's robustness to data augmentation, laying a high-quality feature foundation for subsequent classification tasks. The dynamic semantic alignment strategy corresponding to claim 4: This strategy adaptively corrects mixed label noise caused by inter-class imbalance by utilizing class prototypes and classification confidence. Its direct technical effect is to generate semantically consistent and reliable training labels, thereby reducing the interference of erroneous labels on model training at the source and ensuring the correctness of the learning direction. The dual-branch decoupled classifier corresponding to claim 5: This design decouples the Vanilla branch and the rebalancing branch, and dynamically adjusts the weights by combining a periodic cumulative learning strategy. Its direct technical effect is to achieve a balance between sufficient learning of head class knowledge and targeted reinforcement of tail class representations, thereby effectively suppressing the dominance of head classes in a single model and improving the model's generalization ability under unknown test distributions.
[0008] This solution, which deeply integrates multiple advanced technologies in a specific way to address the root causes of long-tail problems, is not a simple aggregation of existing technologies, but rather demonstrates outstanding substantive characteristics. The resulting significant improvement in overall performance proves remarkable progress and possesses strong inventiveness and patentability. Summary of the Invention
[0009] This invention aims to solve the problems of the prior art. It proposes a semantically consistent dual-contrast learning method for long-tail image recognition. The technical solution of this invention is as follows:
[0010] A semantically consistent dual-contrast learning method for long-tail image recognition includes the following steps:
[0011] Step 1: Supervised contrastive learning BalSCL is performed on the original long-tailed image dataset to optimize the feature space, enhancing intra-class compactness and inter-class discriminativeness; at the same time, unsupervised contrastive learning SimCL is performed on the generated mixed images to improve the model's ability to learn feature invariance to data augmentation transformations.
[0012] Step 2: Based on the prototype-contrast dynamic semantic alignment strategy, the class prototypes and classification probabilities in the semantic space are used to adaptively correct the label noise caused by class imbalance, and the consistency of the feature space is coordinated by prototype-based constraints.
[0013] Step 3: Invest in a dual-branch decoupled classifier and dynamically adjust the classifier weights using a periodic cumulative learning strategy to achieve dynamic attention transfer and tail category reinforcement, suppress the head category dominance effect, and strengthen the tail category representation learning.
[0014] Furthermore, prior to step 1, the following steps are included: After class-weighted sampling of the long-tail data according to the frequency of category samples, two batches of enhanced MixUp and CutMix hybrid images are generated, specifically:
[0015] 1.1 For each batch of datasets, apply two-stream hybrid information enhancement to the initial images based on class frequencies: for the background image ( , Random sampling is performed on the foreground image ( , Perform inverse sampling;
[0016] Mixup enhancement is performed on the two sampled data sets, specifically including:
[0017] 1) λ represents the proportion of foreground and background images in the blended image. It is obtained by randomly sampling from a Beta distribution to randomly blend images and their labels. β is the hyperparameter input.
[0018] (1)
[0019] 2) The definition of mixed images and labels , :
[0020]
[0021] (2)
[0022] in( , ),( , ( ) represents the two original images and their labels used to generate the blended image;
[0023] 3) The two sampled data are blended and enhanced using CutMix. CutMix generates random cropping boxes to crop the foreground image and paste it onto the background image. The blended image is defined as follows:
[0024] (3)
[0025] In the formula, ⊙ represents element-wise multiplication, and M∈ This represents a binary mask matrix indicating the cropped and pasted region between two images, where 1 indicates a mask completely filled with 1s, for bounding box coordinates B=( , , Sampling is performed to indicate and The cropping area above is uniformly sampled for the coordinates of the bounding box.
[0026] (4)
[0027] λ is also sampled from Beta(β,β), and their mixed labels are the same as MixUp.
[0028] Furthermore, step 1: using supervised contrastive BalSCL to optimize intra-class aggregation and inter-class separation capabilities in the feature space on the initial data, and using self-supervised contrastive SimCL to learn data enhancement invariance on the mixed augmented data to enhance feature robustness at the sample level, specifically includes the following steps:
[0029] 1.2 After passing through the backbone network ResNet, a method combining a nonlinear multilayer perceptron (MLP) and L2 normalization is used to project the samples onto the representation. , z j z represents the feature representation of different images of the same category. l The feature representations of different categories of images will Represented as the k-th class and its corresponding class prototype The set of all representations of z, express The positive set, through from Remove from middle The representation of The supervised contrast loss obtained is based on dynamic classification centers. It can be represented as:
[0030] (5)
[0031] 1.3 Self-supervised contrastive branch SimCL: This branch uses foreground-background decoupling to enhance the generation of instance-level sample pairs. , ),( , After the two mixed images pass through the backbone network, their representations are obtained using a projection head. and Then they are processed into output by the prediction head. and Unlike the processing in the supervised comparison branch mentioned above, ( , )and( , Treating these as positive sample pairs, we learn feature invariance under data augmentation by minimizing the negative cosine similarity between these representations.
[0032] (6)
[0033]
[0034] (7)
[0035] By using hyperparameters α and β to balance the contributions of the supervised contrastive branch and the self-supervised contrastive branch, the dual-contrast loss is ultimately defined as:
[0036] (8).
[0037] Furthermore, step 2: reweighting the mixed positive and negative labels of the long-tail data using empirical class frequencies, and then balancing the traditional loss and the rebalanced loss using a factor that accumulates over time, specifically includes the following steps:
[0038] 1) First, calculate the synthetic sample ( , Cosine similarity between the centroids of all k classes:
[0039] (9)
[0040] Where δ represents the normalization operation that applies the constraint that the sum of the weights of all classes equals 1; c k Y represents the first step of calculating the Euclidean distance using the current category prototype; Y represents the total number of categories in the dataset.
[0041] 2) Mixed samples ( , The confidence score depends on the following: and Foreground class and background class The prediction accuracy is then calculated by integrating the blending ratio λ of the blended images using a weighted summation, thus producing the synthetic prediction accuracy for the blended categories.
[0042] (10)
[0043] 3) Then divide by the cumulative sum of prediction accuracies for all categories, and multiply by the scaling factor γ to control the strength of the weighting factor, thus obtaining the weighting factor of the current mixed image:
[0044] (11)
[0045] 4) λ-based hybrid image labels obtained through hybrid data augmentation and semantically derived labels obtained through feature representation computation The weighted sum is used to generate semantically consistent tags:
[0046] (12).
[0047] Furthermore, step 3: using a dual-branch classifier architecture to process the mixed image pairs generated by Mixup and CutMix ( , ),( , Specifically, this includes:
[0048] 1) The Vanilla branch uses a hybrid version of the baseline cross-entropy loss, improved for both hybrid images. :
[0049] (13)
[0050] 2) For the rebalancing branch, use the mixed image category prediction accuracy. To achieve dynamic weighting, scaling factor Higher weights are assigned to categories with lower prediction accuracy, while the weights of categories with higher accuracy are reduced. This yields the rebalancing classification head loss for long-tailed data. :
[0051] (14)
[0052] 3) The attention distribution of the dual decoupled classifier is smoothly shifted from the head class to the tail class, thereby suppressing the head class advantage while strengthening the feature learning of the tail class. The cumulative learning parameter T is formally defined as:
[0053] (15)
[0054] in, This represents the current training period, E represents the preset maximum number of training periods, and the hyperparameter γ=2 controls the smoothness of attention transfer between the two-branch classifiers;
[0055] In the initial training phase where T → 0, the classifier weights are prioritized over... This allows for rapid feature learning by leveraging rich data from the Head class; as training progresses towards T→1, the weighting mechanism gradually shifts its focus to... This strengthens the representation learning for the Tail class; the unified two-branch classifier loss form is expressed as:
[0056] (16).
[0057] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements a semantically consistent dual contrastive learning long-tail image recognition method as described in any one of the claims.
[0058] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a semantically consistent dual contrastive learning long-tail image recognition method as described in any one of the claims.
[0059] The advantages and beneficial effects of this invention are as follows:
[0060] This invention proposes a long-tail classification method based on dual contrastive learning and semantic consistency enhancement, aiming to solve the class imbalance problem in long-tail classification. This method significantly improves the accuracy of long-tail classification by designing a dual contrastive learning framework, correcting label noise caused by class imbalance, and designing a dual-branch decoupled classifier. The main advantages and beneficial effects of this invention are as follows:
[0061] 1. Dual Contrastive Learning Framework: This method optimizes the feature space by fusing supervised contrastive learning, enhancing intra-class compactness and inter-class discriminativity; simultaneously, it introduces self-supervised contrastive learning to improve the model's ability to learn feature invariance to data augmentation transformations. This improves the model's ability to identify long-tailed categories.
[0062] 2. Correcting label noise caused by class imbalance: To address label noise caused by class imbalance, this method designs a prototype-contrast dynamic semantic alignment strategy. By utilizing class prototypes and classification probabilities in the semantic space, the method adaptively corrects label noise caused by class imbalance. The prototype-contrast dynamic semantic alignment strategy coordinates feature space consistency through prototype-based constraints, effectively mitigating label noise caused by class imbalance.
[0063] 3. Two-branch decoupled classifier design: A two-branch decoupled classifier was designed, which dynamically adjusts the classifier weights by combining a periodic cumulative learning strategy. This achieves dynamic attention transfer and tail category reinforcement, suppressing the head category dominance effect and strengthening the tail category representation learning. This further improves the recognition performance for long-tail categories.
[0064] The inventiveness of this invention is mainly reflected in the collaborative system constructed by the technical features protected by claims 3, 4, and 5. This solution forms an organic technical closed loop through a progressive design of "dual contrastive learning (feature space optimization), dynamic semantic alignment (training signal correction), and decoupled classifier dynamic learning (classification decision balancing)".
[0065] In this closed loop, the output of the preceding module optimizes the input conditions of the following module, resulting in deep coupling and synergistic gains among various technical features. This collectively solves the cascading problems of feature representation, label credibility, and classifier bias in long-tail recognition. This approach, which systematically integrates multiple advanced training paradigms to solve fundamental problems, transcends the simple superposition of conventional techniques. It constitutes a complete and non-obvious solution, thereby producing a synergistic technical effect of "1+1+1>3," demonstrating outstanding substantive characteristics and significant progress. Attached Figure Description
[0066] Figure 1 This is a schematic diagram of the overall network model structure of a preferred embodiment of the present invention. Detailed Implementation
[0067] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.
[0068] The technical solution of the present invention to solve the above-mentioned technical problems is:
[0069] This invention is based on the ResNet network. ResNet effectively alleviates the vanishing gradient problem in deep networks through residual learning and skip connections, enabling the successful training of extremely deep network structures and achieving breakthrough performance on datasets such as ImageNet. Its core structure is the residual block, which allows the network to learn the residual mapping between input and output, i.e., H(x) = F(x) + x, thereby simplifying the optimization process and improving feature representation capabilities. For classification, we consider the long-tail recognition problem and perform dual contrastive learning to enhance intra-class compactness and inter-class discriminativity while improving the model's feature invariance learning ability to data augmentation transformations. Then, based on the prototype-contrast dynamic semantic alignment strategy, we adaptively correct label noise caused by inter-class imbalance using class prototypes and classification probabilities in the semantic space. Finally, we design a dual-branch decoupled classifier, combined with a periodic cumulative learning strategy to dynamically adjust classifier weights, achieving dynamic attention transfer and tail-class reinforcement, suppressing the head-class dominance effect, and strengthening tail-class representation learning. This further improves the recognition performance of long-tail categories.
[0070] Figure 1 The overall network model structure of this invention is shown in the diagram, which includes the following steps:
[0071] A long-tail identification method based on information enhancement and reweighted multi-expert networks includes the following steps:
[0072] A long-tail recognition method based on dual contrastive learning and semantic consistency enhancement includes the following steps:
[0073] Step 1: Perform supervised contrastive learning (BalSCL) on the original long-tail dataset to optimize the feature space and enhance intra-class compactness and inter-class discriminability; at the same time, perform unsupervised contrastive learning (SimCL) on the generated mixed images to improve the model's ability to learn feature invariance to data augmentation transformations.
[0074] Step 2: Then, based on the prototype-contrast dynamic semantic alignment strategy, the label noise caused by class imbalance is adaptively corrected by using class prototypes and classification probabilities in the semantic space, and the consistency of the feature space is coordinated by prototype-based constraints.
[0075] Step 3: Finally, the classifier is implemented using a dual-branch decoupled classifier. A periodic cumulative learning strategy is used to dynamically adjust the classifier weights, achieving dynamic attention transfer and tail-category reinforcement. This suppresses the head-category dominance effect and strengthens tail-category representation learning, thereby further improving the recognition performance for long-tail categories.
[0076] Furthermore, prior to step 1, two batches of enhanced MixUp and CutMix hybrid images are generated by performing class-weighted sampling of the long-tail data according to the frequency of class samples. Specifically, this includes the following steps:
[0077] 1.1 For each batch of datasets, apply two-stream hybrid information enhancement to the initial images based on class frequencies: for the background image ( , Random sampling is performed on the foreground image ( , Performing inverse sampling can help mitigate the impact of tail data having a small proportion in the dataset.
[0078] Mixup enhancement is performed on the two sampled data sets, specifically including:
[0079] 1) λ is randomly sampled from a Beta distribution to randomly mix images and their labels, where β is a hyperparameter:
[0080] (1)
[0081] 2) The definition of mixed images and labels , :
[0082]
[0083] (2)
[0084] in( , ),( , ) represents the two original images and their labels used to generate the blended image.
[0085] 3) Perform CutMix blending enhancement on the two sampled data. CutMix generates random cropping boxes to crop the foreground image and paste it onto the background image. The blended image is defined as follows:
[0086] (3)
[0087] In the formula, ⊙ represents element-wise multiplication, and M∈ This represents a binary mask matrix indicating the cropped and pasted region between two images, where 1 indicates a mask completely filled with 1s. Specifically, we define the bounding box coordinates B=( , , Sampling is performed to indicate and The cropping area is defined above. Uniform sampling of the bounding box coordinates is performed.
[0088] (4)
[0089] λ is also sampled from Beta(β,β), and their mixed labels are the same as MixUp.
[0090] Furthermore, step 1 involves using supervised contrastive BalSCL on the initial data to optimize intra-class aggregation and inter-class separation capabilities in the feature space, and using self-supervised contrastive SimCL on the mixed augmented data to learn data enhancement invariance, thereby enhancing the robustness of features at the sample level. Specifically, this includes the following steps:
[0091] 1.2 After passing through the backbone network (ResNet), we project the samples onto the representation using a combination of nonlinear multilayer perceptron (MLP) and L2 normalization. .we will Represented as the k-th class and its corresponding class prototype The set of all representations of z. express The positive set, through from Remove from middle The representation of And thus obtained. Based on the above description, the supervised contrastive loss with dynamic classification centers can be expressed as:
[0092] (5)
[0093] 1.3 Self-Supervised Contrast Branch (SimCL): This branch uses foreground-background decoupling to enhance the generation of instance-level sample pairs. , ),( , After the two mixed images pass through the backbone network, their representations are obtained using a projection head. and Then they are processed into output by the prediction head. and Unlike the processing in the supervised comparison branch mentioned above, we will ( , )and( , These are considered as positive sample pairs. By minimizing the negative cosine similarity between these representations, we learn feature invariance under data augmentation:
[0094] (6)
[0095]
[0096] (7)
[0097] By using hyperparameters α and β to balance the contributions of supervised contrast and self-supervised contrast branches, we evaluated the performance of different parameter configurations in subsequent ablation studies. The dual contrast loss was ultimately defined as:
[0098] (8)
[0099] Furthermore, step 2: reweighting the mixed positive and negative labels of the long-tail data using empirical class frequencies, and then balancing the traditional loss and the rebalanced loss using a factor that accumulates over time, specifically includes the following steps:
[0100] 1) Taking Mixup as an example, first calculate the synthetic sample ( , Cosine similarity between the centroids of all k classes:
[0101] (9)
[0102] Where δ represents the standardized operation that applies the constraint that the sum of the weights of all classes equals 1.
[0103] 2) Mixed samples ( , The confidence score should depend on the following: and Foreground class and background class The prediction accuracy is then calculated by integrating the blending ratio λ of the blended images using a weighted summation. This yields the synthetic prediction accuracy for the blended categories.
[0104] (10)
[0105] 3) Then divide by the cumulative sum of the prediction accuracies of all categories to obtain the weighting factor of the current mixed image:
[0106] (11)
[0107] 4) Lambda-based labels obtained through hybrid data augmentation and semantically derived labels obtained through feature representation computation The weighted sum is used to generate semantically consistent tags:
[0108] (12)
[0109] Furthermore, step 3: using a dual-branch classifier architecture (Vanilla Branch + Re-Balated Branch) to process the mixed image pairs generated by Mixup and CutMix. , ),( , ).
[0110] 1) The Vanilla branch uses a hybrid version of the baseline cross-entropy loss, improved for both hybrid images:
[0111] (13)
[0112] 2) For the rebalancing branch, we utilize the mixed image category prediction accuracy. To achieve dynamic weighting. Scale factor Higher weights are assigned to categories with lower prediction accuracy (typically tail categories), while the weights of categories with higher accuracy are reduced.
[0113] (14)
[0114] 3) The attention distribution of the dual decoupled classifier is smoothly shifted from the head class to the tail class, thereby suppressing the head class advantage while enhancing the feature learning of the tail class. The cumulative learning parameter T is formally defined as:
[0115] (15)
[0116] in, Let E represent the current training period, and let E represent the preset maximum number of training periods. The hyperparameter γ=2 controls the smoothness of attention transfer between the two-branch classifiers. The effectiveness of this mechanism has been confirmed through comprehensive ablation studies.
[0117] 4) In the initial training phase where T → 0, the classifier weights are prioritized over... This allows for rapid feature learning by leveraging rich data from the Head class. As training progresses towards T→1, the weighting mechanism gradually shifts its focus to... This strengthens the representation learning for the Tail class. The unified two-branch classifier loss form is expressed as:
[0118] (16)
[0119] This invention introduces a novel long-tail recognition network—the Semantic Consistent Dual Contrastive Learning Network (SCDC)—by employing dual contrastive learning and dynamic semantic alignment strategies. This network facilitates effective learning of long-tail datasets and enhances the recognition ability of tail categories. The network optimizes the feature space through supervised contrastive learning on the original long-tail dataset, enhancing intra-class compactness and inter-class discriminativity. Simultaneously, unsupervised contrastive learning on generated mixed images improves the model's feature invariance learning ability to data augmentation transformations. This invention effectively solves the problem of poor inter-class discriminativity caused by class imbalance in traditional long-tail recognition methods. Through a prototype-contrastive dynamic semantic alignment strategy, class prototypes and classification probabilities in the semantic space adaptively correct label noise caused by inter-class imbalance, thereby improving the model's overall category recognition performance. Furthermore, the dual-branch decoupled classifier design of this invention, combined with a periodic cumulative learning strategy, dynamically adjusts the classifier weights, achieving dynamic attention transfer and tail category reinforcement, suppressing the head category dominance effect, and strengthening tail category representation learning. This further improves the recognition performance of long-tail categories. Extensive experiments demonstrate that SCDC significantly improves the tail class recognition accuracy of mainstream long-tail benchmarks (ImageNet-LT, CIFAR100-LT, CIFAR10-LT) compared to other long-tail recognition methods, proving its robustness and generalization ability in handling long-tail problems. This establishes a new technical approach for robust feature learning and effective classification decisions under long-tail distributions. In practical applications, especially when dealing with datasets with uneven class distributions, the method of this invention can significantly improve the model's recognition performance and generalization capabilities.
[0120] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.
[0121] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0123] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.
Claims
1. A semantically consistent dual-contrast learning method for long-tail image recognition, characterized in that, Includes the following steps: Step 1: Supervised contrastive learning BalSCL is performed on the original long-tailed image dataset to optimize the feature space, enhancing intra-class compactness and inter-class discriminativeness; at the same time, unsupervised contrastive learning SimCL is performed on the generated mixed images to improve the model's ability to learn feature invariance to data augmentation transformations. Step 2: Based on the prototype-contrast dynamic semantic alignment strategy, the class prototypes and classification probabilities in the semantic space are used to adaptively correct the label noise caused by class imbalance, and the consistency of the feature space is coordinated by prototype-based constraints. Step 3: Invest in a dual-branch decoupled classifier and dynamically adjust the classifier weights using a periodic cumulative learning strategy to achieve dynamic attention transfer and tail category reinforcement, suppress the head category dominance effect, and strengthen the tail category representation learning.
2. The semantically consistent dual-contrast learning long-tail image recognition method according to claim 1, characterized in that, Before step 1, the following steps are also included: after class-weighted sampling of the long-tail data according to the frequency of category samples, two batches of enhanced MixUp and CutMix hybrid images are generated, specifically: 1.1 For each batch of datasets, apply two-stream hybrid information enhancement to the initial images based on class frequencies: for the background image ( , Random sampling is performed on the foreground image ( , Perform inverse sampling; Mixup enhancement is performed on the two sampled data sets, specifically including: 1) λ represents the proportion of foreground and background images in the blended image. It is obtained by randomly sampling from a Beta distribution to randomly blend images and their labels. β is the hyperparameter input. （1） 2) The definition of mixed images and labels , : （2） in( , ),( , ( ) represents the two original images and their labels used to generate the blended image; 3) The two sampled data are blended and enhanced using CutMix. CutMix generates random cropping boxes to crop the foreground image and paste it onto the background image. The blended image is defined as follows: （3） In the formula, ⊙ represents element-wise multiplication, and M∈ This represents a binary mask matrix indicating the cropped and pasted region between two images, where 1 indicates a mask completely filled with 1s, for bounding box coordinates B=( , , Sampling is performed to indicate and The cropping area above is uniformly sampled for the coordinates of the bounding box. （4） λ is also sampled from Beta(β,β), and their mixed labels are the same as MixUp.
3. The semantically consistent dual-contrast learning long-tail image recognition method according to claim 1, characterized in that, Step 1 involves optimizing intra-class aggregation and inter-class separation capabilities in the feature space using supervised contrastive BalSCL on the initial data, and learning data invariance using self-supervised contrastive SimCL on the mixed augmented data to enhance the robustness of features at the sample level. This specifically includes the following steps: 1.2 After passing through the backbone network ResNet, a method combining a nonlinear multilayer perceptron (MLP) and L2 normalization is used to project the samples onto the representation. , z j z represents the feature representation of different images of the same category. l The feature representations of different categories of images will Represented as the k-th class and its corresponding class prototype The set of all representations of z, express The positive set, through from Remove from middle The representation of The supervised contrast loss obtained is based on dynamic classification centers. It can be represented as: （5） 1.3 Self-supervised contrastive branch SimCL: This branch uses foreground-background decoupling to enhance the generation of instance-level sample pairs. , ),( , After the two mixed images pass through the backbone network, their representations are obtained using a projection head. and Then they are processed into output by the prediction head. and Unlike the processing in the supervised comparison branch mentioned above, ( , )and( , Treating these as positive sample pairs, we learn feature invariance under data augmentation by minimizing the negative cosine similarity between these representations. (6) (7) By using hyperparameters α and β to balance the contributions of the supervised contrastive branch and the self-supervised contrastive branch, the dual-contrast loss is ultimately defined as: (8)。 4. The semantically consistent dual-contrast learning long-tail image recognition method according to claim 1, characterized in that, Step 2 involves reweighting the mixed positive and negative labels of the long-tail data using empirical class frequencies, and then balancing the traditional loss and the rebalanced loss using a factor that accumulates over time. This specifically includes the following steps: 1) First, calculate the synthetic sample ( , Cosine similarity between the centroids of all k classes: (9) Where δ represents the normalization operation that applies the constraint that the sum of the weights of all classes equals 1; c k Y represents the first step of calculating the Euclidean distance using the current category prototype; Y represents the total number of categories in the dataset. 2) Mixed samples ( , The confidence score depends on the following: and Foreground class and background class The prediction accuracy is then calculated by integrating the blending ratio λ of the blended images using a weighted summation, thus producing the synthetic prediction accuracy for the blended categories. (10) 3) Then divide by the cumulative sum of the prediction accuracies of all categories, and multiply by the scaling factor γ to control the strength of the weighting factor, thus obtaining the weighting factor of the current mixed image: (11) 4) λ-based hybrid image labels obtained through hybrid data augmentation and semantically derived labels obtained through feature representation computation The weighted sum is used to generate semantically consistent tags: (12)。 5. The semantically consistent dual-contrast learning long-tail image recognition method according to claim 4, characterized in that, Step 3: Process the mixed image pairs generated by Mixup and CutMix using a dual-branch classifier architecture. , ),( , Specifically, this includes: 1) The Vanilla branch uses a hybrid version of the baseline cross-entropy loss, improved for both hybrid images. : (13) 2) For the rebalancing branch, use the mixed image category prediction accuracy. To achieve dynamic weighting, scaling factor Higher weights are assigned to categories with lower prediction accuracy, while the weights of categories with higher accuracy are reduced. This yields the rebalancing classification head loss for long-tailed data. : (14) 3) The attention distribution of the dual decoupled classifier is smoothly shifted from the head class to the tail class, thereby suppressing the head class advantage while strengthening the feature learning of the tail class. The cumulative learning parameter T is formally defined as: (15) in, This represents the current training period, E represents the preset maximum number of training periods, and the hyperparameter γ=2 controls the smoothness of attention transfer between the two-branch classifiers; In the initial training phase where T → 0, the classifier weights are prioritized over... This allows for rapid feature learning by leveraging rich data from the Head class; as training progresses towards T→1, the weighting mechanism gradually shifts its focus to... This strengthens the representation learning for the Tail class; the unified two-branch classifier loss form is expressed as: (16)。 6. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the semantically consistent dual contrastive learning long-tail image recognition method as described in any one of claims 1 to 5.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semantically consistent dual contrastive learning long-tail image recognition method as described in any one of claims 1 to 5.