Class discovery method based on feature alignment and pseudo label quality evaluation

Abstract

The invention discloses a category discovery method based on feature alignment and pseudo label quality evaluation. The core lies in feature alignment and a pseudo label quality evaluation mechanism in self-distillation. In the data preprocessing stage, various image views are generated through data enhancement, multi-scale views with different resolutions are generated for the same image, cross-scale feature alignment loss is used for forcing features with different resolutions to keep consistency, and robustness of a model to scale change is improved. A Teacher model in the self-distillation module updates parameters through EMA to generate soft pseudo labels for unlabeled data, a pseudo label credibility evaluation mechanism is introduced, indexes such as entropy and class prototype distance confidence are synthesized to filter low-credibility labels, and a Student model takes screened high-quality pseudo labels as supervision signals for training. The method provides an efficient solution for generalized category discovery in an open set scene.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, machine learning, and pattern recognition, specifically unsupervised and weakly supervised learning, data mining, and image classification and retrieval. Its core goal is to solve the problem of generalized category discovery, which involves automatically discovering underlying category structures in unlabeled data while simultaneously distinguishing and clustering known and unknown categories. Background Art

[0002] In the fields of computer vision, machine learning, and pattern recognition, generalized category discovery is a key technology dedicated to automatically discovering latent category structures in unlabeled data and accurately distinguishing and clustering known and unknown categories. With the advent of the big data era, the demand for processing massive amounts of unlabeled data is growing, and the limitations of traditional category discovery methods are becoming increasingly apparent.

[0003] Traditional clustering algorithms, such as K-means and hierarchical clustering, are only applicable to closed scenarios where all samples belong to known categories. They cannot effectively identify unknown categories and are easily misclassified as existing categories or noise when faced with newly emerging categories, resulting in biased clustering results. While unsupervised and weakly supervised learning methods based on deep learning have certain advantages in feature extraction, they suffer from two core issues when processing mixed data of known and unknown categories: First, the model lacks the ability to perceive unknown categories, making it prone to overfitting predictions for unknown samples and misclassifying them as known categories. Second, the feature representations generated by existing self-supervised learning methods lack both intra-class compactness and inter-class discrimination, making it difficult to meet the high-precision feature requirements of generalized category discovery.

[0004] In summary, existing technologies, when dealing with generalized category discovery tasks, suffer from insufficient recognition capabilities for unknown categories, poor feature representation quality, and a lack of effective constraint mechanisms. To address these issues, the present invention proposes a category discovery method based on multi-resolution feature alignment and pseudo-label quality assessment. This method enhances the robustness of features to scale changes by constructing a multi-resolution feature alignment mechanism. It also optimizes the learning of unknown category features through a pseudo-label assessment self-distillation process, achieving improved discrimination for known categories and enhanced generalization capabilities for unknown categories. This provides an innovative solution for generalized category discovery in complex data scenarios that balances feature adaptability and label reliability. Summary of the Invention

[0005] The present invention provides a category discovery method based on multi-resolution feature alignment and pseudo-label quality assessment.

[0006] The purpose of this invention is to address the difficulties of existing category discovery technologies in simultaneously processing known and unknown categories, as well as the shortcomings in feature representation and category differentiation accuracy. By integrating multi-resolution feature alignment with pseudo-label evaluation self-distillation technology, a new generalized category discovery model is constructed. This model uses multi-resolution feature alignment to enhance the adaptability of features to images of different resolutions, and uses a pseudo-label evaluation mechanism to screen high-confidence labels, achieving efficient mining of latent category structures in low-labeled data, effectively distinguishing known from unknown categories, and significantly improving the model's generalization ability in open-set scenarios.

[0007] The technical solution adopted by the present invention comprises the following steps:

[0008] (1) Data preprocessing: Collect mixed data containing known and unknown categories, perform preprocessing operations such as normalization and standardization on the data to eliminate data scale differences. At the same time, the known category data are labeled with true labels to construct a known category dataset; the remaining unlabeled data is used as the unknown category dataset;

[0009] (2) Feature extraction: Build a feature extractor based on a convolutional neural network (CNN) or Transformer, perform forward propagation on known category datasets and unknown category datasets, and extract high-dimensional feature vectors;

[0010] (3) Construction and calculation of entropy constraints in the self-distillation module;

[0011] (3a) Teacher-Student model construction: Construct a Teacher model and a Student model with the same or similar structures. The Teacher model performs forward propagation on all samples (including known and unknown categories) and outputs soft labels (probability distribution).

[0012] (3b) Self-distillation learning: The Student model learns to fit the soft labels output by the Teacher model, calculates the distillation loss function, and optimizes the Student model parameters.

[0013] (4) Entropy constraint: For samples of unknown categories, the entropy value of their predicted probability distribution is calculated, and an entropy constraint loss function is constructed. By maximizing the entropy value, the model’s prediction of unknown samples is constrained to avoid overconfident misclassification.

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] (1) Stronger processing capabilities for unknown categories: Existing methods have difficulty effectively identifying unknown categories and are easily misclassified as known categories or noise. This paper uses an entropy-constrained self-distillation module to constrain the entropy value of the model's predicted probability distribution for unknown samples, allowing it to maintain a "cautious" prediction, significantly improving the ability to discover and distinguish unknown categories and effectively avoiding the overfitting prediction problem.

[0016] (2) Better feature representation performance: The feature representations generated by traditional clustering algorithms and single contrastive learning mechanisms are insufficient in terms of intra-class compactness and inter-class discrimination. This paper adopts a dual contrastive learning mechanism, designs contrast strategies for samples of known and unknown categories respectively, and combines it with self-distillation learning to learn more discriminative features, enhance the compactness of similar samples in the feature space and the separability of heterogeneous samples, and improve the overall quality of features.

[0017] (3) Higher generalization capability: Due to the lack of an effective constraint mechanism, existing self-distillation methods have poor generalization capabilities when processing open-set data. This paper achieves end-to-end training by jointly optimizing contrast loss, distillation loss, and entropy constraint loss. This enables the model to adapt to data scenarios with a mixture of known and unknown categories, and has stronger generalization performance in open-set scenarios, meeting the strict requirements of practical applications for data processing accuracy and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart for implementing the present invention.

[0019] Figure 2 Schematic diagram of data set division in the present invention.

[0020] Figure 3 Schematic diagram of the ViT-B / 16 feature extraction module of the present invention model.

[0021] Figure 4 Schematic diagram of the pseudo-label quality assessment module of the self-distillation module of the present invention

[0022] Figure 5 Schematic diagram of the model architecture of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and prediction effect diagrams.

[0024] The process of the present invention is as follows Figure 1 The specific implementation steps are as follows:

[0025] (1) Data preparation phase: Input datasets and obtain relevant image datasets as the original data source for model processing. This is the starting point of the entire process. Data quality and scale have an important impact on the subsequent model training effect. Here we use CFAR-10, CIFAR-100 and Image-100 datasets. The CFAR-10 dataset is divided into Figure 2 The classification of each data set is shown in the following table:

[0026] Dataset Tag Category Unlabeled category CFAR-10 5 5 CIFAR-100 80 20 Image-100 50 50

[0027] (2) Data enhancement and ViT feature extraction;

[0028] (2a) Data augmentation: Perform various augmentation operations on the input image, such as random flipping, rotation, and cropping. Data augmentation can expand the diversity of the dataset, increase the model's robustness to different image transformations, and reduce the risk of overfitting.

[0029] (2b) ViT feature extraction: Use VisionTransformer (ViT) architecture for feature extraction. Figure 2 As shown, the input image (such as 224×224 pixels) is divided into fixed-size image blocks (such as 16×16 pixels), and a total of (224 / 16) 2 = 196 patches. Each patch is mapped to an embedding vector of dimension D (e.g., 768) through a linear projection layer, forming a patch embedding sequence X patch ∈R N×D , N is the number of patches 196. Add a learnable positional encoding E to each patch embedding vector pos ∈R N×D , in order to preserve the spatial position information of the image block and avoid the model treating patches at different positions as equivalent. Then the embedding vector is layer normalized (LayerNorm) to stabilize the subsequent network training, and X = LayerNorm (X patch +E pos ). Then, the semantic association between different patches is captured through the multi-head attention mechanism to generate context-aware feature representation. The MLP module consists of two fully connected layers, which introduces nonlinear transformation to enhance the feature expression ability. A learnable classification tag (CLS Token) is added before the Patch sequence, and after Transformer encoding, its output is used as the global image feature z cls ∈R D , used for subsequent classification or contrastive learning.

[0030] (3) Multi-resolution feature alignment: The input image is first subjected to standardized data enhancement such as random cropping, flipping, and color jittering to generate views of three different resolutions: original size, 1.2x downsampling, and 0.8x downsampling. The pre-trained ViT-B / 16 model is used to extract features from each view (the parameters of the early layer of the model are fixed to retain the basic feature extraction capability, and only the subsequent layers are fine-tuned to learn specific semantic features). The multi-scale feature alignment loss function is used (the sum of the Euclidean distances of the feature vectors of views with different resolutions is calculated and averaged) to force the features to remain consistent. After back-propagation optimization, robust features that integrate multi-scale information are output, thereby improving the model's adaptability to image scale changes and the stability of feature representation.

[0031] (4) EMA self-distillation to generate pseudo labels and pseudo label quality assessment: In the self-distillation module (such as Figure 5 As shown in the right part), first initialize the Student model and based on the exponential moving average (EMA): θ teacher =m·θ teacher +(1-m)·θ student Update the Teacher model. The Teacher model generates pseudo labels based on the extracted features through the pseudo label generation module. These pseudo labels represent the Teacher model's prediction of the sample category or attribute. Then the reliability of the pseudo labels is evaluated through the credibility evaluation module, and the pseudo labels with high credibility are screened out to obtain the prediction distribution of the Student model. Figure 4 As shown in the figure, the pseudo-label quality confidence assessment module is mainly based on two aspects. The first aspect is the entropy uncertainty measurement: the entropy of the pseudo-label probability distribution is calculated. The higher the entropy value, the greater the uncertainty of the model's prediction of the sample class (such as uniform distribution), which may belong to unknown classes or difficult samples. The lower the entropy value, the more concentrated the prediction, and further judgment needs to be made based on the confidence level. The second aspect is the class prototype distance confidence: the cosine similarity (denoted as C) between the sample features and the prototypes of each known class is used to measure the matching degree. The larger the C value (closer to 1), the closer the feature is to a certain class, and the more "confidence" the model has in the class assignment. The smaller the C value, the more likely it belongs to an unknown class or an abnormal sample.

[0032] (6) Entropy constraint: Entropy constraint is performed on the Student model’s prediction distribution. Entropy constraint is designed to adjust the uncertainty of the model’s prediction, making the model’s prediction distribution more uniform or meeting specific distribution requirements, thus preventing the model from being overconfident about certain categories or ignoring certain potential categories, and further improving the model’s generalization ability and stability.

[0033] (7) Model training: The combined loss generated by the contrastive learning module and the entropy-constrained self-distillation loss of the self-distillation module are summed to obtain the total loss. The total loss is used to backpropagate the model, update the model weight parameters, and continuously optimize the model to train it in the direction of reducing the loss.

Claims

1. A category discovery method based on feature alignment and pseudo-label quality assessment, characterized in that: The specific steps are as follows: (1) Data preprocessing step: Perform multi-dimensional enhancement transformation on the input image to generate diverse image views; (1a) Introducing multi-resolution feature alignment: Generate views of different resolutions for the same image (such as original size, 1.2x downsampling, and 0.8x downsampling), and constrain the feature consistency of views of different resolutions through multi-resolution feature alignment loss. (2) Model architecture setup steps: Use the pre-trained ViT-B / 16 model as the feature extraction backbone. During the training process, fix some of the early layer parameters of the ViT-B / 16 model and only fine-tune the subsequent layers. (3) Construction of self-distilled parameterized classifier; (3a) constructing a Teacher-Student model structure, where the Teacher model parameters are updated from the Student model parameters by exponential moving average, and the momentum coefficient of the exponential moving average is set to a value close to 1; (3b) Pseudo-label generation step: The Teacher model generates soft pseudo-labels for unlabeled data. It uses temperature scaling technology to set a higher temperature parameter at the beginning of training to make the pseudo-label distribution softer. As the training progresses, the temperature parameter is linearly reduced to sharpen the pseudo-label distribution. The soft pseudo-label generation is based on the cosine similarity between the sample features and the class prototype vector, and is converted into a probability distribution through the softmax function. (3c) Introducing a pseudo-label credibility evaluation mechanism: The evaluation mechanism consists of two parts: calculating the entropy of the soft pseudo-labels generated by the Teacher model. and the class prototype distance confidence C = max(sim(z,c k )). in represents the probability that the sample belongs to the kth class, Indicates the contribution of each category probability to uncertainty. The sum is the accumulation of the relevant values ​​of all K categories, comprehensively considering the contribution of all category probabilities to uncertainty. The larger the C, the closer the sample characteristics are to a certain category, and the more "confident" the model is in assigning the sample category. The smaller the C, the greater the difference between the sample characteristics and all categories, and it may belong to an unknown category or an abnormal sample. C>T C With the constraints and evaluation of the two conditions, the quality of pseudo labels is improved to a certain extent. (3d) Student model training steps: The Student model uses the soft pseudo labels generated by the Teacher model as supervision signals and is trained using a loss function. At the same time, the class prototype vector is dynamically updated and the class prototype is weighted based on the probability of the sample being assigned to each class. (4) Entropy-constrained prediction bias correction: During model training, prediction bias may lead to a decline in model performance. Entropy constraint is an effective method to correct prediction bias. (4a) The model predicts the sample and converts the unnormalized scores output by the model into the predicted probability distribution of each category through the softmax function; (4b) calculating the entropy of the predicted probability distribution as a regularization term; (4c) Loss function adjustment step: Based on the original classification loss function, the entropy constraint term is added and a suitable weight coefficient is set. The optimization objective becomes minimizing the sum of the classification loss and the entropy constraint term.

2. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1 is characterized in that In step (1), an irregular window is used to extract sub-regions of different positions and sizes from the original image, and the window size and aspect ratio are randomly generated according to a uniform distribution. Alternatively, a mirror flip along the horizontal axis (Horizontal Axis Reflection) and a mirror flip along the vertical axis (Vertical Axis Reflection) are performed. The color is adjusted by adjusting the HSV color space parameters, performing additive offset on the image pixel values, and adjusting the pixel value distribution range through linear transformation.

3. According to claim 1, the category discovery method based on feature alignment and pseudo-label quality assessment is characterized in that The multi-scale feature loss function formula in step (1a) is: in, are different scale views of the same image, f θ It is a feature extraction function that improves the robustness of the model to scale changes by enforcing cross-scale feature similarity.

4. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1 is characterized in that In step (2), some of the early layer parameters of the ViT-B / 16 model are used to fine-tune only the subsequent layers, where the early layers are used to extract basic features of the image and the subsequent layers are used to learn semantic features related to specific tasks.

5. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1 is characterized in that In step (3b), the temperature parameter is set to 0.07 at the initial stage of training. As the training progresses, the temperature parameter is linearly reduced to 0.

04.

6. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1, characterized in that In step (4), when the prediction distribution is uniform, the entropy value is the largest, indicating that the model treats all categories equally; when the distribution is concentrated in a few categories, the entropy value decreases, reflecting the model's excessive preference for certain categories. By maximizing the batch average prediction, the distribution is forced to be uniform.

7. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1 is characterized in that The calculation formula of the entropy value in step (4b) is: Where p(k) is the probability value distribution of the kth class.

8. The category discovery method based on feature alignment and pseudo-label quality assessment according to claim 1, characterized in that The original classification loss function in step (4c) is l, the entropy constraint term is O(P), the weight coefficient is ε, and the adjusted loss function is l is the correlation function of the self-distilled pseudo-labels, ensuring that the model learns reliable category knowledge. ε is the regularization coefficient, balancing classification accuracy and distribution uniformity, and is set to 0.1.

Images