A data relabeling classification algorithm based on clustering learning
By using dynamic clustering and pseudo-label-optimized data relabeling classification algorithms, the overfitting and underfitting problems of long-tail datasets in facial expression recognition are solved, thereby improving the model's classification and recognition performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI DROIDUP CO LTD
- Filing Date
- 2024-12-05
- Publication Date
- 2026-06-12
AI Technical Summary
Existing technologies for facial expression recognition suffer from the problem of long-tailed datasets, which leads to deep learning models overfitting to head data and underfitting to tail categories. Furthermore, traditional resampling and reweighting methods are difficult to effectively improve recognition performance and take into account intra-class sample differences.
A cluster-based data relabeling classification algorithm is adopted. The minority class samples are split into subclass datasets through a dynamic clustering module. By combining pseudo-labels and class balance loss function, the model training process is optimized and the model's recognition performance for the majority class and subclasses is improved.
The combination of dynamic clustering and pseudo-labels can adapt to changes in data distribution, alleviate class imbalance, improve the overall recognition performance of the model, and avoid the reduction in recognition accuracy of traditional methods.
Smart Images
Figure CN122196598A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and more specifically to a data relabeling and classification algorithm based on clustering learning. Background Technology
[0002] In traditional classification and recognition tasks, the distribution of training data is often manually balanced, meaning the number of samples from different classes is not significantly different. A balanced dataset greatly simplifies the requirements for algorithm robustness and, to some extent, ensures the reliability of the resulting model. However, as the number of classes of interest increases, maintaining balance between classes leads to an exponential increase in data collection costs. This is particularly true in facial expression recognition technology, which relies heavily on large labeled expression datasets for training. However, real-world datasets often suffer from class imbalance, forming long-tail datasets. A few classes in the training set (head class) contain a large amount of labeled data, while the majority of classes (tail class) have only a small amount of labeled data. Figure 3 As shown, the number of samples of some normal facial expressions is far greater than the number of samples of other abnormal facial expressions. Normal facial expression categories are relatively scarce and simple, while other abnormal facial expression categories are complex and diverse. If a classification and recognition system is trained directly using long-tail datasets, it will often overfit the head classes and underfit the tail classes.
[0003] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has achieved remarkable results in the field of image recognition. For example, patent document CN113903064A discloses a method for facial expression recognition and emotion tracking based on a complex optimized dataset, including: S1: classifying and constructing facial expression data according to function; S2: using a confidence learning algorithm with complex analysis assisted by manual mixing to process the corresponding functional datasets separately, obtaining datasets with low noise; S3: training a CNN model based on each dataset to obtain real-time recognition results of facial expression components; S4: designing an emotion tracking algorithm based on the single-frame results of expression recognition to obtain emotion tracking results. This technical solution uses a confidence learning algorithm with complex analysis assisted by manual mixing to process the data, reducing noise in the original dataset, and then trains a CNN model. However, this method of filtering the original dataset cannot solve the problem that the long-tail distribution of the facial expression dataset affects the generalization ability and recognition accuracy of the deep learning model.
[0004] To address the class imbalance problem in facial expression datasets, traditional methods such as resampling and reweighting can alleviate the issue to some extent. Resampling methods balance the dataset by increasing samples from the majority class or decreasing samples from the minority class, which can easily lead to information loss, sacrificing the recognition performance of the minority class (head class), and can still cause overfitting. While reweighting methods can enhance the recognition ability of the majority class (tail class), they often sacrifice the recognition accuracy of the minority class (head class), resulting in poor overall performance. Furthermore, these methods do not consider the differences between samples within a class, making it difficult to fully mine and utilize the potential information of the samples to improve recognition performance. Summary of the Invention
[0005] To address the shortcomings of existing machine learning techniques, this invention proposes a cluster-based data relabeling classification algorithm that can improve the classification and recognition performance of imbalanced facial expression datasets and fully mine and utilize the potential information in the sample datasets.
[0006] The specific technical solution is as follows:
[0007] A data relabeling and classification algorithm based on clustering learning includes the following steps:
[0008] S1. After preprocessing the data in the original samples, feature vectors are extracted through the feature extraction module to form sample data;
[0009] S2. The sample dataset of the minority class is split into several sub-class datasets by the dynamic clustering module, while the sample dataset of the majority class remains unchanged. The extracted feature vector is used as the input of the dynamic clustering module. If the sample data belongs to the sample dataset of the minority class, the sample data is assigned to the nearest sub-class dataset according to the distance between the sample data and the cluster center. Then, pseudo-labels are assigned to all sample data.
[0010] S3. Use pseudo-labels to train the subclasses, and use the label mapping module to map the prediction results of the subclass dataset to which the subsequent sample data belongs back to the real label space based on the mapping relationship learned during the training process.
[0011] Furthermore, the data in its original samples are facial expression image data, and its feature extraction module is implemented using a CNN backbone network.
[0012] Furthermore, its feature extraction module is implemented using convolutional neural networks such as ResNet, VGG, or MobileNet.
[0013] Furthermore, pseudo-labels are used to calculate the loss during training, and class-balanced loss is introduced to further balance the weights of different classes, enabling the model to pay more attention to the majority class and the split subclasses during training.
[0014] Furthermore, the class imbalance loss mentioned above is calculated using the CE loss classification loss function or the CB loss class imbalance loss function.
[0015] Furthermore, S2 specifically refers to:
[0016] S2.1 The dynamic clustering module uses the K-means unsupervised learning clustering algorithm or its variants to split the sample dataset of a few classes into SN subclass datasets;
[0017] S2.2. Use the extracted feature vector as input to the dynamic clustering module. If the sample data belongs to a minority class of sample datasets, calculate the distance between the sample data and the cluster center of the minority class of sample datasets.
[0018] The cluster centers are determined using the exponential moving average method, calculated as shown in the following formula:
[0019]
[0020] α represents the degree of weight decay, with a value range of (0,1) and an optimal value of 0.99; yt represents the average value of a certain category feature in the t-th batch; SCt represents the exponential moving average of the category features in the t-th batch of sample data during training, which is the cluster center of the sample dataset of that minority category.
[0021] The maximum distance between the sample data and the cluster centers is uniformly divided into SN intervals, each interval being a subclass dataset Pn, and the length of each interval is dt. The calculation formula is as follows:
[0022]
[0023] Where Si represents the i-th sample in this category; SN represents the number of subcategories divided into this category; |S i -SC t | represents the distance between the i-th sample data and the cluster centers of the minority class sample dataset;
[0024] S2.3. Assign the sample data to the nearest subclass dataset Pn. The specific algorithm is as follows:
[0025]
[0026] Where n∈[0, 1, ..., SN-1], that is, the distance between the sample data and the cluster centers of the minority class sample dataset is within (nd... t , (n+1)d t If the sample data falls within the range of ), then the sample data is assigned to the corresponding subclass dataset P. n ;
[0027] Then, pseudo-labels are assigned to all sample data.
[0028] Furthermore, the label mapping module learns and constructs a mapping relationship through a three-layer fully connected network model to achieve accurate conversion from subclass pseudo-labels to original category labels. Ultimately, the model can classify and predict unknown samples based on these mapped labels.
[0029] The beneficial effects of this invention are as follows: The data relabeling classification algorithm based on dynamic clustering learning closely integrates dynamic clustering with model training, adapting to changes in data distribution by updating cluster centers and pseudo-labels in real time, thereby improving the model's classification and recognition performance. Its advantages include: dynamic clustering can fully consider the real-time distribution of samples, avoiding the limitations caused by the unchanging cluster centers in static clustering; by dividing into subclasses, it can alleviate the imbalance problem between minority and majority classes; the introduction of pseudo-labels ensures a balanced improvement in recognition performance between subclasses and between subclasses and the majority class; and the class balance loss function further enhances the model's ability to recognize the majority class, avoiding the problem of reduced recognition accuracy for minority classes in traditional methods. Attached Figure Description
[0030] Figure 1 This is a flowchart of the technology of the present invention.
[0031] Figure 2 This is the overall algorithm framework for the training process of this invention.
[0032] Figure 3 This is a schematic diagram of a long-tail dataset. Detailed Implementation
[0033] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.
[0034] Example:
[0035] like Figure 1 and Figure 2 As shown: A data relabeling and classification algorithm based on clustering learning includes the following steps:
[0036] S1. After preprocessing the data in the original sample, feature vectors are extracted through the feature extraction module to form sample data. The data in the original sample is facial expression image data, or it can be other imbalanced datasets.
[0037] Its feature extraction module is implemented using the backbone network of CNN (Convolutional Neural Network), which includes, but is not limited to, the implementation of convolutional neural networks such as ResNet (Residual Network), VGG or MobileNet (Lightweight Neural Network). These network modules have good generalization ability in the field of image recognition.
[0038] S2. The sample dataset of the minority class is split into several subclass datasets by the dynamic clustering module, while the sample dataset of the majority class remains unchanged. In the imbalanced dataset, the dataset of the minority class with more samples is defined as the sample dataset of the minority class, while the dataset of the majority class with fewer samples is defined as the sample dataset of the majority class. The sample size of the majority class is much smaller than that of the majority class. The training dataset contains data and its corresponding labels.
[0039] The extracted feature vectors are used as input to the dynamic clustering module, which works in conjunction with subsequent modules to ensure the stability and efficiency of the entire system. During clustering, if the sample data belongs to a minority of sample datasets, the sample data is assigned to the nearest subclass dataset based on the distance between the sample data and the cluster center, and then pseudo-labels are assigned to all sample data.
[0040] A dynamic clustering strategy is introduced in the feature clustering and relabeling module, which iteratively optimizes the intra-class distance of samples in the feature space. Specifically, for each foreground category, the algorithm first calculates the similarity metric between samples, and then uses a clustering algorithm (such as K-means unsupervised learning clustering or its variants) to divide the samples into multiple subclasses. This process not only reveals the complex internal structure of the data, but also achieves detailed partitioning and relabeling of the original dataset by assigning unique pseudo-labels to each subclass sample. The innovation of this step lies in its dynamism, that is, the cluster centers and subclass partitioning are continuously optimized as the data is updated to adapt to changes in the data distribution.
[0041] The specific steps are as follows:
[0042] S2.1 The dynamic clustering module uses the K-means unsupervised learning clustering algorithm or its variants to split the sample dataset of a few classes into SN subclass datasets;
[0043] S2.2. The extracted feature vector is used as the input of the dynamic clustering module. If the sample data belongs to a minority class of sample datasets, the distance between the sample data and the cluster center of the minority class of sample datasets is calculated. In one epoch (the process of training once using all the samples in the training set), multiple batches of data will come in. Each time the data comes in, the subclass sample center, i.e. the cluster center, will be updated. It is not clustered in a fixed set of data, but new data will be added one by one.
[0044] The cluster centers are determined using the exponential moving average method to reduce computational costs and improve their stability. The calculation is shown in the following formula:
[0045]
[0046] α represents the degree of weight decay, with a value ranging from (0,1) and an optimal value of 0.99; y t SC represents the average value of a characteristic of a certain category in the t-th batch; t It represents the exponential moving average of the class features in the t-th batch of sample data during training, that is, the cluster center of the sample dataset of the minority class;
[0047] Divide the sample data and the maximum distance between the cluster centers into SN intervals, with each interval being a subclass dataset P. n And the length of each interval is d t The calculation formula is as follows:
[0048]
[0049] Among them, S i This represents the i-th sample in this category; SN represents the number of subcategories within this category, which is determined by the subcategory partitioning array. The subcategory number parameter is set according to the data distribution; |S i -SC t | represents the distance between the i-th sample data and the cluster centers of the minority class sample dataset;
[0050] S2.3. Assign the sample data to the nearest subclass dataset P. n P n The algorithm for representing the nth subclass dataset is as follows:
[0051]
[0052] Where n∈[0,1,...,SN-1], that is, if the distance between the i-th sample data and the cluster center of the minority class sample dataset is within (nd... t, (n+1)d t If the sample data falls within the range of ), then the sample data is assigned to the corresponding subclass dataset P. n Then, pseudo-labels are assigned to all sample data.
[0053] Its advantages lie in the fact that dynamic clustering can fully consider the real-time distribution of samples, avoiding the limitations caused by the invariant cluster centers in static clustering. By dividing into subclasses, the imbalance between the minority and majority classes can be alleviated to some extent; the introduction of pseudo-labels improves the recognition performance between subclasses and between subclasses and the majority classes in a balanced way; and the class balance loss function further enhances the model's ability to recognize the majority class, avoiding the problem of reduced recognition accuracy for the minority class in traditional methods.
[0054] S3. Pseudo-labels are used to train the subclasses. The label mapping module maps the predicted results of the subclass datasets to the true label space based on the mapping relationships learned during training. Pseudo-labels are used to calculate the loss during training, and a class balancing loss is introduced to further balance the weights of different classes, enabling the model to pay more attention to the majority class and the split subclasses during training.
[0055] The aforementioned class imbalance loss is calculated using either the CE loss (classification loss function) or the CB loss (class imbalance loss function). There are multiple options for this loss calculation, and the optimal loss needs to be selected through experimentation. During model training, the difference between the predicted value and the true value is calculated, and this difference is the loss value. The optimizer calculates the gradient based on this value and backpropagates to update the weights. After updating, the model is trained again, and this process is repeated until all the training data has been learned. This is one epoch process, and it generally takes about 200 epochs to learn.
[0056] The label mapping module learns and constructs a mapping relationship through a three-layer fully connected network model, achieving accurate conversion from subclass pseudo-labels to original category labels. Ultimately, the model can classify and predict unknown samples based on these mapped labels. The mapping relationship is learned by the fully connected model and is non-linear. The last layer of the feature extraction module outputs the prediction result, which is a set of probability results between 0 and 1. If batchsize = 32, and the subclass division array is [10, 8, 6, 5, 3, 2, 1], a total of 35 classes are divided. The shape and size of the output prediction result are [32, 35], representing the probability value of each class corresponding to a sample. The largest probability value is the predicted class. [0.12, 0.98, 0.23, ..., 0.1] indicates that the prediction is class 2. These predictions are all about which subclass it is, so they need to be mapped to the real labels through the mapping network.
[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.
Claims
1. A data relabeling and classification algorithm based on clustering learning, characterized in that, Includes the following steps: S1. After preprocessing the data in the original samples, feature vectors are extracted through the feature extraction module to form sample data; S2. The sample dataset of the minority class is split into several sub-class datasets by the dynamic clustering module, while the sample dataset of the majority class remains unchanged. The extracted feature vector is used as the input of the dynamic clustering module. If the sample data belongs to the sample dataset of the minority class, the sample data is assigned to the nearest sub-class dataset according to the distance between the sample data and the cluster center. Then, pseudo-labels are assigned to all sample data. S3. Use pseudo-labels to train the subclasses, and use the label mapping module to map the prediction results of the subclass dataset to which the subsequent sample data belongs back to the real label space based on the mapping relationship learned during the training process.
2. The data relabeling and classification algorithm based on clustering learning according to claim 1, characterized in that, The original sample data consists of facial expression images, and its feature extraction module is implemented using a CNN backbone network.
3. The data relabeling classification algorithm based on clustering learning according to claim 1 or 2, characterized in that, Its feature extraction module is implemented using convolutional neural networks such as ResNet, VGG, or MobileNet.
4. The data relabeling and classification algorithm based on clustering learning according to claim 1 or 2, characterized in that, During training, pseudo-labels are used to calculate the loss. Class-balanced loss is introduced to further balance the weights of different classes, enabling the model to pay more attention to the majority class and the split subclasses during training.
5. The data relabeling and classification algorithm based on clustering learning according to claim 4, characterized in that, The class imbalance loss mentioned above is calculated using the CE loss classification loss function or the CB Loss class imbalance loss function.
6. The data relabeling and classification algorithm based on clustering learning according to claim 1, 2, or 5, characterized in that, S2 specifically refers to: S2.1, the dynamic clustering module uses the K-means unsupervised learning clustering algorithm or its variants to split the sample dataset of a minority class into SN subclass datasets; S2.
2. Use the extracted feature vector as input to the dynamic clustering module. If the sample data belongs to a minority class of sample datasets, calculate the distance between the sample data and the cluster center of the minority class of sample datasets. The cluster centers are determined using the exponential moving average method, calculated as shown in the following formula: α represents the degree of weight decay, and the value of α ranges from (0,1). yt represents the average value of a certain category feature in the t-th batch, and SCt represents the exponential moving average of the category features in the t-th batch of sample data during training. The maximum distance between the sample data and the cluster centers is uniformly divided into SN intervals, each interval being a subclass dataset Pn, and the length of each interval is dt. The calculation formula is as follows: Where Si represents the i-th sample in this category; SN represents the number of subcategories divided into this category; |S i -SC t | represents the distance between the i-th sample data and the cluster centers of the minority class sample dataset; S2.
3. Assign the sample data to the nearest subclass dataset Pn. The specific algorithm is as follows: Where n∈[0, 1, ..., SN-1], that is, the distance between the sample data and the cluster centers of the minority class sample dataset is within (nd... t , (n+1)d t If the sample data falls within the range of ), then the sample data is assigned to the corresponding subclass dataset P. n ; Then, pseudo-labels are assigned to all sample data.
7. The data relabeling classification algorithm based on clustering learning according to claim 1, 2, or 5, characterized in that, The label mapping module learns and constructs a mapping relationship through a three-layer fully connected network model, realizing the accurate conversion from subclass pseudo-labels to original category labels. Finally, the model can classify and predict unknown samples based on these mapped labels.