Regularized Deep Embedded Clustering for Imbalanced Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep embedded clustering (DEC) techniques face challenges in accurately classifying clusters with a small number of samples and suffer from performance deterioration with imbalanced datasets, where the number of data points in each class is unequal.
Innovation Solution
The integration of data augmentation technology, specifically virtual adversarial training (VAT), with the DEC technique, which involves perturbing the target data to enhance clustering accuracy by regularizing the neural network and pushing similar data points closer together in the latent space, resulting in the regularized deep embedded clustering (RDEC) method.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional deep embedded clustering (DEC) technique is used, then clustering can be performed using deep neural networks, but classification accuracy deteriorates for clusters with small number of samples and imbalanced datasets
Solution Approach 1:
The patent applies data augmentation through virtual adversarial training (VAT) as a preliminary action before clustering. By generating adversarial examples and augmenting the training data beforehand, the method prepares more robust training samples that help the neural network learn better cluster representations, particularly for underrepresented classes in imbalanced datasets.
Solution Approach 2:
The patent modifies the training process by introducing VAT-based data augmentation that changes the distribution of training samples. This parameter change in the data distribution allows the model to learn more invariant features and improve generalization to imbalanced datasets without changing the fundamental DEC architecture.
2Measurement precision
If data augmentation through virtual adversarial training (VAT) is integrated with DEC, then clustering accuracy improves particularly for imbalanced datasets, but computational complexity increases
Solution Approach 1:
The patent uses virtual adversarial training to generate synthetic adversarial copies of training samples. These copied adversarial examples are then used to augment the training dataset, allowing the model to learn from additional virtual samples without requiring more physical training data, thus improving accuracy while managing computational resources.
3Ease of manufacture
If conventional DEC technique is used, then the method is relatively simple to implement, but centroids positioning is unstable leading to poor clustering performance
Solution Approach 1:
The patent incorporates virtual adversarial training that provides feedback loops during the training process. By iteratively generating adversarial examples and retraining the model, the system receives feedback that stabilizes centroid positioning and improves clustering consistency without requiring complex manual intervention.
Data Source
AI summary
According to one embodiment, a communication device includes one or more processors. The processors decide on a first-type parameter representing a conversion operation for converting input data into first-type conversion data. The processors calculate first-type predicted distributions based on second-type parameters, each representing one of the plurality of clusters. The processors update the first-type parameter and the second-type parameters so as to achieve optimization of first-type differences representing differences between the first-type predicted distributions and a target distribution, and second-type differences representing differences between the first-type predicted distributions and second-type predicted distributions that indicate probability at which second-type converted data. The second-type converted data is obtained by converting data, which is formed by augmentation of the input data, using the first-type parameter belongs to the clusters. The processors classify the input data into the plurality of clusters based on the first-type predicted distributions calculated using the updated second-type parameters.