Entropy-Based Synthetic Data Generation for Imbalanced Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data classification systems face challenges in accurately classifying imbalanced data sets, where one class significantly outnumber the other, leading to poor performance due to the lack of representative minority class instances in synthetic training data.
Innovation Solution
An entropy-based data augmentation system is introduced, which selects training instances from both minority and majority classes based on their entropy values to generate synthetic training instances, thereby creating an augmented training data set that re-trains the classification system, ensuring better representation and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional synthetic data generation methods are used, then training data volume increases, but minority class representation remains insufficient
Solution Approach 1:
The patent applies local quality by differentiating the treatment of minority and majority class instances through entropy calculation. Synthetic instances are generated selectively based on local entropy characteristics of minority class regions, ensuring that data augmentation focuses precisely where representation is insufficient rather than applying uniform augmentation across all classes.
Solution Approach 2:
The patent changes parameters by introducing entropy as a selection criterion and using interpolation between high-entropy minority instances and low-entropy majority instances. This parameter-based approach transforms the synthetic data generation process from random or uniform sampling to an entropy-driven selective generation method that preserves minority class characteristics.
2Quantity of substance
If more synthetic training data is generated, then training data set size increases, but classification accuracy deteriorates due to remaining imbalance
Solution Approach 1:
The patent implements feedback by using entropy calculations to evaluate and select training instances for synthetic data generation. The entropy metric provides feedback on instance quality and representativeness, allowing the system to iteratively refine which instances should be used as parents for synthetic generation, thereby improving classification accuracy while maintaining data set size.
Solution Approach 2:
The patent applies preliminary action by pre-selecting high-entropy minority class instances and low-entropy majority class instances before synthetic data generation. This preliminary selection based on entropy ensures that the synthetic data will be generated from representative parent instances, preventing the propagation of imbalance and preserving classification accuracy from the outset.
3Ease of manufacture
If synthetic instances are generated without entropy-based selection, then data generation is simpler, but training instance quality decreases
Solution Approach 1:
The patent applies self-service by enabling the training data set to evaluate and select its own high-quality instances through entropy calculation. The entropy metric allows the data set to self-identify which instances are most representative and informative, automatically guiding the synthetic data generation process without requiring external manual selection or complex preprocessing.
Data Source
AI summary
A data classification system is trained to classify input data into multiple classes. The system is initially trained by adjusting weights within the system based on a set of training data that includes multiple tuples, each being a training instance and corresponding training label. Two training instances, one from a minority class and one from a majority class, are selected from the set of training data based on entropies for the training instances. A synthetic training instance is generated by combining the two selected training instances and a corresponding training label is generated. A tuple including the synthetic training instance and the synthetic training label is added to the set of training data, resulting in an augmented training data set. One or more such synthetic training instances can be added to the augmented training data set and the system is then re-trained on the augmented training data set.