Multi-Class Classification Using Hierarchical Binary Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-class classification models face challenges with imbalanced training data sets, where a majority class dominates, leading to inadequate training for non-majority classes, resulting in reduced accuracy and increased resource intensity.
Innovation Solution
The approach transforms an original multi-class training data set into coarse-level and fine-level adapted training data sets, using a binary classification model to improve classification accuracy by separating the majority class from non-majority classes, and further refining predictions through a multi-level binary classification process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single multi-class classification model is used to handle all classes, then the model structure is simple, but the classification accuracy for non-majority classes deteriorates due to data imbalance
Solution Approach 1:
The patent divides the multi-class classification problem into multiple binary classification tasks organized in a hierarchical tree structure. Each binary classifier handles a specific subset of classes, allowing the model to focus on distinguishing between two classes at each node rather than all classes simultaneously. This segmentation improves accuracy for non-majority classes by dedicating specific binary classifiers to handle them, while maintaining overall system manageability through the hierarchical structure.
2Measurement precision
If multiple binary classification models are used to improve accuracy for non-majority classes, then classification accuracy improves, but the system complexity and resource consumption increase
Solution Approach 1:
The patent implements a hierarchical tree structure where binary classifiers are nested in a parent-child relationship. Each binary classifier's output feeds into the next level of the hierarchy, creating a nested evaluation process. This nesting allows the system to use multiple binary classifiers without proportionally increasing complexity, as classifiers are organized in a structured manner where lower-level classifiers only process cases that pass through higher-level filters. The tree structure enables efficient pruning of evaluation paths, reducing the average computational burden despite having multiple classifiers.
3Productivity
If traditional multi-class classification is used with imbalanced data, then training is faster, but the reliability of predictions for minority classes deteriorates
Solution Approach 1:
By segmenting the multi-class problem into binary classification tasks, each binary classifier receives a more balanced dataset focused on distinguishing between two specific classes rather than handling all classes with highly imbalanced distributions. This segmentation allows each binary classifier to achieve better reliability for its specific task, while the overall system maintains good training efficiency through the modular nature of binary classifiers and the ability to train them independently or in a staged manner.
Data Source
AI summary
A multi-class classification system performs multi-class classification using an adapted, multi-level binary classification model in conjunction with pre-classification and post-classification processing. Original, multi-class training data is transformed into coarse-level adapted training data and fine-level adapted training data. In the coarse-level adapted training data, objects from the original training set are classified into either a majority classification set (representing the largest original classification set) or a non-majority classification set representing a combination of the remaining classification sets from the original training data. In the fine-level adapted training data, objects from the non-majority classification set retain their original classifications from the original training data. The adapted training data is used to train coarse-level and fine-level instances of the binary classification model, which are used to generate coarse-level and fine-level binary classifications of objects with respect to an input data set. The binary classifications are transformed into a multi-class classification.