Document Image Classification via HMM Log-Likelihood Binning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for classifying data with highly imbalanced class distributions, where one class has significantly more samples than others, often result in poor classification accuracy due to the dominance of the majority class.
Innovation Solution
The use of a Hidden Markov Model (HMM) to generate log-likelihood scores for data instances, followed by ranking and binning these scores to apply a one-sided sampling technique, which prunes redundant and borderline examples, thereby creating a balanced training dataset for machine learning classifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional sampling methods or oversampling techniques are used to handle imbalanced data, then the majority class representation is improved, but classification accuracy for the minority class deteriorates
Solution Approach 1:
The patent transforms the imbalanced dataset into a balanced one by applying parameter changes to the sample distribution. Specifically, it uses HMM log-likelihood scores to identify and remove borderline examples from the majority class, thereby changing the class distribution parameters from imbalanced to balanced without simply adding or copying samples
Solution Approach 2:
The patent extracts and removes redundant and borderline examples from the majority class based on HMM log-likelihood scoring. By taking out these low-quality samples that contribute to the imbalance problem, the method creates a balanced training set that improves minority class classification accuracy without introducing synthetic samples
2Quantity of substance
If the training dataset includes all available data samples, then the model learns from maximum data, but the imbalanced class distribution causes poor performance on minority classes
Solution Approach 1:
The patent changes the distribution parameter of the training data by removing borderline examples from the majority class. This parameter change transforms the imbalanced dataset into a balanced one, allowing the model to learn from representative samples of both classes with improved prediction reliability for minority classes
Solution Approach 2:
The patent performs preliminary action by pre-processing the training data to remove redundant and borderline examples before model training. By applying HMM-based scoring and removal in advance, it prepares a balanced dataset that ensures reliable predictions for minority classes from the outset
Data Source
AI summary
A hybrid Hidden Markov Model (HMM) and Machine Learning (ML) systems and apparatus for classification in the case of data instances with imbalanced class distribution, including a Hidden Markov Model for generating a log-likelihood score for each data instance. Implementations of the hybrid system and method detect fraudulent activity and classifies documents with accuracy that surpasses conventional classifiers. In one implementation, Hidden Markov Model (HMM) for generating a log-likelihood score based on an attribute value vector for a set of keyword features characterizing a Web page. In one implementation, the HMM generates a log-likelihood score based on an attribute value vector for page layout characterizing a document image. Resulting attribute value vectors are ranked and divided into bins grouped by log-likelihood scores within equal ranges. Various machine learning models are trained using the balanced vectors obtained by accumulating from all the bins of vectors.