Boosting Feature Selection for Document Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In document classification, especially when the proportion of different types of documents varies greatly, relying solely on features with the highest document frequency can overlook features that appear in fewer documents but hold important information about non-dominant patterns, leading to inadequate classification performance.
Innovation Solution
Combining features determined by a document frequency process with those obtained through a boosting process that uses an approximation for one-dimensional optimization with an upper bound, creating a combined feature set that includes both high-frequency and low-frequency features correlated with specific categories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If features are selected based on highest document frequency, then classification speed and simplicity are improved, but classification accuracy deteriorates due to overlooking non-dominant patterns
Solution Approach 1:
The patent merges two feature selection approaches: document frequency-based selection (for speed and simplicity) and boosting-based selection (for accuracy and non-dominant pattern detection). The combined feature set integrates both high-frequency and low-frequency features, allowing the system to benefit from both approaches simultaneously - maintaining efficiency while improving accuracy through the inclusion of boosting-selected features that capture non-dominant patterns.
2Device complexity
If only high-frequency features are used, then feature selection simplicity is improved, but information about non-dominant patterns is lost
Solution Approach 1:
The patent segments feature selection into two distinct processes: (1) document frequency-based feature identification, which captures high-frequency dominant patterns, and (2) boosting-based feature identification, which captures low-frequency non-dominant patterns. This segmentation allows each process to specialize in different aspects, with the document frequency process handling simplicity and the boosting process handling information completeness, thereby reducing overall complexity while preventing information loss.
Solution Approach 2:
The patent combines the results of both feature selection processes into a unified feature set. By merging the high-frequency features from document frequency analysis with the low-frequency features from boosting analysis, the system recovers information about non-dominant patterns that would otherwise be lost, while maintaining the simplicity of the overall feature selection approach through the integration of both methods.
3Reliability
If boosting process is applied to select features, then classification accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies boosting as a partial feature selection mechanism rather than using it exclusively. By combining boosting with document frequency-based selection, the system uses boosting only where needed - to identify non-dominant patterns that document frequency alone would miss - while relying on the simpler document frequency method for the majority of feature selection. This partial application of boosting reduces computational complexity compared to using boosting alone, while still achieving improved accuracy through the targeted identification of important non-dominant features.
Data Source
AI summary
Determining indicative features may be provided. First, a first set of features may be determined using a document frequency process. Then a second set of features may be determined using a boosting process. Using the boosting process may comprise using an approximation for a one-dimensional optimization. The approximation may include an upper bound. Next, the first set of features and the second set of features may be combined into a combined set of features. The combined set of features may comprise a union of the first set of features and the second set of features. At least one document may then be classified based on the combined set of features.