Hierarchical Text Classification Using N-gram Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text classification techniques face challenges in classifying text into fine-grained target categories with high precision and recall, leading to less relevant content recommendations for users.
Innovation Solution
The method involves classifying text into hierarchical categories using a combination of text classifiers, filtering categories based on thresholds of relatedness, and calculating weights using tf-idf values and n-gram extraction to select relevant categories for content recommendation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If text classification uses traditional techniques (naive Bayes, tf-idf, SVM), then the classification process is simple and fast, but the precision and recall for fine-grained categories are insufficient
Solution Approach 1:
The patent segments the classification process into multiple stages: initial classification using traditional techniques, followed by refinement through hierarchical category analysis and n-gram based scoring. This segmentation allows the system to maintain simplicity in the initial filtering stage while achieving high precision in the final classification through progressive refinement.
Solution Approach 2:
The patent introduces an intermediary refinement stage that sits between traditional classification and final category assignment. This intermediary process uses n-gram extraction, hierarchical relationship analysis, and weighted scoring to bridge the gap between simple traditional classification and the need for high-precision fine-grained categorization.
2Adaptability or versatility
If text classification aims for fine-grained categories, then content recommendations become more relevant, but the classification accuracy and recall rate decrease
Solution Approach 1:
The patent performs preliminary classification using traditional techniques to identify candidate categories before applying the more sophisticated n-gram based refinement process. This preliminary action narrows down the search space and ensures that only relevant fine-grained categories are considered, maintaining both granularity and reliability.
Solution Approach 2:
The patent adds a new dimension to classification by incorporating hierarchical category relationships and n-gram based semantic analysis. This additional dimension allows the system to achieve fine-grained categorization while maintaining reliability through multi-factor scoring that considers both traditional classification results and n-gram matching quality.
3Measurement precision
If the system uses multiple classification techniques and thresholds, then classification accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent applies partial action by using traditional classification techniques for initial filtering to identify candidate categories, then applying the more computationally intensive n-gram analysis only to these candidates rather than all possible categories. This selective application maintains high accuracy while reducing overall processing time.
Solution Approach 2:
The classification process operates in periodic stages: first traditional classification, then n-gram extraction and scoring, followed by threshold-based filtering and final category selection. This periodic multi-stage approach allows the system to balance accuracy and processing time by breaking down the complex classification task into manageable sequential steps.
Data Source
AI summary
Systems, methods and program products for classifying text. A system classifies text into first subject matter categories. The system identifies one or more second subject matter categories in a collection of second subject matter categories, each of the second categories is a hierarchical classification of a collection of confirmed valid search results for queries, in which at least one query for each identified second category includes a term in the text. The system filters the identified categories by excluding identified categories whose ancestors are not among the first categories. The system selects categories from the filtered categories based on one or more thresholds in which a threshold specifies a degree of relatedness between a selected category and the text. The selected categories are a sufficient basis for recommending content to a user, the content being associated with one or more of the selected categories.