Large-Scale Text Classification Using Explicit Semantic Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large-scale classification methods, such as Rocchio classification, face challenges in scalability and performance due to memory requirements and robustness issues when dealing with tens or hundreds of thousands of classes, leading to slow model load and scoring times, and poor prediction accuracy.
Innovation Solution
A system and method for large-scale classification that represents training data items in a conceptual space using explicit semantic analysis, creating a sparse matrix with class vectors and attribute weights, allowing for efficient classification by computing similarity measures and selecting relevant classes based on a threshold, while also implementing text summarization using word embeddings to capture semantic relatedness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Rocchio classification is used for large-scale classification, then classification capability is provided, but memory requirements increase and scoring times slow down when dealing with tens or hundreds of thousands of classes
Solution Approach 1:
The patent segments the classification process into two distinct phases: an offline training phase where class centroids are pre-computed and stored, and an online scoring phase where only similarity computations are performed. This segmentation allows the system to handle tens or hundreds of thousands of classes efficiently by moving the computationally intensive centroid computation to offline, thereby reducing online scoring time while maintaining the ability to scale to large numbers of classes.
Solution Approach 2:
The patent performs preliminary computation of class centroids during the offline training phase, storing them for reuse during online scoring. This preliminary action eliminates the need to recompute centroids during each scoring operation, significantly reducing scoring time while enabling the system to scale to large numbers of classes. The pre-computed centroids are stored in memory, allowing rapid similarity computations during online phase.
2Quantity of substance
If Rocchio classification is used for large-scale classification, then classification capability is provided, but prediction accuracy deteriorates due to robustness issues
Solution Approach 1:
The patent implements a feedback mechanism where the system iteratively refines class centroids by incorporating feedback from the training data. During the offline training phase, the system computes centroids based on training instances, then uses these centroids to classify training data, and iteratively updates the centroids based on the classification results. This feedback loop improves prediction accuracy and robustness while maintaining the ability to handle large numbers of classes.
3Reliability
If explicit semantic analysis with conceptual space representation is implemented, then prediction accuracy and memory efficiency are improved, but computational complexity increases during model building
Solution Approach 1:
The patent segments the computational workload by performing complex semantic analysis and centroid computation during offline training, then using only simple similarity computations during online scoring. This segmentation reduces the apparent computational complexity during deployment while maintaining high prediction accuracy through the use of explicit semantic analysis and conceptual space representation with weighted vectors.
4Reliability
If weighted vectors with attribute weights aggregated from training data items are used, then classification performance is improved, but memory storage requirements increase
Solution Approach 1:
The patent changes the parameter representation by using weighted vectors where weights are aggregated from training data items. This parameter transformation allows the system to capture the distribution and characteristics of training data in a compact form. The weighted vectors store aggregated statistics rather than individual training instances, improving classification performance while managing memory storage requirements through efficient aggregation.
Data Source
AI summary
Approaches for large-scale classification and text summarization. In one embodiment, for example, the approach for large-scale classification includes predicting relevant classes of a new unseen case based on a classification model that is learned from a given knowledge base comprising labeled training data items. In another embodiment, for example, the approach for text summarization includes repurposing explicit semantic analysis (ESA) techniques for computing a text summary of a given text document.