Supervised Feature Extraction for Text Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text classification methods, such as latent semantic analysis and latent Dirichlet allocation, are inflexible and inefficient, struggling to accurately classify documents due to limited keyword coverage and reliance on large training datasets, making them suboptimal for specific topics like medical reports on lung diseases.
Innovation Solution
A method for determining supervised features in text classification using a classifier hyperplane in a multidimensional space, where each word's vector is trained with ground truth labels to generate accurate and flexible classifiers, allowing for iterative refinement and expansion of keywords for improved classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual keyword listing is used for text classification, then the method is simple and easy to implement, but the keyword coverage is limited and classification accuracy is sub-optimal
Solution Approach 1:
The system automatically generates keywords and features through unsupervised topic modeling and supervised feature determination, eliminating the need for manual keyword listing while improving classification accuracy through data-driven feature extraction
Solution Approach 2:
The patent replaces the mechanical manual keyword listing process with automated computational methods including topic modeling algorithms and supervised learning, substituting human effort with machine-based feature extraction
2Adaptability or versatility
If unsupervised topic models are used to construct topics from text corpora, then the model can capture polysemy and produce semantically coherent word groups, but the topics are relatively inflexible and may not be of sufficient relevance to a desired topic
Solution Approach 1:
The system transitions from static unsupervised topic models to dynamic supervised feature determination, allowing the feature extraction process to adapt to specific classification tasks through training data while maintaining the semantic coherence benefits of unsupervised modeling
Solution Approach 2:
The patent introduces supervised features as an intermediary layer between unsupervised topic models and final classification, using training data to bridge the gap between general semantic coherence and task-specific relevance
3Measurement precision
If the granularity of topics in the model is increased to improve classification accuracy, then the quality of individual topics improves, but the total number of topics increases which is inefficient
Solution Approach 1:
The patent extracts only the most relevant features for each class from the vocabulary, rather than creating topics for all possible granularities, thereby achieving high classification accuracy without proportionally increasing the number of topics or computational complexity
4Measurement precision
If large training datasets are used to train topic models to generate accurate topics, then the classification accuracy improves, but the data requirement is necessarily large which is inefficient
Solution Approach 1:
The system determines supervised features for each word in the vocabulary rather than requiring large amounts of training data, using a more targeted approach that achieves accurate classification with less data by focusing on feature determination at the word level
Data Source
Figure 1
Figure 2
Figure 3~4c
AI summary
A method of determining a supervised feature, for use in text classification, for each word of a vocabulary, is disclosed. The vocabulary comprises a plurality of words each associated with a word embedding comprising a vector representing the word in a multidimensional space. The method comprises obtaining training data comprising one or more input words, one or more word embeddings each associated with a respective one of the input words, and one or more ground truth labels, each associated with a respective one of the input words, and each indicating an association of the respective input word with a given class. The method comprises training, based on the training data, a classifier model thereby to generate a classifier hyperplane in the multidimensional space, and determining, based on the classifier hyperplane, for each word of the vocabulary, a supervised feature for use in text classification. An apparatus is also disclosed.