Hybrid Text Classification Model for Imbalanced Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based text classification models require large amounts of labeled data for accurate training, making them inefficient for applications with limited data, while matching-network-based models provide less reliable results with fewer samples, especially when training data is imbalanced.
Innovation Solution
A method that combines deep-learning-based and matching-network-based text classification models by generating text embedding vectors and prototype vectors to estimate probabilities, using weighting values to combine outputs and normalize results for accurate classification across few-shot and many-shot classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based text classification models are used, then classification accuracy is improved, but large amounts of labeled training data are required
Solution Approach 1:
The patent combines two different classification approaches: a deep learning-based classifier (CNN or LSTM) and a matching-network-based classifier. The deep learning model processes the text embedding to generate first classification outputs, while the matching network compares the text embedding against prototype vectors to generate second classification outputs. These two outputs are then merged through weighted averaging to produce the final classification result, thereby achieving high accuracy without requiring large amounts of training data for both models simultaneously.
Solution Approach 2:
The patent introduces text embedding vectors as an intermediary representation that bridges the input text and the classification models. The embedding layer transforms the input text into a dense vector representation that captures semantic meaning. This embedding serves as the input to both the deep learning classifier and the matching network, allowing both models to work with the same compressed representation and reducing the overall data requirements.
2Quantity of substance
If matching-network-based classification models are used with fewer samples, then training data requirements are reduced, but classification reliability deteriorates
Solution Approach 1:
The patent merges the outputs of two classification systems: the matching network (which can work with fewer samples) and the deep learning classifier (which provides higher reliability). The final classification probability is computed as a weighted average of the first output vector from the deep learning model and the second output vector from the matching network. This combination allows the system to leverage the data efficiency of matching networks while maintaining the reliability of deep learning through the integrated approach.
3Quantity of substance
If manually generated training data is used, then data availability is improved, but time consumption increases
Solution Approach 1:
The patent enables the classification system to serve itself by using a small amount of labeled data to train the embedding layer and prototype vectors, then automatically classifying new texts without requiring manual labeling. The system self-adapts to new domains and languages by learning embeddings from the available data, eliminating the need for time-consuming manual annotation of large datasets.
Data Source
AI summary
A method of text classification includes generating a text embedding vector representing a text sample and applying weights of a regression layer to the text embedding vector to generate a first data model output vector. The method also includes generating a plurality of prototype embedding vectors associated with a respective classification labels and comparing the plurality of prototype embedding vectors to the text embedding vector to generate a second data model output vector. The method further includes assigning a particular classification label to the text sample based on the first data model output vector, the second data model output vector, and one or more weighting values.