Text Classification Using Dual LSTM Vector Combination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text classification systems using neural networks like LSTM units face performance and generalization issues due to unsupervised training methods that ignore category comparisons and do not provide accurate classification results.
Innovation Solution
A method and system that generate first and second vector representations of text using trained LSTM units, combining them with a selected combiner operator based on classifier accuracy, to enhance classification performance and generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised training methods (such as tf-idf approach) are used to derive n-dimensional representation, then the training process is simpler, but the classification accuracy and generalization performance deteriorate because category comparisons are ignored
Solution Approach 1:
The patent applies preliminary action by pre-training two separate LSTM units before combining their outputs. The first LSTM unit is trained to capture semantic similarities, while the second LSTM unit is trained to capture category-specific features. This preliminary training of multiple specialized models before combination resolves the contradiction by enabling category comparisons to be made during the training phase, thereby improving classification accuracy without requiring complex supervised training of a single model.
Solution Approach 2:
The patent employs dimensionality change by transitioning from a single n-dimensional vector representation to a combined representation that integrates outputs from two separate LSTM units. The first LSTM produces a semantic vector and the second LSTM produces a category-specific vector, which are then combined through element-wise multiplication. This dimensional expansion and integration allows the system to capture both semantic meaning and category information, thereby improving classification accuracy while maintaining training simplicity through unsupervised contrastive loss.
2Device complexity
If single LSTM unit with unsupervised training is used, then device complexity is reduced, but classification performance and generalization ability worsen due to lack of category comparison
Solution Approach 1:
The patent applies segmentation by dividing the text classification task into two separate processing streams using two distinct LSTM units. The first LSTM unit focuses on semantic representation while the second LSTM unit focuses on category-specific features. Each unit is trained independently with simpler unsupervised contrastive loss, avoiding the complexity of training a single complex supervised model. The segmented approach improves reliability by ensuring both semantic and category information are captured, while keeping individual model components relatively simple.
Solution Approach 2:
The patent employs merging by combining the outputs of two separately trained LSTM units through element-wise multiplication. The first LSTM output (semantic vector) and the second LSTM output (category-specific vector) are merged to create a final combined representation that is fed to the classifier. This merging strategy improves classification performance and generalization by integrating multiple types of information, while the overall device complexity remains manageable because each individual LSTM unit is relatively simple and trained using unsupervised methods.
3Ease of operation
If existing neural networks are used for text-to-vector conversion, then implementation is straightforward, but generalization accuracy deteriorates due to ignoring category differences in training
Solution Approach 1:
The patent applies preliminary action by performing unsupervised contrastive training of two LSTM units before the actual classification task. During this preliminary phase, the model learns to distinguish between similar and dissimilar text pairs without requiring labeled category information. This preliminary training enables the model to develop good feature representations that capture both semantic meaning and category-specific patterns, thereby improving generalization accuracy while keeping the implementation straightforward through the use of standard LSTM architectures and simple contrastive loss functions.
Data Source
AI summary
Embodiments of the present disclosure discloses method and system for determining classification of text. The present disclosure discloses to receive text from plurality of texts and generating a pair of vector representation of the text using trained model parameters of a pair of LSTM units. The trained model parameters are obtained based on training of classification system using plurality of similar pair of texts and plurality of dissimilar pair of texts from the plurality of texts. Further, pair of vector representations are combined using a combiner operator to obtain a combined vector representation. The combiner operator is selected from a plurality of combiner operators based on the training using accuracy of classifier of classification system. The combined vector representation is provided to the classifier for determining classification of text. The present disclosure enhances the performance and generalisation of a classifier in cases of a multi-class classification.