Text Categorization Using Bidirectional LSTM and CNN
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural language processing techniques face challenges in efficiently categorizing large volumes of textual data due to the subjective nature of human communication and the inefficiency of human-implemented categorization, often resulting in reduced accuracy and increased processing time, especially when not utilizing domain-specific corpora.
Innovation Solution
The implementation of a method using bidirectional long short-term memory (LSTM) techniques and convolutional neural networks (CNNs) for text classification, combined with a nonparametric paired comparison, such as the Wilcoxon Signed Rank test, to improve classification accuracy and efficiency by determining context relationships and groupings of attributes within a domain-specific corpus.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If human-implemented categorization is used, then subjective interpretation is possible, but processing speed and efficiency are reduced
Solution Approach 1:
The patent replaces human mechanical categorization with an automated neural network system that processes text through multiple layers (embedding layer, convolutional layers, pooling layers, fully connected layers) to automatically classify text without human intervention, thereby maintaining accuracy while dramatically improving processing speed
Solution Approach 2:
The neural network system performs self-learning and self-categorization through training on labeled data, automatically adjusting its internal parameters and weights to improve classification accuracy over time without requiring continuous human guidance or intervention
2Productivity
If traditional text classification methods are used, then processing time is reduced, but classification accuracy and recall are lowered
Solution Approach 1:
The patent segments the text classification process into distinct functional layers: embedding layer for token representation, convolutional layers for local pattern detection, pooling layers for feature aggregation, and fully connected layers for final classification. This segmentation allows each layer to specialize in specific tasks, improving overall accuracy while maintaining efficient processing
Solution Approach 2:
The patent transforms the classification problem by adding dimensional depth through multiple neural network layers. The embedding layer converts tokens into high-dimensional vectors, and subsequent layers process these vectors through multiple transformation stages, effectively solving the classification problem in a higher-dimensional space that captures complex relationships
3Productivity
If domain-specific corpora are not utilized, then processing time is reduced, but classification accuracy deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-training the neural network on domain-specific corpora before deploying it for actual classification tasks. This pre-training phase allows the model to learn domain-specific vocabulary, patterns, and relationships in advance, so that during actual processing it can achieve high accuracy without requiring extensive processing time during deployment
Data Source
AI summary
A method performed by a device may include identifying a plurality of samples of textual content; performing tokenization of the plurality of samples to generate a respective plurality of tokenized samples; performing embedding of the plurality of tokenized samples to generate a sample matrix; determining groupings of attributes of the sample matrix using a convolutional neural network; determining context relationships between the groupings of attributes using a bidirectional long short term memory (LSTM) technique; selecting predicted labels for the plurality of samples using a model, wherein the model selects, for a particular sample of the plurality of samples, a predicted label of the predicted labels from a plurality of labels based on respective scores of the particular sample with regard to the plurality of labels and based on a nonparametric paired comparison of the respective scores; and providing information identifying the predicted labels.