Neural Network for Sparsely Labeled Text Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text classification methods require a high percentage of labeled training data, leading to significant manual labor and inefficiencies in understanding and classifying documents, especially when only a fraction of the data is labeled.
Innovation Solution
A neural network architecture that processes sparsely labeled text data by extracting local n-gram features, global characteristics, and sequential dependencies using convolutional layers and a long short-term memory layer, with forward-oriented dynamic routing and virtual adversarial training to preserve semantics and prevent overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional text classification methods are used, then classification accuracy can be achieved, but a high percentage of labeled training data is required leading to significant manual labor
Solution Approach 1:
The system performs preliminary classification actions using a small set of labeled data to establish initial classification rules, then applies these rules to automatically classify the majority of unlabeled data, reducing the need for extensive manual labeling while maintaining classification accuracy
Solution Approach 2:
The text classification system enables itself to perform classification tasks automatically by learning from sparse labeled data and applying the learned patterns to unlabeled data, eliminating the need for continuous manual intervention and labeling
2Measurement precision
If more labeled training data is used, then classification performance improves, but the complexity and cost of data preparation increases
Solution Approach 1:
The system applies partial action by using only a small fraction of data labeling (sparse labeling) rather than requiring complete labeling of all training data, achieving satisfactory classification performance with reduced data preparation effort and complexity
Solution Approach 2:
The classification system is designed to handle both labeled and unlabeled data universally, allowing it to learn from the small labeled portion and automatically apply learned patterns to the larger unlabeled portion, reducing overall system complexity
3Loss of information
If conventional text classification is applied, then document understanding can be achieved, but the system fails to preserve semantic sequences and local features
Solution Approach 1:
The system segments text into local n-gram features and processes them through capsule networks that preserve sequential relationships, allowing semantic information to be maintained while breaking down the complex task of text understanding into manageable local feature extractions
Solution Approach 2:
The system transitions from conventional flat text processing to a multi-dimensional approach using capsule networks with dynamic routing, preserving semantic sequences by processing text in multiple hierarchical dimensions rather than a single linear pass
Data Source
AI summary
A method of training a neural network includes receiving a text corpus containing a labeled portion and an unlabeled portion, extracting local n-gram features and a sequence of the local n-gram features from the text corpus, processing the text corpus, using convolutional layers, according to the local n-gram features to determine capsule parameters of capsules configured to preserve the sequence of the local n-gram features, performing a forward-oriented dynamic routing between the capsules using the capsule parameters to extract global characteristics of the text corpus, and processing the text corpus according to the global characteristics using a long short-term memory layer to extract global sequential text dependencies from the text corpus, wherein parameters of the neural network are updated according to the local n-gram features, the capsule parameters, global characteristics, and global sequential text dependencies.