Deep Skip-Gram Network for Text Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text classification methods fail to effectively capture long-range or abstract-level features within texts due to the varying expressions of human language, limiting their ability to accurately classify texts.
Innovation Solution
The integration of a deep skip-gram network architecture that combines skip-gram convolution with recurrent neural networks (RNNs) to generate non-consecutive n-gram sequences, detect local patterns, and capture long-range features through a chain-like architecture, utilizing max-overtime-pooling to reduce redundancy and extract important features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing text classification methods are used, then the classification process is simple, but the ability to capture long-range or abstract-level features is insufficient
Solution Approach 1:
The patent segments text processing into multiple parallel convolutional branches, each detecting different n-gram features (bi-gram, tri-gram, etc.). This segmentation allows the system to capture various local patterns simultaneously while maintaining manageable complexity through modular architecture design
Solution Approach 2:
The patent transitions from traditional sequential text processing to a multi-dimensional feature space by extracting n-gram features at different levels and combining them through max-pooling operations. This dimensional expansion enables capture of both local and long-range features simultaneously
2Measurement precision
If deep skip-gram network architecture is used, then classification accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent merges multiple convolutional feature detectors into a unified deep skip-gram network architecture. By combining bi-gram, tri-gram, and higher-order n-gram convolutions in parallel and merging their outputs through max-pooling, the system achieves comprehensive feature capture while optimizing computational efficiency through shared weight matrices
Solution Approach 2:
The patent performs preliminary feature extraction through multiple convolutional layers that detect n-gram patterns before final classification. This preliminary action of extracting and pooling n-gram features prepares the data in advance, reducing the computational burden on subsequent classification layers
Data Source
AI summary
Described herein are embodiments for systems and methods to incorporate skip-gram convolution to extract non-consecutive local n-gram patterns for comprehensive information for varying text expressions. In one or more embodiments, one or more recurrent neural networks are employed to extract long-range features from localized level to sequential and global level via a chain-like architecture. Comprehensive experiments on large-scale datasets widely used for the text classification task were conducted to demonstrate the effectiveness of the presented deep skip-gram network embodiments. Performance evaluation on various datasets demonstrates that embodiments of the skip-gram network are powerful for general text classification task set. The skip-gram models are robust and may be generalized well on different datasets, even without tuning the hyper-parameters for specific dataset.