Probability-Based Guider for Neural Network Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks for natural language processing tasks are computationally expensive and wasteful, as they often apply excessive computation to achieve narrow accuracy improvements in irrelevant linguistic areas.
Innovation Solution
A probability-based guider system that combines a cheap bag of words model with an expensive LSTM model, using probability thresholding to determine when to invoke the LSTM model for accurate classification, thereby reducing computational cost while maintaining high accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deep neural network (LSTM) is used for natural language processing, then classification accuracy is improved, but computational cost increases significantly
Solution Approach 1:
The system segments the classification task by dividing inputs into two groups: easy sentences handled by the simple bag-of-words model and difficult sentences handled by the complex LSTM model. This segmentation allows the system to achieve high accuracy on difficult cases while avoiding unnecessary computational cost on easy cases, directly resolving the contradiction between accuracy and computational expense.
Solution Approach 2:
The probability threshold acts as an intermediary mechanism that mediates between the simple bag-of-words model and the complex LSTM model. By thresholding the probability outputs of the bag-of-words model, the system determines which cases require the more expensive LSTM processing, thus controlling the trade-off between accuracy and computational cost without directly invoking the LSTM for all inputs.
2Measurement precision
If deep neural networks are applied to all inputs, then accuracy is maximized, but computational resources are wasted on easy cases
Solution Approach 1:
The system applies partial action by using the simple bag-of-words model for all inputs and only invoking the expensive LSTM model partially for difficult cases identified by low probability thresholds. This avoids the excessive application of deep neural networks to all inputs while maintaining accuracy where needed, directly addressing the waste of computational resources on easy cases.
3Use of energy by moving object
If a simple bag of words model is used, then computational cost is reduced, but classification accuracy decreases
Solution Approach 1:
The system merges two classification models with different characteristics: the simple bag-of-words model for speed and the accurate LSTM model for difficult cases. By combining these models in a cascaded architecture where the bag-of-words model handles easy cases and the LSTM model handles difficult cases, the system achieves both low computational cost and high accuracy, resolving the contradiction between these two parameters.
Data Source
AI summary
The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.