Context-Masked Multi-Task Sentiment Model for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning techniques for sentiment analysis suffer from inaccuracies and latency issues, particularly when using separate single-task models for sentence-level and aspect-based sentiment analysis, leading to conflicting predictions and increased computational load.
Innovation Solution
A multi-task model is developed that integrates sentence-level and aspect-based sentiment analysis using context masking, allowing a single NLP model to provide input to both heads, reducing latency and improving accuracy by selectively considering neighboring sentence contexts for aspect-based analysis while maintaining current sentence focus for sentence-level analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate single-task models are used for sentence-level and aspect-based sentiment analysis, then each model can be optimized for its specific task, but the system experiences increased latency and computational load
Solution Approach 1:
The patent combines sentence-level sentiment analysis and aspect-based sentiment analysis into a single multi-task model. The model shares common layers for processing input text and employs task-specific output heads for each analysis type, enabling both tasks to be performed simultaneously on the same input document, thereby reducing inference latency while maintaining accuracy.
Solution Approach 2:
The multi-task model is designed to perform multiple sentiment analysis functions using a single unified architecture. The model processes both sentence-level sentiment and aspect-based sentiment through shared computational resources, making the system more versatile and efficient compared to separate specialized models.
2Measurement precision
If separate single-task models are used for sentence-level and aspect-based sentiment analysis, then each model can focus on its specific task, but the system experiences increased computational load
Solution Approach 1:
The patent merges sentence-level and aspect-based sentiment analysis into a single multi-task model that shares common computational layers. This consolidation reduces redundant computations and lowers overall computational resource consumption while maintaining the specialized capabilities needed for accurate sentiment analysis.
3Measurement precision
If context from neighboring sentences is included for aspect-based sentiment analysis, then prediction accuracy improves, but processing time increases
Solution Approach 1:
The patent segments the input document into chunks, where each chunk contains a target sentence along with selected neighboring sentences. This segmentation allows the model to efficiently process context only where needed for aspect-based sentiment analysis, rather than processing the entire document sequentially, thus improving accuracy without proportionally increasing processing time.
Solution Approach 2:
The model applies partial context inclusion by selectively adding neighboring sentences to chunks based on the specific needs of aspect-based sentiment analysis. Rather than processing all possible context, the model uses just enough surrounding context to improve aspect sentiment prediction accuracy, avoiding unnecessary computational overhead.
Data Source
AI summary
A method includes accessing document including sentences, document being associated with configuration flag indicating whether ABSA, SLSA, or both are to be performed; inputting the document into language model that generates chunks of token embeddings for the document; and, based on the configuration flag, performing at least one from among the ABSA and the SLSA by inputting the chunks of token embeddings into a multi-task model. When performing the SLSA, a part of token embeddings in each of the chunks is masked, and the masked token embeddings do not belong to a particular sentence on which the SLSA is performed.