Masked Topic Sentiment Prediction for Context-Based NLP Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sentiment identification methods, both rule-based and machine learning approaches, suffer from lower-than-desired accuracy due to the manual creation and maintenance of negation dictionaries and reliance on dominant sentiment frequencies, leading to inaccurate predictions for noun phrases.
Innovation Solution
A machine learning model is trained with masked noun phrases using techniques like ClinicalBERT or BERT, where topics are replaced with arbitrary identifiers, allowing the model to predict sentiments more accurately by focusing on context rather than training set frequencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional machine learning approaches are used to predict sentiment, then the model can process large amounts of training data, but the accuracy deteriorates due to reliance on dominant sentiment frequencies in the training set
Solution Approach 1:
The patent extracts and masks the noun phrase (topic) from the training data, replacing it with a placeholder token. This allows the model to learn sentiment patterns based on contextual relationships rather than relying on the frequency of specific topics or dominant sentiments in the training set, thereby improving accuracy while still utilizing large volumes of training data.
Solution Approach 2:
The masked noun phrase acts as an intermediary element in the training process. By replacing the actual topic with a mask token, the model learns to predict sentiment based on the surrounding context and relationships, rather than memorizing topic-sentiment associations from the training data, which improves generalization accuracy.
2Measurement precision
If rule-based approaches with negation dictionaries are used, then the system can identify negation patterns, but the complexity increases due to manual creation and maintenance requirements
Solution Approach 1:
The patent replaces the mechanical rule-based system with a neural network model that automatically learns negation patterns from data. Instead of manually creating and maintaining negation dictionaries and rules, the model uses deep learning to automatically detect and understand negation contexts, significantly reducing system complexity while maintaining or improving detection accuracy.
Solution Approach 2:
The neural network model performs self-learning and self-adjustment during training, automatically discovering negation patterns and relationships without human intervention. The model serves itself by learning from training data and improving its own performance, eliminating the need for manual rule creation and maintenance.
3Measurement precision
If larger models are used to improve sentiment prediction accuracy, then the model can capture more complex patterns, but the computational resources and hardware requirements increase
Solution Approach 1:
The patent changes the approach by masking noun phrases in the training data, which allows smaller models to achieve better accuracy. This parameter change in the training methodology enables the use of less computationally intensive models that consume fewer resources while still capturing complex sentiment patterns effectively.
Data Source
AI summary
A facility for determining sentiments expressed by a natural-language text string for each of one or more topics is described. In the natural-language text string, the facility identifies one or more topics. For each identified topic, the facility replaces the topic in the natural-language text string with a masking tag that occupies the same position in the natural-language text string as the topic. After the replacing, the facility applies a machine learning model to the natural-language text string to obtain a predicted sentiment for each of the identified topics.