Generating Sentiment Models from Discourse Marker Weak Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Pretrained language models struggle with sentiment classification in zero or few-shot settings due to scarce labeled data, leading to suboptimal performance.
Innovation Solution
Generate weakly-labeled data using sentiment-carrying discourse markers to inter-train pretrained language models, creating a discourse marker-based general sentiment-aware model and adapting it for specific domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pretrained language models are used for sentiment classification, then the model has general language understanding capabilities, but it struggles in zero or few-shot settings due to scarce labeled data
Solution Approach 1:
The patent applies preliminary action by pre-processing the training data to identify and extract discourse markers before model training. Sentences are systematically analyzed to detect discourse markers, and weak labels are generated in advance based on these markers. This preliminary structuring of data with identified discourse markers and preliminary sentiment labels enables the model to learn effective sentiment classification patterns even with limited labeled data, resolving the contradiction between model reliability and data scarcity.
2Reliability
If more training data is used to improve model performance, then sentiment classification accuracy improves, but the complexity of data preparation and processing increases
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate weak labels for training data through discourse marker detection. The discourse marker-based labeling system autonomously processes sentences, identifies sentiment-carrying markers, and assigns labels without requiring extensive manual annotation. This self-service approach to data preparation significantly reduces the complexity of data processing while still providing sufficient training data to improve sentiment classification performance.
3Productivity
If weak labels generated from discourse markers are used for inter-training, then the model adapts better to sentiment classification with fewer parameters, but the quality of labels may be lower than manual annotations
Solution Approach 1:
The patent uses discourse markers as an intermediary to bridge the gap between automatic labeling and manual annotation quality. Discourse markers serve as reliable linguistic indicators that carry sentiment information and can be systematically detected. By using these markers as an intermediary labeling mechanism, the system achieves a practical balance between training efficiency and label quality, enabling effective model adaptation without requiring costly manual annotation while maintaining sufficient precision for sentiment classification tasks.
Data Source
AI summary
An example system includes a processor to receive a list of sentiment carrying discourse markers. The processor is to select sentences in a text corpus that begin with a discourse marker from the list of sentiment carrying discourse markers followed by a comma. The processor is to remove each discourse marker and comma from a beginning of the selected sentences and labeling each of the sentences with a sentiment associated with to a corresponding removed discourse marker to generate a weakly labeled dataset. The processor is to inter-train a pretrained language model using the generated weakly labeled dataset to generate a sentiment model.