Sentiment Data Augmentation for Negation and Fairness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence-based chatbots face challenges in accurately handling negation and fairness in sentiment analysis due to inadequate training data, leading to incorrect sentiment predictions and perpetuation of biases.
Innovation Solution
The method involves data augmentation by creating negation pairs and fairness invariance data sets through rewriting examples with negation cues, sentiment prefixes/suffixes, and demographic word substitutions, followed by batch balancing to enhance model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard training data is used for machine learning models, then the model can be trained quickly, but the model incorrectly predicts sentiment class frequently when training data is inadequate
Solution Approach 1:
The system performs preliminary actions by generating synthetic negation training examples before actual model training. It creates negation pairs by inserting negation words into template sentences and identifies negation cues in existing data, preparing the training data in advance to improve sentiment prediction accuracy when dealing with negated statements.
Solution Approach 2:
The system creates copies of existing training examples by generating synthetic negation versions. It copies template sentences and inserts negation words at different positions to create multiple negation variants, and copies existing labeled examples to create negation pairs for training the model to recognize sentiment in negated statements.
2Ease of operation
If chatbots are developed to simulate conversations intelligently, then conversational experience is improved, but the development becomes difficult and costly requiring specialized developers
Solution Approach 1:
The system enables self-service by allowing organizations to automatically generate and prepare training data for their own chatbot sentiment analysis models without requiring specialized developers. The automated negation example generation and training data preparation tools empower users to build intelligent conversational systems independently.
3Reliability
If training data is enriched with negation examples, then the model handles negation better, but the data preparation process becomes more complex
Solution Approach 1:
The system applies parameter changes by systematically varying the position of negation words in template sentences and adjusting the insertion points of negation cues in existing data. It changes the structural parameters of training examples to create diverse negation variants while maintaining the underlying sentiment patterns, making data preparation more systematic.
4Measurement precision
If sentiment analysis models are trained to recognize all sentiment patterns, then prediction accuracy improves, but the model perpetuates biases present in training data
Solution Approach 1:
The system applies preliminary anti-action by proactively identifying and addressing bias in training data before model training. It detects biased language patterns and sentiment associations in the training corpus and prepares corrected or balanced training examples to counteract the harmful biases before they can be learned by the model.
Data Source
AI summary
Techniques for augmentation and batch balancing of training data to enhance negation and fairness of a machine learning model. In one particular aspect, a method is provided that includes generating a list of demographic words associated with a demographic group, searching an unlabeled corpus of text to identify unlabeled examples in a target domain comprising at least one demographic word from the list of demographic words, rewriting the unlabeled examples to create one or more versions of each of the unlabeled examples and generate a fairness invariance data set, and training the machine learning model using unlabeled examples from the fairness invariance data set.