Bias Detection in Intelligent Virtual Assistant Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Language models in intelligent virtual assistants (IVAs) often introduce bias due to training data containing biased correlations, leading to inaccurate user intent detection and metadata analysis, such as assuming a user is a senior citizen based on associations with senior travel and boarding assistance, or mislabeling user sentiment.
Innovation Solution
A system and method to detect and adjust bias in training data by using a bias scoring system that identifies co-occurrence and single class population biases through digital processing, constructing co-occurrence matrices, and recommending adjustments to the training data to balance representations, allowing human review and correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If training data is used to train language models, then the IVA can perform natural language understanding, but bias is introduced leading to inaccurate intent detection
Solution Approach 1:
The system performs preliminary bias detection and adjustment on training data before it is used to train the language model. By identifying and correcting biased correlations in advance (such as spurious associations between senior citizens and boarding assistance), the system ensures that the language model learns accurate intent patterns rather than biased stereotypes, thereby improving intent detection accuracy while preventing bias introduction
Solution Approach 2:
The system implements feedback mechanisms where bias detection results are fed back into the training data adjustment process. The bias scoring system continuously monitors training data for biased correlations and provides feedback to adjust the data before model training, creating a closed-loop system that ensures high reliability in intent detection while actively preventing harmful biases from entering the model
2Reliability
If the training data is processed to detect bias, then bias can be reduced, but additional processing time and complexity are required
Solution Approach 1:
The bias detection and adjustment processes are performed as preliminary actions during the data preparation phase, before the actual model training begins. By completing these tasks in advance on the training data, the system ensures that the subsequent model training process works with cleaned, bias-adjusted data, improving reliability without significantly extending the overall development timeline
Solution Approach 2:
The system employs automated bias detection and adjustment mechanisms that process training data autonomously without requiring extensive manual intervention. The bias scoring system automatically identifies biased correlations and the adjustment mechanisms automatically modify the data, reducing both time and complexity compared to manual bias review while maintaining high bias reduction effectiveness
Data Source
AI summary
Disclosed is a system and method for detecting and addressing bias in training data prior to building language models based on the training data. Accordingly system and method, detect bias in training data for Intelligent Virtual Assistant (IVA) understanding and highlight any found. Suggestions for reducing or eliminating them may be provided This detection may be done for each model within the Natural Language Understanding (NLU) component. For example, the language model, as well as any sentiment or other metadata models used by the NLU, can introduce understanding bias. For each model deployed, training data is automatically analyzed for bias and corrections suggested.