Conversational Safety Rewriting for Unsafe Chatbot Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current conversational AI models, such as chatbots, often generate unsafe responses due to training on large corpora containing offensive and toxic content, leading to risks of direct offensiveness and harmful advice.
Innovation Solution
The development of the SafeConv dataset, which includes utterance-level safety labels, unsafe spans, and safe alternative responses, along with a neural network model for contextual rewriting, to generate context-relevant and safe alternatives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If chatbots are trained end-to-end with Language Modeling objectives on large corpora, then the conversational capability is improved, but unsafe behavior and toxic content generation occur
Solution Approach 1:
The patent introduces an intermediary safety checking mechanism between the language model and the final output. A separate safety classifier model is trained independently to detect unsafe spans in generated responses, acting as a mediator that filters harmful content before it reaches the user, thus resolving the contradiction between conversational capability and safety
Solution Approach 2:
The patent segments the safety evaluation process by identifying and labeling specific unsafe spans within responses rather than evaluating the entire response as a single unit. This segmentation allows for precise detection and correction of harmful portions while preserving safe and relevant content, maintaining conversational capability while reducing unsafe behavior
2Reliability
If safety checking is performed on generated responses, then unsafe behavior is reduced, but processing time and computational resources increase
Solution Approach 1:
The patent applies partial action by focusing safety checking only on specific spans of text that are likely to contain unsafe content, rather than uniformly processing every token in the response. The model identifies and flags only the problematic portions, reducing overall processing time while maintaining high safety standards
3Reliability
If unsafe spans are identified and corrected in responses, then safety is improved, but response coherence and contextual relevance may deteriorate
Solution Approach 1:
The patent extracts only the unsafe spans from the response for correction, leaving the rest of the context intact. By removing and replacing only the harmful portions rather than the entire response, the system maintains safety while preserving the original contextual relevance and coherence of the safe content
Data Source
AI summary
Method, apparatus, and non-transitory storage medium for augmenting datasets for conversational safety, including generating a safety label for an utterance. The process may include identifying one or more inappropriate spans of a plurality of words for the utterance, and determining one or more corrective spans of the plurality of words for replacing the one or more inappropriate spans in the utterance. The process may also include generating revised utterance based on the one or more corrective spans and the utterance.