AI Query Toxicity Detection Using Semantic Similarity Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information handling systems struggle to effectively identify and respond to toxic language in user query inputs, particularly due to the limitations of keyword-based methods like TF-IDF, which fail to consider context, leading to inaccurate identification and inappropriate responses.
Innovation Solution
Implementing a hybrid approach that combines semantic similarity search, such as cosine similarity, with TF-IDF to accurately identify toxic utterances by analyzing context and relevance, and providing appropriate responses based on predefined toxic utterance policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If keyword-based methods like TF-IDF are used to identify toxic language, then the system can process user inputs, but the accuracy of toxic utterance identification deteriorates due to failure to consider context
Solution Approach 1:
The patent combines multiple language analysis methods (TF-IDF, cosine similarity, and additional contextual analysis methods) into a hybrid approach. This merging of methods allows the system to leverage the strengths of each individual method while compensating for their weaknesses, thereby improving overall identification accuracy without relying on a single flawed approach.
Solution Approach 2:
The system employs a multi-functional language analysis module that can perform various tasks including toxic utterance identification, context analysis, and relevance determination. By making the analysis system multi-functional, it can handle diverse language patterns and contextual nuances that single-purpose methods would miss, improving accuracy across different scenarios.
2Measurement precision
If context-aware analysis methods are implemented to improve identification accuracy, then toxic language detection improves, but processing time and system complexity increase
Solution Approach 1:
The system performs preliminary actions by pre-processing user inputs to extract key features, terms, and contextual information before the main analysis. This preliminary extraction and organization of data allows the subsequent toxic utterance identification to be more efficient, as it works with pre-processed information rather than raw text, reducing overall processing time while maintaining high accuracy.
3Ease of operation
If semantic similarity search is used instead of keyword matching, then response appropriateness improves, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary layer of semantic representation and vector space modeling between the raw user input and the toxic utterance identification. This intermediary semantic layer translates textual data into mathematical representations that capture meaning and context, enabling more appropriate responses while managing complexity through structured transformation rather than direct complex analysis.
Data Source
AI summary
An information handling system executing computer readable code instructions for an on the box artificial intelligence (AI) productivity tool may comprise a hardware processor executing computer-readable code instructions for generating toxicity intent values from natural language descriptions of defined toxic utterances that include metadata identifying a toxicity type and a defined response to the toxic utterance, generating a query input intent value for a user query input received via a user requesting an action by an AI productivity tool-enableable software, performing a cosine semantic similarity search comparing the toxicity intent values to the query input intent value to identify a matching toxic utterance in the user query input having a toxicity intent value that generates a highest toxicity cosine semantic similarity search score, and instructing the user interface to provide the defined response from metadata for the identified toxic utterance of a heightened type.