Predictive Text Filtering Model for Offensive Language Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text prediction systems struggle to effectively filter out offensive language, especially when it is composed of seemingly harmless words or terms, and with the increasing complexity of neural prediction models, traditional methods are insufficient.
Innovation Solution
A predictive text filtering model is implemented, which maintains a list of precarious terms associated with a classification and applies filtering rules based on the cooccurrence of these terms within a specific word range, using range resetting characters and cooccurrence exceptions to determine whether to filter or surface candidates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional dictionary-based filtering is used to suppress obscene language, then single-word offensive terms can be filtered, but compound offensive concepts composed of multiple seemingly harmless words cannot be detected
Solution Approach 1:
The patent segments the filtering task into multiple components: a dictionary-based component for single-word filtering and a neural network-based component for detecting compound offensive concepts. The neural network processes sequences of words to identify harmful combinations that individual dictionary lookups would miss, thereby improving detection accuracy without requiring complete system redesign
Solution Approach 2:
The patent introduces a neural network as an intermediary layer between the input text and the final filtering decision. This intermediary processes the text through learned representations to detect subtle offensive patterns, bridging the gap between simple dictionary matching and comprehensive offensive language detection
2Productivity
If neural prediction models are used to generate predictive text, then prediction capability and sequence length are improved, but the ability to predetermined and control offensive language outputs is lost
Solution Approach 1:
The patent applies filtering rules preliminarily to the vocabulary and candidate generation process of neural models. By pre-configuring the neural network with harmful concept definitions and filtering criteria, the system maintains prediction capability while ensuring offensive outputs are prevented before they reach the user
Solution Approach 2:
The patent implements feedback mechanisms where the filtering model continuously monitors neural model outputs and provides correction signals. When potentially offensive predictions are detected, the feedback loop adjusts the prediction process to avoid harmful outputs, maintaining reliability without sacrificing prediction quality
3Object-affected harmful factors
If comprehensive filtering rules are applied to all predictive text candidates, then offensive language is effectively blocked, but legitimate text predictions may be incorrectly filtered
Solution Approach 1:
The patent applies different filtering strictness levels to different contexts and candidate types. Rather than uniformly filtering all predictions, the system adjusts filtering intensity based on local characteristics such as context relevance, prediction confidence, and specific harmful concept identification, allowing legitimate text to pass while blocking offensive content
Data Source
AI summary
In non-limiting examples of the present disclosure, systems, methods and devices for filtering predictive text surfacing candidates are provided. A predictive text filtering model may be maintained. The predictive text filtering model may comprise a plurality of terms that are associated in the predictive text filtering model with a precarious classification, and a range of a number of words for filtering cooccurrences of precarious and other precarious or blocklist terms from the plurality of terms. A text input may be processed with a predictive text model. A plurality of surfacing candidates may be determined based on the processing of the text input with the predictive text model. The predictive text filtering model may be applied to a surfacing candidate of the plurality of candidates. The surfacing candidate may be filtered from further processing. At least one non-filtered surfacing candidate may be displayed.