Context-Aware Multilingual Profanity Detection With Dual-Level Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional profanity detection methods in large language models (LLMs) suffer from false positives due to context insensitivity and high computational latency, and existing text classification models struggle with inconsistent dual-level profanity classification, leading to inaccurate and inefficient profanity mitigation.
Innovation Solution
A machine learning model is trained using a regularization term to optimize both sentence-level and token-level profanity classification, employing context-aware embeddings and a co-optimization loss function to improve accuracy and reduce computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional profanity detection methods are used in large language models, then profanity can be detected, but false positives occur due to context insensitivity
Solution Approach 1:
The patent segments the profanity detection task into two distinct levels: token-level classification (identifying individual profane tokens) and sentence-level classification (determining whether the entire sentence contains profanity). This segmentation allows the model to analyze profanity at multiple granularities, reducing false positives by considering both individual token context and overall sentence context. The multi-level approach enables the system to distinguish between profane tokens used in profane contexts versus profane tokens used in non-profane contexts.
Solution Approach 2:
The patent introduces a dual-level classification dimension by simultaneously performing token-level and sentence-level profanity classification. This dimensional expansion allows the model to capture profanity nuances that single-level approaches miss, particularly for polysemic tokens that may appear profane in isolation but are used innocuously in context. The regularization term enforces consistency across these dimensions, improving reliability.
2Measurement precision
If conventional profanity detection methods are used in large language models, then profanity can be detected, but computational latency increases
Solution Approach 1:
The patent merges the token-level classification and sentence-level classification into a single unified model architecture that processes input text once and generates both levels of classification simultaneously. This merging eliminates the need for separate processing passes, reducing computational overhead and latency while maintaining the accuracy benefits of multi-level analysis. The shared encoder and classification layers enable efficient joint optimization of both tasks.
3Adaptability or versatility
If existing text classification models perform dual-level profanity classification, then both token-level and sentence-level detection are achieved, but classification consistency deteriorates
Solution Approach 1:
The patent implements feedback through a regularization term in the loss function that monitors and enforces consistency between token-level and sentence-level classifications. The regularization term calculates disagreement between the two levels and penalizes inconsistent predictions, providing continuous feedback during training to align the classifications. This feedback mechanism ensures that when a sentence is classified as profane at the sentence level, the token-level classifications support this determination, and vice versa.
Solution Approach 2:
The patent creates a composite loss function that combines the token-level classification loss, sentence-level classification loss, and regularization loss into a unified objective. This composite loss function integrates multiple objectives (accuracy at both levels and consistency between levels) into a single optimization target, enabling the model to learn coherent profanity detection patterns that satisfy all requirements simultaneously.
4Measurement precision
If conventional profanity detection is implemented, then profanity can be identified, but granular profanity identification and correction become inefficient
Solution Approach 1:
The patent segments the profanity mitigation process into targeted token-level corrections based on the dual-level classification results. By identifying which tokens are profane at the token level and which sentences are profane at the sentence level, the system can apply corrections only where necessary rather than processing entire texts uniformly. This segmentation enables efficient, granular mitigation that maintains productivity while achieving high precision.
Data Source
AI summary
Certain aspects of the disclosure relate to profanity detection and mitigation. A method generally includes training a machine learning (ML) model using labeled training data instances by, for each training data instance: providing the tokens of the respective training data instance to an input layer of the ML model; receiving a first output for each token of the respective training data instance classifying the respective token as a profanity-containing or a non-profanity-containing token; receiving a second output for the respective training data instance classifying the respective training data instance as a profanity-containing or a non-profanity-containing instance; determining a loss value based on the first output for each token and the second output using a loss function comprising a regularization term configured to increase loss based on disagreement between the first output for each token and the second output; and modifying parameter(s) of the ML model based on the loss value.