Transformer Text Moderation via Token Manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated text processing systems face challenges in ensuring the quality of online content due to the difficulty in detecting and moderating socially unacceptable language, as users often employ countermeasures to evade detection by using alternative or obfuscated words.
Innovation Solution
A framework for text moderation using pre-trained transformers that generates context-rich training data, allowing for simultaneous actions and manipulations of tokens to normalize text, enabling context-sensitive moderation by learning which tokens to modify and how to modify them based on contextual information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional word-based text processing systems are used, then the system complexity is low, but the ability to detect and moderate socially unacceptable language is insufficient
Solution Approach 1:
The patent replaces traditional mechanical word-based processing systems with transformer-based neural network models that use self-attention mechanisms and contextual embedding to understand and moderate text content, significantly improving detection accuracy while managing complexity through pre-trained models
Solution Approach 2:
The patent changes the fundamental parameters of text processing by transitioning from discrete word tokenization to subword tokenization (Byte Pair Encoding), and from flat feature representations to high-dimensional contextual embeddings, enabling more nuanced language understanding
2Reliability
If users employ alternative or obfuscated words to evade detection, then the ease of operation for users increases, but the reliability of text moderation decreases
Solution Approach 1:
The patent applies preliminary anti-action by pre-training transformer models on diverse text corpora that include obfuscated and alternative forms of unacceptable language, enabling the model to recognize and moderate such content before it can effectively evade detection
Solution Approach 2:
The patent implements dynamic text moderation by using contextual embeddings that adaptively interpret words based on their surrounding context, allowing the system to dynamically distinguish between legitimate alternative expressions and obfuscated unacceptable language
3Measurement precision
If context-sensitive text moderation is implemented, then the measurement precision of inappropriate content detection is improved, but the device complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-training transformer models on large-scale text corpora before deployment, allowing the models to learn contextual patterns and nuances offline, thereby achieving high detection precision without increasing the complexity of the deployed system
Solution Approach 2:
The patent uses copying by creating multiple token representations (original tokens, subword tokens, and contextual embeddings) that capture different aspects of text meaning, enabling precise detection while managing complexity through hierarchical representation
Data Source
AI summary
The present teaching relates to method, system, medium, and implementations for text processing. Upon receiving input data including an original token and a ground truth token label for the original token, a manipulation is applied to the original token to generate a manipulated token based on which to generate manipulated input data. Training data is generated based on the manipulated input data, the ground truth token label, and a ground truth action that, when applied to the manipulated token, yields the original token with the ground truth token label. A text moderation model is trained based on the training data.