Multi-Task Neural Network for Contextual Toxicity Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing toxicity detection systems in interactive platforms suffer from high false positives and negatives, particularly in detecting covert toxicity, due to reliance on keyword-based approaches that disregard context and simplistic machine-learning methods that learn spurious correlations.
Innovation Solution
A multi-task neural network is trained to perform toxicity detection and attribute prediction using a combination of task-specific and task-common feature extractors, with adversarial losses to disentangle task-specific information and mitigate biases, enabling more accurate and interpretable toxicity detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If keyword-based approaches are used for toxicity detection, then the system is simple to implement, but it produces high false positives and negatives due to lack of context understanding
Solution Approach 1:
The patent replaces simple keyword-matching mechanisms with a neural network-based system that uses transformers and attention mechanisms to understand contextual relationships between words, enabling accurate detection of covert toxicity while maintaining implementation feasibility through standardized deep learning frameworks
2Ease of manufacture
If simplistic machine-learning methods are used, then the system is easier to train, but it learns spurious correlations and fails to detect nuanced toxicity
Solution Approach 1:
The patent changes the model architecture parameters by using transformer-based models with attention mechanisms instead of simple machine learning algorithms, enabling the system to capture nuanced contextual patterns and spurious correlations while maintaining trainability through standard deep learning optimization techniques
Solution Approach 2:
The patent segments the toxicity detection task into multiple sub-tasks by training separate models for different toxicity types (e.g., hate speech, harassment, threats) and using multi-label classification, allowing the system to learn nuanced patterns for each type while maintaining overall system simplicity
3Reliability
If a multi-task neural network with task-specific and task-common feature extractors is used, then detection accuracy and interpretability are improved, but device complexity increases
Solution Approach 1:
The patent implements a multi-task neural network where a single model architecture performs multiple functions: toxicity detection, attribute prediction (sentiment, topic, intent), and context understanding through shared task-common feature extractors, reducing overall system complexity compared to separate specialized models while maintaining high detection accuracy
Solution Approach 2:
The patent adds a task dimension to the feature extraction architecture by introducing task-specific feature extractors that operate on top of shared task-common features, enabling the system to handle multiple toxicity types and attributes simultaneously without proportionally increasing computational complexity
4Object-affected harmful factors
If comprehensive toxicity detection is implemented, then user safety is improved, but storage and computational resources are consumed
Solution Approach 1:
The patent performs preliminary toxicity screening using the multi-task neural network to identify and flag only the most suspicious content for human moderator review, rather than storing all user-generated content, thereby reducing storage requirements while maintaining comprehensive safety coverage through prioritized intervention
Data Source
AI summary
This specification provides a computer-implemented method for detecting toxic user-generated textual content. The method comprises obtaining input data comprising a representation of user-generated textual content. A toxicity prediction and a prediction for each of one or more attributes for the user-generated textual content are generated by processing the input data using a multi-task neural network.