Conditional-Likelihood Dataset Filtering for Harmful Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large pretrained neural language models trained on open web datasets propagate systemic biases and harmful content, making it difficult to filter harmful text effectively due to the scale of the datasets and the impracticality of human evaluation.
Innovation Solution
A method using a pretrained language model to compute the conditional likelihood of trigger phrases conditioned on each document in the corpus, allowing for the filtering of documents that increase the likelihood of biased or hateful rhetoric, thereby creating a filtered dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If word-level blocklists are used to filter harmful text, then obvious hateful text can be removed, but nuanced or context-dependent hateful text may be missed and non-hateful documents may be erroneously flagged
Solution Approach 1:
The patent introduces an intermediary classification system with multiple categories (obvious hateful text, nuanced hateful text, context-dependent text, and non-hateful text) between the raw input and final filtering decision. This intermediary classification allows for more precise identification of harmful content by evaluating multiple dimensions rather than relying on simple keyword matching, thereby improving filtering accuracy while reducing false positives.
Solution Approach 2:
The patent changes the parameters used for filtering from simple keyword presence to a multi-parameter evaluation system that includes conditional likelihood scores, contextual analysis, and multiple classification categories. By transforming the filtering criteria from binary (blocklist match/no match) to multi-dimensional parameter assessment, the system achieves higher precision in identifying harmful text while preserving legitimate content.
2Object-affected harmful factors
If vocabulary shifting is used to boost non-toxic tokens, then non-toxic content can be enhanced, but tokens are assigned negative connotations regardless of context leading to inaccurate filtering
Solution Approach 1:
The patent implements a dynamic filtering system that adjusts its evaluation criteria based on contextual factors rather than applying static vocabulary shifts. The system dynamically assesses each token's meaning based on surrounding context, document type, and linguistic patterns, allowing the same token to be classified differently in different contexts. This dynamic approach preserves contextual adaptability while effectively identifying harmful content.
Solution Approach 2:
The patent applies different filtering criteria and evaluation standards to different parts of the text based on their local context. Rather than uniformly applying negative connotations to certain tokens throughout the document, the system evaluates each token's significance in its specific local context, allowing academic, rhetorical, or expository uses to be distinguished from genuinely harmful uses. This local quality approach maintains contextual versatility.
3Measurement precision
If manual evaluation and filtration are used to remove harmful content, then filtering accuracy can be improved, but the large size of datasets makes it impractical if not impossible
Solution Approach 1:
The patent segments the filtering task into multiple automated sub-tasks: initial classification into categories, conditional likelihood scoring for each category, contextual analysis, and final decision-making. By dividing the comprehensive filtering process into manageable automated segments, the system achieves manual-level accuracy without requiring manual evaluation of entire datasets, thereby maintaining high productivity.
Solution Approach 2:
The patent implements automated feedback loops where the classification system continuously refines its evaluations based on learned patterns from the data. The conditional likelihood scoring mechanism provides feedback on the probability of harmful content, allowing the system to adapt and improve its filtering accuracy automatically without human intervention, thus maintaining both high precision and efficiency.
4Difficulty of detecting and measuring
If a language model is trained on an unfiltered corpus to compute conditional likelihood, then the model can identify biases in text, but the model may still generate harmful text when prompted with innocuous prompts
Solution Approach 1:
The patent applies preliminary filtering actions during the training phase by computing conditional likelihood scores and removing documents that exhibit harmful patterns before the model is fully trained. This preliminary action prevents the model from learning harmful associations in the first place, addressing the root cause of harmful text generation rather than merely detecting biases after training. The filtering occurs in advance of model training, ensuring the model learns from cleaner data.
Data Source
AI summary
A system and method are provided for generating a trained model to filter data sets for filtering hate speech. The method includes obtaining an unfiltered corpus of data, obtaining a set of trigger phrases, and using the set of trigger phrases to generate a trained model which comprises at least one conditional likelihood of the trigger phrases conditioned on documents in the corpus of data. A system and method are also provided for filtering data sets for hate speech using pre-trained models. The method includes obtaining a pretrained model generated using a set of trigger phrases and which comprises at least one conditional likelihood of the trigger phrases conditioned on document in a corpus of data used to generate the pretrained model; using the pretrained model to filter an unfiltered dataset and generate a filtered dataset; and outputting the filtered dataset.