Classifier for Filtering Objectionable Language in Machine Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine translation systems often introduce objectionable language into translations, either by incorrect mapping of source words, inconsistencies in training data, or human translators omitting explicit content, leading to unintended offense to the original speaker.
Innovation Solution
A classifier, potentially using a support vector machine, is employed to identify and remove or replace objectionable words and phrases from machine-generated translations, operating within the translation system or as a standalone service, trained with target word lists and user reports to prevent the introduction of explicit content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a machine translation system is trained using automatically collected training data, then the translation coverage and language pairs supported are improved, but objectionable language may be introduced into the translation
Solution Approach 1:
The classifier is trained in advance using target word lists and user reports to identify objectionable language patterns before translation occurs. This preliminary training enables the system to proactively detect and prevent objectionable language in translations without compromising the broad language coverage achieved through automatic data collection
Solution Approach 2:
A classifier component is introduced as an intermediary between the translation model and the output. This classifier acts as a filter that detects objectionable language in translated text while allowing the translation system to maintain its broad language coverage and automatic training data collection capabilities
2Productivity
If a translation model incorrectly maps source words to destination words, then translation speed is maintained, but objectionable language is introduced
Solution Approach 1:
The classifier provides feedback by detecting objectionable language in translated output and triggering replacement with appropriate alternatives. This feedback mechanism corrects incorrect word mappings while maintaining translation speed, as the classifier operates efficiently on the already-generated translation without requiring retranslation
3Ease of manufacture
If human translators soften explicit content in training data, then the training process is simplified, but the translation accuracy for explicit content is reduced
Solution Approach 1:
The classifier serves as an intermediary that handles explicit content detection and management separately from the translation process. This allows human translators to focus on simplifying the training process while the classifier ensures translation accuracy for explicit content by detecting and appropriately handling such cases in the output
4Object-affected harmful factors
If a classifier is applied to filter translations, then objectionable language is removed, but additional processing time is required
Solution Approach 1:
The classifier is trained in advance on target word lists and user reports to build robust detection capabilities. This preliminary action enables the classifier to quickly identify objectionable language during translation processing, minimizing the additional processing time required while effectively removing harmful content
Data Source
AI summary
Exemplary embodiments relate to detecting, removing, and/or replacing objectionable words and phrases in a machine-generated translation. A classifier identifies translations containing target words or phrases. The classifier may be applied to the output translation to remove target words and phrases from the translation, or to prevent target words and phrases from being automatically presented. Further, the classifier may be applied to a translation model to prevent the target words and phrases from appearing in the output translation. Still further, the classifier may be applied to training data so that the translation model is not trained using the target words of phrases. The classifier may remove target words or phrases only when the target words or phrases appear in the output translation but not the source language input data. The classifier may be provided as a standalone service, or may be employed in the context of a machine translation system.