Multilingual NLP Model Training Using Parallel Corpora
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches to providing multilingual support in natural language processing (NLP) models face challenges such as high latency, low coverage of foreign language inputs, loss of contextual information, and the need for extensive and labor-intensive human-labeled non-English training data.
Innovation Solution
The proposed solution involves generating training data for multilingual NLP models by selecting English-language content items and their non-English translations, generating candidate labels using English-language NLP models, and fine-tuning a pretrained multilingual NLP model with this data to improve its performance across languages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine translation is used to translate non-English input to English for processing by NLP models, then multilingual support is provided, but latency becomes too high to be acceptable
Solution Approach 1:
The patent creates synthetic training data by copying English training data and translating it to non-English languages using machine translation. This copied and translated data is then used to train the NLP model directly in multiple languages, eliminating the need for runtime translation and reducing latency while maintaining multilingual support
Solution Approach 2:
The patent performs machine translation during the training data preparation phase rather than during runtime inference. By pre-translating English training data into multiple languages and training the model on this multilingual data beforehand, the system avoids translation latency during actual usage while still providing multilingual support
2Loss of time
If bilingual dictionary mapping is used to map foreign tokens to English, then latency issues are overcome, but coverage of foreign language inputs becomes low and contextual information is lost
Solution Approach 1:
The patent changes the training parameter approach by training the NLP model directly on multilingual data rather than using dictionary-based token mapping. This allows the model to learn contextual meanings of words in different languages, preserving contextual information while maintaining low latency during inference
3Quantity of substance
If machine translation is used to create non-English training data, then training data is generated, but errors in machine translation are carried over to the training data
Solution Approach 1:
The patent implements a feedback mechanism where multiple NLP models analyze the translated training data to detect inconsistencies and errors. The system uses the original English data as a reference and compares it with translated versions, identifying and correcting translation errors before finalizing the training data, thus maintaining reliability while generating multilingual training data
Data Source
AI summary
A data processing system for generating training data for a multilingual NLP model implements obtaining a corpus including first and second content items. The first content items are English-language textual content, and the second content items are translations of the first content items in one or more non-English target languages. The system further implements selecting a first content item from the first content items, generating a plurality of candidate labels for the first content item by analyzing the first content item with a plurality of first English-language NLP models, selecting a first label from the plurality of candidate labels, generating first training data by associating the first label with the first content item, generating second training data by associating the first label with a second content item of the second content items, and training a pretrained multilingual NLP model with the first training data and the second training data.