Machine Translation Accuracy Evaluation via Vector Space Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for evaluating the accuracy of machine translations rely on human subjective measures or require a reference corpus, which are not scalable and often lack sufficient reference translations, especially for less common languages.
Innovation Solution
A computing device uses machine learning techniques to analyze parallel texts and determine translation accuracy without a reference corpus, employing word embeddings to align languages in a shared vector space, calculate similarity scores, and assess the minimum number of edits needed to match source and translated text sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human translators are used to evaluate translation accuracy, then measurement precision is improved, but productivity deteriorates due to inability to scale for high volume evaluations
Solution Approach 1:
The patent replaces the mechanical human evaluation system with an automated computational system using machine learning models. The system processes translations through trained models that compare source and target texts, generating accuracy scores without human intervention, thereby enabling high-volume automated evaluation while maintaining objective measurement.
2Measurement precision
If reference corpus is used to train evaluation models, then measurement precision is improved, but adaptability deteriorates for languages with insufficient reference translations
Solution Approach 1:
The patent performs preliminary training of machine learning models on available reference corpora for major language pairs. These pre-trained models are then adapted to evaluate translations in languages with limited reference data by leveraging transfer learning and cross-lingual similarities, allowing the system to provide evaluation capabilities for low-resource languages without requiring extensive language-specific training data.
3Adaptability or versatility
If subjective human measures are used for evaluation, then adaptability to nuanced translation quality is improved, but loss of information increases due to subjectivity variability
Solution Approach 1:
The patent implements a feedback mechanism where the machine learning model generates objective accuracy scores based on systematic comparison of source and target texts. The model provides consistent, repeatable evaluations that eliminate human subjectivity variability, while still capturing nuanced translation quality through learned patterns from training data, thus reducing information loss from evaluator inconsistency.
Data Source
AI summary
Disclosed are various embodiments for evaluating the accuracy of a translation of a source text. Word embeddings from a first language and a second language are aligned in a shared vector space. Word pairs from the sourced text and translated text are then identified. Subsequently, similarity scores between respective word embeddings for the words in the word pair are calculated. Word pairs are then selected based on the similarity scores. The accuracy of the translation is then evaluated based at least in part on the selected word pairs.