Translation Quality Evaluation Using Distributed Representation Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automatic translation quality evaluation methods, such as BLEU, are limited in accurately assessing translation quality as they only consider exact correspondence between translated and reference texts, leading to poor grading of excellent translations and high grading of incorrect ones.
Innovation Solution
A distributed representation model using neural networks converts automatic and reference translations into vector representations, allowing for similarity evaluation based on distances like Euclidean, Hamming, or cosine distance, enabling a more accurate assessment of translation quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional automatic evaluation method (BLEU) is used to evaluate translation quality, then evaluation can be performed automatically without manual intervention, but the evaluation accuracy deteriorates because it only considers exact correspondence between translated and reference texts
Solution Approach 1:
The patent changes the evaluation parameter from exact n-gram matching to distributed vector representation similarity. By converting translations into distributed representations and evaluating based on vector similarity metrics (cosine similarity, Euclidean distance), the system maintains automated evaluation while significantly improving accuracy in assessing translation quality, thereby resolving the contradiction between automation and measurement precision
Solution Approach 2:
The patent replaces the mechanical string-matching mechanism of BLEU with a neural network-based distributed representation system. This substitution allows the evaluation to capture semantic similarity rather than just surface-level word overlap, achieving both automation and improved measurement precision through the use of learned vector embeddings
2Measurement precision
If distributed representation model is used to evaluate translation quality, then evaluation accuracy is improved by capturing semantic similarity, but computational complexity increases due to neural network processing
Solution Approach 1:
The patent applies preliminary action by pre-training the distributed representation model on large corpora before actual translation evaluation. The neural networks (Word2Vec, GloVe, or FastText) are trained in advance to learn semantic relationships, so that during evaluation, only forward propagation through the trained model is needed, significantly reducing the computational complexity during the actual evaluation phase while maintaining high measurement precision
Data Source
AI summary
An apparatus and method for evaluating quality of an automatic translation is disclosed. An apparatus for evaluating quality of automatic translation includes a converter which converts an automatic translation and a reference translation of an original text to a first distributed representation and a second distributed representation, respectively, using a distributed representation model and a quality evaluator which evaluates quality of automatic translation data based on similarity between the first distributed representation and the second distributed representation.