Statistical Machine Translation Annotation Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Statistical machine translation (SMT) engines trained with translation memory (TM) systems face inaccuracies when translating documents with annotations, as the abstract tags in TM databases interfere with the training process and result in less accurate translation probabilities for segments not rewritten in the same manner.
Innovation Solution
A method and system that integrate translation memory systems with statistical machine translation engines by defining translation rules for annotations, allowing the engine to distinguish and process annotations separately, thereby improving translation accuracy and control over document translation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If translation memory includes abstract tags to identify occurrences such as product names and numerals, then the TM can provide perfect translations for matched segments, but the tags interfere with the training process of SMT engine and result in less accurate translation probabilities for segments not rewritten via the same process
Solution Approach 1:
The patent segments the translation process into two distinct parts: (1) annotation processing using translation rules for tagged elements like product names and numerals, and (2) statistical machine translation for the remaining text. This segmentation allows the SMT engine to be trained on data with annotations without the tags interfering with probability learning, while still achieving accurate translations for both annotated and non-annotated segments.
Solution Approach 2:
The patent introduces an intermediary annotation processing layer that sits between the TM database and the SMT engine. Translation rules act as mediators that handle annotated segments separately, preventing the tags from directly interfering with the SMT training process while still enabling accurate translation of annotated content. This intermediary layer reconciles the conflict between TM exact matching and SMT probabilistic translation.
2Adaptability or versatility
If the modified TM with tags is used to train the SMT engine, then the training process can incorporate structured annotation data, but the translation accuracy deteriorates for segments that have not been rewritten with the same tag format
Solution Approach 1:
The patent applies local quality by treating annotated and non-annotated segments differently during translation. Annotated segments undergo specific rule-based processing that preserves the structure and meaning of tagged elements, while non-annotated segments are translated using standard SMT methods. This localized approach ensures high accuracy for both types of segments without requiring all segments to be rewritten with tags.
Solution Approach 2:
The patent implements a dynamic translation process that adapts its methodology based on the presence of annotations. The system dynamically selects between rule-based annotation processing and probabilistic SMT translation for each segment, allowing it to leverage the benefits of both approaches without the rigidity of a single fixed process. This dynamic adaptation maintains translation accuracy across diverse input formats.
Data Source
AI summary
A method, system, and computer readable medium for translating a document is provided. A statistical machine translation engine is trained using a translation memory comprising an annotation. A translation rule associated with the annotation is defined. A source document in a source language is received. The source document comprises an instance of the annotation and a string. The string is translated using the statistical machine translation engine. The instance of the annotation is processed according to the translation rule. A target document in a target language is generated based on the translated string and the processed annotation.