Embedding Transformation for Machine Translation Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models for machine translation tasks require high learning and computing costs due to their complex structure and large number of weight parameters, making them inefficient for training.
Innovation Solution
An embedding transformation method and system that transforms source-side data into target-side embedding vectors using a transformation model, which includes implicit and attention layers, and is trained using pretrained source-side and target-side embedding models to reduce computing costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep learning models with complex structure and large number of weight parameters are used for machine translation tasks, then translation performance is improved, but learning cost and computing cost increase significantly
Solution Approach 1:
The patent segments the translation task into two independent phases: (1) encoding source text into embedding vectors using a source embedding model, and (2) decoding embedding vectors into target text using a target decoding model. This segmentation allows each model to be specialized and optimized independently, reducing the overall complexity and computational requirements compared to a single large-scale translation model.
Solution Approach 2:
The patent applies preliminary action by pre-training embedding models on large corpora to learn meaningful representations of text. These pre-trained embedding models capture linguistic patterns and semantic relationships, so when used for translation tasks, they provide a strong foundation that reduces the additional training cost and data requirements for the specific translation task.
2Reliability
If deep learning models with complex structure are used for machine translation tasks, then translation performance is improved, but device complexity increases
Solution Approach 1:
The patent divides the translation system into separate encoding and decoding components with distinct functions. The source embedding model focuses on converting source text to meaningful vectors, while the target decoding model focuses on generating target text from vectors. This functional segmentation simplifies the overall system architecture compared to a monolithic complex model.
Solution Approach 2:
The patent introduces embedding vectors as an intermediary representation between source text and target text. Instead of directly mapping source text to target text through a complex model, the system uses embedding vectors as a compact, information-rich intermediate form that captures the semantic meaning, thereby reducing the complexity of the transformation process.
3Reliability
If deep learning models are trained to outperform SMT models, then translation quality is improved, but time cost and computing cost increase enormously
Solution Approach 1:
The patent leverages pre-trained embedding models that have already learned linguistic patterns from large corpora. This preliminary action of pre-training captures essential language knowledge, so the models require less additional training time for specific translation tasks while still achieving high translation quality that outperforms traditional SMT approaches.
Solution Approach 2:
The patent uses pre-trained embedding models that have been trained on extensive data, effectively copying proven successful representations from the pre-training phase. This allows the system to benefit from the knowledge embedded in these models without incurring the full training cost, thereby reducing training time while maintaining high translation quality.
Data Source
AI summary
Provided is a embedding transformation method performed by at least one computing device. The method comprises obtaining a source-side embedding model, transforming source-side data into a first embedding vector through the source-side embedding model and transforming the first embedding vector into a second embedding vector located in a target-side embedding space through a transformation model.