Method for performing multi-language neural machine translation in resource-constrained environment

Abstract

The invention relates to the technical field of language neural machine translation, and discloses a method for performing multi-language neural machine translation in a resource-constrained environment, which comprises the following steps of: firstly, reducing the model size and the operation complexity by adopting a model compression technology and technologies such as quantification, pruning and knowledge distillation; the system can run on low-power-consumption equipment; then, a network structure shared by multiple languages is designed, and by designing a neural network structure capable of processing multiple languages at the same time and by sharing an encoder and a decoder, the parameter quantity of the model is reduced, and the translation efficiency is improved. Neural machine translation becomes a preferred technology due to translation quality and efficiency, traditional NMT models usually need a large number of computing resources, application of the traditional NMT models in resource limited environments (such as mobile equipment and Internet of Things equipment) is limited, and certain limitation exists when the traditional NMT models are used.

Description

Technical Field

[0001] The present invention relates to the technical field of language neural machine translation, and in particular to a method for performing multilingual neural machine translation in a resource-constrained environment. Background Art

[0002] Neural machine translation is an important branch of machine translation. It uses deep learning technology, especially neural network models, to achieve automatic translation from one language to another. Unlike traditional statistical or rule-based machine translation methods, NMT can learn the complex mapping relationship between the source language and the target language, thereby generating more natural, fluent and accurate translation results. Encoder: The encoder encodes the source language sentence into one or a series of vector representations. This process is usually completed using models such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs) or gated recurrent units (GRUs). The self-attention mechanism is also widely used due to its parallel processing capabilities and efficiency. The decoder (Deco der): The decoder gradually generates the translation result of the target language based on the vector representation generated by the encoder. The decoder can also use RNN, LSTM, GRU or Transformer structure. During the generation process, the decoder may use the attention mechanism to focus on different parts of the source language sentence to generate a more accurate translation. The development of neural machine translation has greatly promoted the performance of machine translation, making the quality of automatic translation close to or even exceeding human translation in some scenarios. The training of NMT models usually requires a large amount of bilingual corpus and high computing resources. With the continuous advancement of technology, NMT still has a lot of research space in processing low-resource languages, improving translation speed and reducing computing costs. Neural machine translation has become the preferred technology due to its translation quality and efficiency. Traditional NMT models usually require a lot of computing resources, which limits their application in resource-constrained environments (such as mobile devices and IoT devices), resulting in certain limitations in the use of traditional NMT models. Summary of the invention

[0003] The purpose of the present invention is to provide a method for multilingual neural machine translation in a resource-constrained environment, so as to achieve the purpose of solving the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for performing multilingual neural machine translation in a resource-constrained environment, the method comprising the following steps: S1. First, we use model compression technology to reduce the model size and computational complexity by using technologies such as quantization, pruning, and knowledge distillation, so that it can run on low-power devices. Then, we design a multi-language shared network structure. By designing a neural network structure that can process multiple languages ​​at the same time and sharing encoders and decoders, we can reduce the number of model parameters and improve translation efficiency. S2. Adopt an adaptive training strategy that can dynamically adjust the training data and model structure according to different translation tasks and language pairs to optimize translation quality and resource utilization; S3. At the same time, you can also use cross-language pre-training, using large cross-language pre-training models (such as XLM-R) to enhance the model's understanding and generation capabilities for multiple languages, especially for language pairs with fewer resources, further expanding the scope of translation; S4. Adopting transfer learning and fine-tuning strategies and multi-language joint training methods, the use of multi-language joint methods can effectively improve the translation effect of low resources and the translation rate of low resources. The transfer learning method can save the learned content for better expansion in the future. S5. Use semi-supervised and unsupervised learning methods to help learners better master knowledge. At the same time, develop semi-supervised or unsupervised machine translation methods, use monolingual data and a small amount of bilingual data to train translation models, and reduce dependence on a large amount of parallel corpus; S6. Use the corpus circulation method to expand the translation content through bilingual seed resources and ultra-large-scale monolingual resources, thereby improving the quality of translation; S7. Use multilingual attention mechanism and multilingual embedding space to set language mechanism to facilitate better translation in different regions and environments; S8. Optimize the model structure to a certain extent, and use external knowledge resources to assist in translating the language of local resources; S9. Effectively improve the language translation rate through model compression and acceleration technology, and adopt model compression and hardware acceleration technology to run in an environment with limited computing resources while maintaining translation quality.

[0005] Preferably, the transfer learning and fine-tuning strategy adopts a transfer learning method to transfer the knowledge of the existing high-resource language translation model to the low-resource language translation task, and adapts to the characteristics of the specific language pair through fine-tuning.

[0006] Preferably, the multilingual joint training is to perform joint training by combining rich bilingual corpora with less multilingual corpora to improve the performance of low-resource language translation.

[0007] Preferably, the corpus loop advancement can design a multi-loop framework and a semi-supervised algorithm, using bilingual seed resources of appropriate scale and ultra-large-scale monolingual resources to expand bilingual sentence pair resources, gradually improve translation quality, and also improve the translation of the entire sentence.

[0008] Preferably, the multilingual attention mechanism mainly designs a language-sensitive attention mechanism to dynamically select different attention modes to adapt to the translation needs of different language pairs.

[0009] Preferably, the multilingual embedding space can construct a multilingual embedding space so that semantically similar words in different languages ​​have similar representations, thereby enabling better understanding and generation during translation.

[0010] Preferably, the model structure optimization mainly includes optimizing the structure of the neural machine translation model, such as the design of the encoder and decoder, so as to reduce the parameter scale and improve the efficiency and performance of the model.

[0011] Preferably, the use of external knowledge resources is to assist in learning the translation model of low-resource languages ​​by integrating external knowledge resources such as linguistic rules, bilingual dictionaries, and translation memories.

[0012] The present invention provides a method for multilingual neural machine translation in a resource-constrained environment. It has the following beneficial effects: (1) The present invention adopts transfer learning and fine-tuning strategies and multi-language joint training methods, and uses multi-language joint methods to effectively improve the translation effect of low resources and the translation rate of low resources. The transfer learning method can save the learned content, which is convenient for better expansion in the later stage. The semi-supervised and unsupervised learning methods are adopted, and the learners can better master the knowledge through semi-supervised and unsupervised learning methods. At the same time, semi-supervised or unsupervised machine translation methods are developed, and monolingual data and a small amount of bilingual data are used to train the translation model, reducing the dependence on a large amount of parallel corpus. The corpus is cycled and promoted. The bilingual seed resources and ultra-large-scale monolingual resources are used to expand the translation content, thereby improving the translation quality. The existing neural machine translation has become the preferred technology due to its translation quality and efficiency. The traditional NMT model usually requires a large amount of computing resources, which limits their application in resource-constrained environments (such as mobile devices, Internet of Things devices), resulting in certain limitations in the use of the traditional NMT model.

[0013] (2) The present invention uses a multilingual attention mechanism and a multilingual embedding space, sets a language mechanism, and optimizes the model structure to facilitate better translation in different regions and environments. It can also use external knowledge resources to assist in translating the language of local resources. It can effectively improve the language translation rate through model compression and acceleration technology, and adopt model compression and hardware acceleration technology to run in an environment with limited computing resources while maintaining translation quality. DETAILED DESCRIPTION

[0014] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation methods of the present invention are now described.

[0015] The present invention provides a method for performing multilingual neural machine translation in a resource-constrained environment: A method for performing multilingual neural machine translation in a resource-constrained environment, the method comprising the following steps: S1. First, we use model compression technology to reduce the model size and computational complexity by using technologies such as quantization, pruning, and knowledge distillation, so that it can run on low-power devices. Then, we design a multi-language shared network structure. By designing a neural network structure that can process multiple languages ​​at the same time and sharing encoders and decoders, we can reduce the number of model parameters and improve translation efficiency. S2. Adopt an adaptive training strategy that can dynamically adjust the training data and model structure according to different translation tasks and language pairs to optimize translation quality and resource utilization; S3. At the same time, you can also use cross-language pre-training, using large cross-language pre-training models (such as XLM-R) to enhance the model's understanding and generation capabilities for multiple languages, especially for language pairs with fewer resources, further expanding the scope of translation; S4. Adopting transfer learning and fine-tuning strategies and multi-language joint training methods, the use of multi-language joint methods can effectively improve the translation effect of low resources and the translation rate of low resources. The transfer learning method can save the learned content for better expansion in the future. S5. Use semi-supervised and unsupervised learning methods to help learners better master knowledge. At the same time, develop semi-supervised or unsupervised machine translation methods, use monolingual data and a small amount of bilingual data to train translation models, and reduce dependence on a large amount of parallel corpus; S6. Use the corpus circulation method to expand the translation content through bilingual seed resources and ultra-large-scale monolingual resources, thereby improving the quality of translation; S7. Use multilingual attention mechanism and multilingual embedding space to set language mechanism to facilitate better translation in different regions and environments; S8. Optimize the model structure to a certain extent, and use external knowledge resources to assist in translating the language of local resources; S9. Effectively improve the language translation rate through model compression and acceleration technology, and adopt model compression and hardware acceleration technology to run in an environment with limited computing resources while maintaining translation quality.

[0016] The transfer learning and fine-tuning strategy uses transfer learning methods to transfer the knowledge of existing high-resource language translation models to low-resource language translation tasks, and fine-tune them to adapt to the characteristics of specific language pairs.

[0017] Multilingual joint training combines rich bilingual corpora with less multilingual corpora for joint training to improve the performance of low-resource language translation.

[0018] Corpus loop advancement can design multi-loop frameworks and semi-supervised algorithms, using appropriately sized bilingual seed resources and ultra-large-scale monolingual resources to expand bilingual sentence pair resources, gradually improve translation quality, and improve overall sentence translation.

[0019] The main content of the multilingual attention mechanism is to design a language-sensitive attention mechanism to dynamically select different attention modes to adapt to the translation needs of different language pairs.

[0020] Multilingual embedding space can construct a multilingual embedding space so that semantically similar words in different languages ​​have similar representations, which can be better understood and generated during translation.

[0021] Model structure optimization mainly includes optimizing the structure of the neural machine translation model, such as the design of the encoder and decoder, to reduce the parameter scale and improve the efficiency and performance of the model.

[0022] The use of external knowledge resources is to assist in learning translation models for low-resource languages ​​by integrating external knowledge resources such as linguistic rules, bilingual dictionaries, and translation memories.

[0023] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for multilingual neural machine translation in a resource-constrained environment, characterized in that; The method comprises the following steps: S1. First, we use model compression technology to reduce the model size and computational complexity by using technologies such as quantization, pruning, and knowledge distillation, so that it can run on low-power devices. Then, we design a multi-language shared network structure. By designing a neural network structure that can process multiple languages ​​at the same time and sharing encoders and decoders, we can reduce the number of model parameters and improve translation efficiency. S2. Adopt an adaptive training strategy that can dynamically adjust the training data and model structure according to different translation tasks and language pairs to optimize translation quality and resource utilization; S3. At the same time, you can also use cross-language pre-training, using large cross-language pre-training models (such as XLM-R) to enhance the model's understanding and generation capabilities for multiple languages, especially for language pairs with fewer resources, further expanding the scope of translation; S4. Adopting transfer learning and fine-tuning strategies and multi-language joint training methods, the use of multi-language joint methods can effectively improve the translation effect of low resources and the translation rate of low resources. The transfer learning method can save the learned content for better expansion in the future. S5. Use semi-supervised and unsupervised learning methods to help learners better master knowledge. At the same time, develop semi-supervised or unsupervised machine translation methods, use monolingual data and a small amount of bilingual data to train translation models, and reduce dependence on a large amount of parallel corpus; S6. Use the corpus circulation method to expand the translation content through bilingual seed resources and ultra-large-scale monolingual resources, thereby improving the quality of translation; S7. Use multilingual attention mechanism and multilingual embedding space to set language mechanism to facilitate better translation in different regions and environments; S8. Optimize the model structure to a certain extent, and use external knowledge resources to assist in translating the language of local resources; S9. Effectively improve the language translation rate through model compression and acceleration technology, and adopt model compression and hardware acceleration technology to run in an environment with limited computing resources while maintaining translation quality.

2. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The transfer learning and fine-tuning strategy adopts a transfer learning method to transfer the knowledge of the existing high-resource language translation model to the low-resource language translation task, and adapts it to the characteristics of a specific language pair through fine-tuning.

3. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The multilingual joint training is to perform joint training by combining rich bilingual corpora with less multilingual corpora to improve the performance of low-resource language translation.

4. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The corpus loop advancement can design a multi-loop framework and a semi-supervised algorithm, use an appropriate scale of bilingual seed resources and ultra-large-scale monolingual resources to expand bilingual sentence pair resources, gradually improve the translation quality, and improve the translation of the entire sentence.

5. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The main content of the multilingual attention mechanism is to design a language-sensitive attention mechanism to dynamically select different attention modes to adapt to the translation needs of different language pairs.

6. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The multilingual embedding space can construct a multilingual embedding space so that semantically similar words in different languages ​​have similar representations, thereby enabling better understanding and generation during translation.

7. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The model structure optimization mainly includes optimizing the structure of the neural machine translation model, such as the design of the encoder and decoder, to reduce the parameter scale and improve the efficiency and performance of the model.

8. The method for performing multilingual neural machine translation in a resource-constrained environment according to claim 1, characterized in that: The use of external knowledge resources is to assist in learning the translation model of low-resource languages ​​by integrating external knowledge resources such as linguistic rules, bilingual dictionaries, and translation memories.