The invention relates to the technical field of
natural language processing, and discloses a super-large-scale multi-language neural
machine translation method based on a MoE
system architecture, which comprises the following steps: S1, an
encoder and a decoder adopt a Transform-based architecture, and a MoE layer is embedded in the Transform-based architecture; s2, the MoE layer comprises a plurality of expert models for specific language pair or language characteristic optimization; s3, designing a dynamic gating mechanism, and selecting the most suitable expert model for prediction according to input data features; s4, introducing a load balancing mechanism to ensure that all expert models are subjected to balanced training and use; s5, performing model training by adopting a combined strategy of
data parallelism and MoE parallelism; and S6, compressing and optimizing the model by
applying knowledge distillation and quantification technologies. The invention discloses a super-large-scale multi-language neural
machine translation method based on a MoE
system architecture. The super-large-scale multi-language neural
machine translation method is an innovative and challenging technology. By reasonably designing and optimizing a model structure, a gating mechanism and a training strategy, the performance and efficiency of multi-language
machine translation can be remarkably improved.