Non-autoregressive machine translation method and system for adaptive end-to-end quantization alignment

Abstract

According to the non-autoregressive machine translation method and system for adaptive end-to-end quantization alignment, a semantic quantization space is constructed to serve as a bridge and a constraint of an auxiliary translation model in training; during training, the translation model and the semantic quantization space are collaboratively optimized to ensure that the translation model can capture the semantic consistency of a source language and a target language, and only source statement information is input in training and reasoning without additionally introducing target information, so that the unification of the translation model in the training and reasoning processes is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to machine translation, and in particular relates to a non-autoregressive machine translation method and system with adaptive end-to-end quantization alignment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, the Transformer model has become one of the most widely used neural networks for machine translation tasks due to its powerful language modeling capabilities and precise translation quality. However, the original Transformer model's left-to-right autoregressive (AT) word-by-word translation method results in high decoding latency. The emergence of non-autoregressive (NAT) machine translation offers a promising solution to the high latency issue inherent in AT networks. NAT networks break away from the AT's word-by-word translation strategy and can generate multiple words in a single decoding pass, making them more suitable for real-world applications such as conversations, conferences, and live broadcasts. While NAT networks significantly improve speed through parallel decoding, they often suffer from reduced translation quality due to insufficient modeling of dependencies between translated words. Current NAT improvements primarily focus on two distinct paradigms: iterative non-autoregressive decoding, which gradually optimizes the translation output over multiple iterations to achieve a balance between translation quality and decoding efficiency; and non-iterative non-autoregressive decoding, which generates a complete translation in a single forward pass, leveraging the advantages of NAT decoding.

[0004] The machine translation task can be summarized as follows: given a parallel corpus dataset ,in, Represents the source sentence. Represents the target sentence. The goal of the neural network is to learn the given arrive The existing NAT system uses conditional masked language modeling (CMLM) to improve NAT's ability to model the dependencies between translated words (target words). In addition to inputting the source sentence, this method inputs some observable target words at the decoder end, that is, some words in the target sentence are masked as prediction targets, and the unmasked target words are used as the input of the decoder to predict the masked target words. This explicitly guides the model to learn from the source sentence. and observable target words to the masked target word The mapping relationship between them.

[0005] However, the training paradigm based on masked language modeling This leads to a discrepancy between NAT training and inference: the training phase uses additional observable target information to reconstruct masked target words, while the inference phase masks all target words to predict the complete translation. According to the inventors, this training-inference gap prevents NAT from fully unleashing its performance. Existing solutions have shown that simply annealing the number of masked target word samples to 0 during training does not address this shortcoming and still leads to degraded model performance.

[0006] Therefore, how to eliminate this training-inference difference to ensure that NAT performance can be fully released is of great research significance. Summary of the Invention

[0007] To overcome the above-mentioned shortcomings of the existing technology, the present invention provides a non-autoregressive machine translation method and system with adaptive end-to-end quantization alignment. Through the semantic quantization space, it assists in the training of the translation model to ensure that the translation model can capture the semantic consistency of the source language and the target language, achieving unity in the training and inference processes.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a non-autoregressive machine translation method with adaptive end-to-end quantization alignment, comprising: Construct a semantic quantization space and perform pre-alignment training; Constructing a semantic quantization alignment loss based on the consistency of the hidden representation output by the encoder in the translation model and the hidden representation output by the decoder in the translation model in the semantic quantization space; Obtaining a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sampling from the discrete vector, using the sampling result as input to the decoder, and constructing a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The translation model is trained based on the semantic quantization alignment loss and the translation maximization likelihood loss, and the input sentence is translated based on the trained translation model.

[0009] In a second aspect, the present invention provides a non-autoregressive machine translation system with adaptive end-to-end quantization alignment, comprising: A semantic quantization space construction module is configured to: construct a semantic quantization space and perform pre-alignment training; A first loss module is configured to: construct a semantic quantization alignment loss based on the consistency of a hidden representation output by an encoder in the translation model and a hidden representation output by a decoder in the translation model in the semantic quantization space; A second loss module is configured to: obtain a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sample from the discrete vector, use the sampling result as the input of the decoder, and construct a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The training and translation module is configured to: train the translation model based on the semantic quantization alignment loss and the translation maximization likelihood loss, and translate the input sentence based on the trained translation model.

[0010] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0011] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0012] One or more of the above technical solutions have the following beneficial effects: In the present invention, a semantic quantization space is constructed as a bridge and constraint for the auxiliary translation model in training. During training, the translation model and the semantic quantization space are collaboratively optimized to ensure that the translation model can capture the semantic consistency of the source language and the target language. Training and reasoning only input source sentence information without additional introduction of target information, thereby achieving the unity of the translation model in the training and reasoning processes.

[0013] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0015] Figure 1 Schematic diagram of the impact of training-inference discrepancy on MLM-based non-autoregressive models; Figure 2 This is an overall diagram of the adaptive end-to-end quantization alignment framework in Example 1 of the present invention; Figure 3 This is an example of applying alignment and reordering to semantic sequence representation in the first embodiment of the present invention; Figure 4 This is a comparison chart of the ability of different models to handle the N-tuple duplication problem in Example 1 of the present invention; Figure 5 The figure is a comparative diagram of the decoding acceleration ratio and performance trade-off between different models in the first embodiment of the present invention. DETAILED DESCRIPTION

[0016] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0017] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0018] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0019] Example 1 As mentioned in the background technology, current NAT mainly relies on masked language modeling (MLM) to enhance the dependencies between target words. However, the MLM-based method causes a difference between NAT training and reasoning, that is, the training phase gives additional observable target information to reconstruct the masked target words, while the reasoning phase masks all target words to predict the complete translation. This difference leads to the decline of NAT performance. Annealing the sampling rate of observable target words to 0 can be simply regarded as achieving the unification of training and reasoning, but empirical evidence shows that this actually leads to a decline in NAT performance. Figure 1 As shown, Figure 1 The solid line in the middle shows the performance change of the non-autoregressive model when inputting target words at different sampling rates during the inference phase. The dotted line shows the model performance obtained under the original inference conditions (i.e., ignoring this training-inference difference). CMLM stands for the Conditional Masked Language Model method, GLAT stands for the Glancing Transformer model, DAT stands for the Directed Acyclic Transformer model, DAT* stands for the Directed Acyclic Transformer model with an additional linear layer, and AT stands for the standard Transformer model.

[0020] To address this issue, this embodiment proposes a non-autoregressive machine translation framework, AEQA-NAT, with adaptive end-to-end quantization alignment. The basic idea is to construct a semantic quantization space outside the NAT model as a bridge and constraint to assist the NAT system in training. During training, the NAT model and the semantic quantization space are collaboratively optimized to ensure that NAT can capture the semantic consistency of the source and target languages. Training and reasoning only input source sentence information without introducing additional target information, thereby achieving the unity of the NAT system during training and reasoning.

[0021] This embodiment proposes a non-autoregressive machine translation method with adaptive end-to-end quantization alignment, including: Construct a semantic quantization space and perform pre-alignment training; Constructing a semantic quantization alignment loss based on the consistency of the hidden representation output by the encoder in the translation model and the hidden representation output by the decoder in the translation model in the semantic quantization space; Obtaining a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sampling from the discrete vector, using the sampling result as input to the decoder, and constructing a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The translation model is trained based on the semantic quantization alignment loss and the translation maximization likelihood loss, and the input sentence is translated based on the trained translation model.

[0022] like Figure 2 As shown in Figure 2, AEQA-NAT consists of two stages: pre-alignment and training. The AEQA-NAT framework mainly consists of three modules: an encoder , a shared semantic quantization space (SQS) and a non-autoregressive decoder .

[0023] Input source sequence , n is the number of tokens in the sequence, AEQA-NAT predicts the target sequence The process can be expressed mathematically as: (1) Prediction module with extra length The softcopy technique is used to initialize the decoder input. The length prediction module is a single-layer neural network that is used to train the difference between the input length and the actual translation length.

[0024] For the predicted length , soft copy initializes the decoder Input: (2) Among them, the weight By source text position and the target text position The distance relationship is determined by It is a trainable parameter used to adjust the expansion amplitude during replication. is the target sequence length.

[0025] The following is a detailed description of the various modules and training processes involved in the AEQA-NAT framework: Pre-aligned Semantic Quantization Space (SQS): The source and target languages are pre-aligned in the semantic quantization space using the encoder of the multilingual pre-trained model mBART.

[0026] Specifically, the semantic quantization space can be viewed as a Dimensional Vocabulary ,in, is the number of word vectors, is the dimension of each word vector. Set the dimension of the word vector to the same size as the encoder in the translation model.

[0027] Given the hidden variables of the encoder and , is the source sequence length, is the target sequence length.

[0028] Respectively and Each dimension of is mapped to the semantic quantization space embedding, expressed as: (3) (4) in, Representing the minimized distance in the semantic quantization space Vector Right now The index of the embedding vector of Representing the minimized distance in the semantic quantization space Vector Right now The index of the embedding vector of Representing embeddings in a semantic quantization space.

[0029] In addition, a commitment loss is used to constrain the encoder and decoder outputs to vectors in the semantic quantization space. This commitment loss helps the translation model remain stable and consistent during the encoding process, preventing it from learning overly free or unconstrained representations and ensuring that the encoder output representation remains consistent with the semantic quantization space.

[0030] Specifically, this embodiment uses exponential moving average (EMA) to update the semantic quantization space: (5) in, To represent the function, is the encoder hidden layer representation of and, is the attenuation factor. The embedding vectors calculated from the past and the new By averaging, represents the exponential weighted average of the past. In this way, the embedding vector in the semantic quantization space SQS will gradually adapt to the data distribution.

[0031] In order to ensure effective alignment, a fine-tuning task on the pre-trained model is designed to optimize the pre-aligned SQS: It is worth noting that the SQS at this time contains the pre-aligned source language vector and target language vector, i.e. and To ensure translation quality, we tend to use Keep the original embedding position so that Towards Clustering. Define training objectives To align the two language representations in SQS: (6) (7) (8) in, It is control The hyperparameters of is the stop-gradient operation, is the hidden representation of the target text output by the encoder, which is In addition, Depend on To obtain, Depend on Obtain.

[0032] The training process of the adaptive end-to-end quantization alignment training framework mainly optimizes two training objectives: semantic quantization alignment loss and semantic quantization alignment loss. and translation maximizes likelihood loss : (1) For semantic quantization alignment loss ,This embodiment uses a pre-aligned semantic quantization space to facilitate the mapping of the translation model from the source language to the target language, so as to ensure the consistency of the output of the encoder and decoder in the semantic quantization space during the training process, such as Figure 2 Specifically, there are: (9) (10) (11) (12) in, represents the output of the last decoder layer (before mapping to the vocabulary size linear layer), which is consistent with the vector dimension of the semantic quantization space.

[0033] (2) Maximizing the likelihood loss for translation It is known that conditional language modeling can significantly enhance the dependency of NAT target tokens. To ensure that the translation model can generate complete translations while still maintaining this ability, this embodiment designs a training strategy compatible with Glancing training. Specifically, given an input , Maximize the target sequence The conditional probability of: (13) (14) in, Represents the Glancing Sampling glance sampling strategy, Used to determine the number of samples of masked tokens. Note that, unlike existing methods, the masked tokens replaced in this embodiment are not target tokens, but discrete vectors output from the semantic quantization space. Represents the input after mask, that is, the data after partially masking the discrete vector output of the semantic quantization space during training.

[0034] For Glancing Sampling, it is mainly divided into two steps: (1) Measuring the degree of difference: Use Hamming distance to quantify whether each predicted word is correct (i.e., whether it is equal to the true answer), in the form of:

[0035] Where T is the length of the real translation, Y is the actual target sequence, The target sequence predicted by the decoder.

[0036] (2) Sampling based on quantization results:

[0037] in, is a hyperparameter that controls the sampling rate. The final sampling number m is obtained. Random sampling is performed based on the obtained sampling number.

[0038] Finally, the overall training loss is: (15) in, and is a hyperparameter that controls the impact of semantic quantization alignment loss and length prediction loss, Prediction loss for length.

[0039] Based on the above multi-loss joint optimization, the model can more effectively utilize semantic information, enhance language quantization consistency, and optimize the entire end-to-end translation process.

[0040] Aligned Reordering: According to formula (1), the decoder input The intermediate representation output by the encoder is obtained through the semantic quantization space, that is, It is a discrete representation in the semantic quantization space that corresponds to each word in the source sequence. It can be understood that the input has a stronger correlation with the source sequence than the target sequence, that is, the discrete representation may not conform to the grammatical rules of the target language. To address this problem, this solution designs an alignment reordering strategy to correct ,like Figure 3 shown.

[0041] Note that alignment reordering only changes the word order, not the word vector itself. This example defines an alignment probability distribution matrix To adjust the word order. First, given the representation from the semantic quantization space SQS and target sequence , the alignment probability distribution matrix is calculated as follows: (16) in, , which is row normalized to capture Each word in the target sequence The probability distribution of Figure 3 As shown, according to the maximum probability position of each line, the final correct word order is found .

[0042] The reasoning process of this embodiment utilizes source text and quantized alignment information, which is consistent with the training process. Specifically, there are: (17) Thanks to the fact that the training process does not introduce additional target information, the non-autoregressive adaptive end-to-end quantization alignment training framework (AEQA-NAT) can seamlessly generalize the knowledge in the training data to the inference stage.

[0043] Table 1. Comparison results of the present invention with existing methods on the benchmark indicators WMT14 EN-DE and WMT16 EN-RO

[0044] As shown in Table 1, Transformer (autoregressive model) is the standard Transformer model, while the other comparison models are non-autoregressive models. CMLM stands for the Conditional Masked Language Model method, Con-NAT stands for the contrastive non-autoregressive method, GLAT stands for the Glancing Transformer model, DAT stands for the Directed Acyclic Transformer model, and Rephraser stands for the rephrasing the reference for NAT method. The model scores for each translation direction are based on the BLEU metric, with higher scores indicating better performance. Original and distilled data indicate that the model was trained using original and distilled data, respectively. Distilled data can be considered a data distribution composed of a mixture of generated translations from a pretrained teacher model and original data. Distilled data is easier to train and is a common auxiliary technique for training non-autoregressive methods. The speedup is based on the inference time of the Transformer model on the WMT 14EN-DE test set with a batch size of 1.

[0045] Compared with the autoregressive method Transformer, this embodiment increases the inference acceleration ratio to 9.1-12.2 times, significantly improving the inference speed. The translation quality gap with Transformer in all translation directions is less than 1 BLEU, further narrowing the performance gap between non-autoregressive and autoregressive methods.

[0046] Compared with iterative non-autoregressive methods (CMLM, Con-NAT, and CMLMC), this embodiment significantly improves inference speed, generates all translations with only one decoding, and surpasses iterative methods in multiple translation directions. While maintaining the decoding advantage, this embodiment achieves translation quality comparable to iterative methods.

[0047] Compared with fully non-autoregressive methods (GLAT, DAT, and Rephraser), the solution of this embodiment achieved the best performance on all translation benchmarks, which shows that the solution of this embodiment significantly enhances the dependency between target words and better captures the data distribution characteristics.

[0048] In addition, the solution of this embodiment can be effectively implemented for the basic NAT model and DAT model, which proves that the solution has good scalability and portability, and provides a good foundation for subsequent improvement of non-autoregressive methods. Table 2. Comparison of modeling capabilities of the present invention and other methods for raw data features (dependence on knowledge distillation)

[0049] Knowledge distillation (KD) is widely used in non-autoregressive learning. As shown in Table 2, after using knowledge distillation, the Transformer's translation quality improved by only 0.03, indicating that it has strong modeling capabilities for the original data and low dependence on distilled data. Intuitively, the stronger the model's ability to capture data distribution characteristics, the lower its dependence on distilled data. GLAT has a high dependence on distilled data, and its BLEU score increased to 4.21 after using KD. DAT significantly narrowed this gap to 0.81. This embodiment reduces this gap to 0.29 for the first time. This demonstrates that AEQA-NAT can accurately capture the distribution characteristics of the original data and has excellent dependency modeling capabilities.

[0050] Table 3. Comprehensive evaluation of the performance of this scheme and other non-autoregressive methods on multiple benchmarks

[0051] In order to comprehensively evaluate the performance of AEQA-NAT, this embodiment performs indicator evaluation on multiple key benchmarks. As shown in Table 3, for the original data, AEQA-NAT outperforms other NAT models in both rule-based and model-based indicators, which shows that the solution of this embodiment can generate more coherent and higher-quality translations. For distilled data, AEQA-NAT performs better on model-based indicators, which shows that the introduction of semantic quantization space SQS better captures the semantic relationship between the source language and the target language, because this indicator evaluates the translation quality by measuring the semantic relevance between two sentences based on parameterized knowledge. In summary, the solution of this embodiment outperforms existing solutions on multiple benchmarks, fully demonstrating the accuracy of the translation quality generated by this solution.

[0052] The ability of AEQA to handle the N-tuple duplication problem was evaluated. Figure 4 As shown in the figure, AEQA-DAT significantly outperforms the DAT model in reducing n-gram repetitions. This is due to AEQA's reduced input length and reduced number of references on the directed acyclic graph, which is particularly effective when the n-gram size is less than 4. Compared to all NAT models, AEQA-NAT maintains a consistently low n-gram repetition count, demonstrating its ability to robustly handle multimodal problems.

[0053] like Figure 5 As shown in the figure, the proposed scheme (AEQA-NAT) performs well in terms of relative decoding speed, achieving an acceleration ratio that exceeds all other models. This result shows that the proposed scheme has significant superiority in decoding efficiency over the NAT model. In terms of translation quality, AEQA - NAT also performs well, with its BLEU score exceeding that of other NAT models. This shows that the translations generated by AEQA - NAT are closer in quality to the reference translations, further narrowing the performance gap with the autoregressive Transformer (AT) model. Overall, AEQA - NAT achieves the best trade-off between performance and speed. Example 2 The purpose of this embodiment is to provide a non-autoregressive machine translation system with adaptive end-to-end quantization alignment, including: A semantic quantization space construction module is configured to: construct a semantic quantization space and perform pre-alignment training; A first loss module is configured to: construct a semantic quantization alignment loss based on the consistency of a hidden representation output by an encoder in the translation model and a hidden representation output by a decoder in the translation model in the semantic quantization space; A second loss module is configured to: obtain a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sample from the discrete vector, use the sampling result as the input of the decoder, and construct a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The training and translation module is configured to: train the translation model based on the semantic quantization alignment loss and the translation maximization likelihood loss, and translate the input sentence based on the trained translation model.

[0054] In further embodiments, there is also provided: An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor. When the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0055] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0056] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0057] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in embodiment 1 is performed.

[0058] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0059] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0060] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions contained in program modules, which are executed in a device on a real or virtual processor of a target to perform the process / method described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided between program modules as needed. The machine-executable instructions for the program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0061] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the computer or other programmable data processing device, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0062] In the context of the present invention, computer program code or related data can be carried by any appropriate carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0063] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0064] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A non-autoregressive machine translation method with adaptive end-to-end quantization alignment, characterized by: include: Construct a semantic quantization space and perform pre-alignment training; Constructing a semantic quantization alignment loss based on the consistency of the hidden representation output by the encoder in the translation model and the hidden representation output by the decoder in the translation model in the semantic quantization space; Obtaining a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sampling from the discrete vector, using the sampling result as input to the decoder, and constructing a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The translation model is trained based on the semantic quantization alignment loss and the translation maximization likelihood loss, and the input sentence is translated based on the trained translation model.

2. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 1, characterized in that: The source and target languages are pre-aligned in the language quantization space through a multilingual pre-training model, and the embedding vector of the semantic quantization space is updated using exponential moving average.

3. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 1, wherein: The semantic quantization alignment loss is specifically: in, represents the output of the last layer decoder, which is consistent with the vector dimension of the semantic quantization space; is the source language, is to stop the gradient operation, is the hidden representation of the target text output by the encoder, Depend on To obtain, Depend on To obtain, is the encoder hidden layer representation, .

4. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 1, wherein: The translation maximization likelihood loss is specifically: in, represents the glance sampling strategy, The number of samples used to determine the mask mark, Represents the input after the mask, X represents the input source sequence, and Y represents the predicted target sequence.

5. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 1, wherein: Also includes: The decoder input is corrected by the alignment reordering strategy so that the decoder input conforms to the grammatical rules of the target language, specifically: Calculate the representation of the semantic quantization space and the alignment probability distribution matrix of the target sequence; The calculated alignment probability distribution matrix is row-normalized to capture the probability distribution of each word to the target sequence in the representation of the semantic quantization space; The final correct word order is determined based on the maximum probability position of each line.

6. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 5, characterized in that: The alignment reordering only changes the word order, without changing the word vectors themselves.

7. The non-autoregressive machine translation method with adaptive end-to-end quantization alignment according to claim 1, wherein: The translation model is inferred using the quantized alignment information of the source language and the semantic quantization space.

8. A non-autoregressive machine translation system with adaptive end-to-end quantized alignment, characterized by: include: A semantic quantization space construction module is configured to: construct a semantic quantization space and perform pre-alignment training; A first loss module is configured to: construct a semantic quantization alignment loss based on the consistency of a hidden representation output by an encoder in the translation model and a hidden representation output by a decoder in the translation model in the semantic quantization space; A second loss module is configured to: obtain a discrete vector based on the hidden representation output by the encoder through the semantic quantization space, sample from the discrete vector, use the sampling result as the input of the decoder, and construct a translation maximization likelihood loss based on maximizing the generation of the target language sequence; The training and translation module is configured to: train the translation model based on the semantic quantization alignment loss and the translation maximization likelihood loss, and translate the input sentence based on the trained translation model.

9. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 7 is completed.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the method according to any one of claims 1 to 7.

Images