A low-resource machine translation method using semi-autoregressive fusion of domain terminology

Abstract

The present invention relates to a low-resource machine translation method that utilizes semi-autoregressive fusion of domain terminology, and belongs to the technical field of natural language processing machine translation. The present invention uses an external knowledge-guided decoder by fusing a low-resource machine translation method for specific domain terminology, combines autoregressive and non-autoregressive decoding methods, and does not sacrifice decoding efficiency while introducing prior knowledge, thereby meeting the translation needs of vertical fields. In the semi-autoregressive decoder, the translation sequence to be generated is divided into multiple groups of blocks, with an autoregressive method used within the block and a non-autoregressive method used outside the block. In the inference stage, information is extracted from the additional constraints provided by deleting erroneous words in historical translations, inserting prior knowledge terminology constraint words, predicting reasonable translation words and retaining the mechanism, and then fusing external prior knowledge. The present invention can not only flexibly incorporate external prior knowledge and adapt to different translation models in multiple fields, but also greatly improve the decoding rate compared to the autoregressive translation method.

Description

Technical Field

[0001] This invention relates to a low-resource machine translation method that utilizes semi-autoregressive incorporation of domain terminology. By using semi-autoregressive decoding in neural machine translation, this method achieves a machine translation technology that can flexibly incorporate prior knowledge from specific domains while significantly improving decoding speed. This technology belongs to the field of natural language processing and machine translation technology. Background Art

[0002] In recent years, neural machine translation (NMT) has developed rapidly, achieving remarkable results. However, as a data-driven training method, NMT technology has limited translation effectiveness in low-resource environments. Low resources refer to the limited availability and low quality of parallel corpora in specific domains or language categories. Because translation needs vary across different fields, training translation models for specific application scenarios often requires data from those domains. However, obtaining parallel corpora for specific domains is often difficult. Furthermore, most current neural machine translation methods, whether based on recurrent neural networks (RNNs) or attention mechanisms, employ autoregressive (AR) translation methods. These methods rely on historical information to generate the translation during the decoding phase, making the decoding process non-parallel and resulting in slow translation efficiency.

[0003] Integrating prior knowledge into NMT models is crucial. It can guide the entire NMT learning process and bring significant application value to the industry. Typically, annotated data, such as pre-prepared domain-specific terminology and phrase lists, is collectively referred to as prior knowledge. However, currently, there is no ideal method for incorporating domain-specific prior knowledge into NMT models. This is because, firstly, neural networks use continuous, weighted vectors to represent all linguistic structures involved in the translation process. While these vector representations can implicitly capture translation patterns, interpreting the hidden states of each layer in the neural network from a linguistic perspective remains challenging. Secondly, prior knowledge in NMT typically exists in the form of discrete symbols, making it difficult to convert this discrete prior knowledge into the continuous form required by neural networks. Since neural networks typically make strong independence assumptions between hidden states, simply modifying the model structure is also difficult to achieve. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies in the existing technology and to solve the problem of machine translation in vertical fields, and to creatively propose a low-resource machine translation method that utilizes semi-autoregressive fusion of domain terminology.

[0005] The innovation of this invention lies in: a low-resource machine translation method that incorporates domain-specific terminology, uses external knowledge to guide the decoder, and combines autoregressive and non-autoregressive decoding methods. This approach incorporates prior knowledge without sacrificing decoding efficiency, thus meeting the translation needs of specific domains. In the semi-autoregressive decoder, the target translation sequence is divided into multiple blocks, with an autoregressive approach used within the blocks and a non-autoregressive approach used outside the blocks. During the inference phase, information is extracted from the additional constraints provided by deleting incorrect words from historical translations, inserting prior knowledge terminology constraints, and predicting and retaining reasonable translation words, thereby integrating external prior knowledge.

[0006] The present invention is implemented by adopting the following technical solutions.

[0007] A low-resource machine translation method using semi-autoregressive fusion of domain terminology, comprising the following steps:

[0008] Step 1: Construct a decoding method based on a semi-autoregressive model to generate sequences in a semi-autoregressive manner.

[0009] Step 2: Build a terminology library that integrates specific fields, by deleting incorrect words in historical translations, inserting prior knowledge terminology constraints, and predicting and retaining reasonable translation words, so as to integrate external knowledge into the translation sentences.

[0010] Step 3: Use a sentence-level knowledge distillation method, taking advantage of the autoregressive model's hints during training, to allow the non-autoregressive model to learn the distribution of latent variables and attention of the autoregressive model, thereby enhancing the model's capabilities and improving translation results.

[0011] Beneficial effects

[0012] Compared with the prior art, the present invention has the following advantages:

[0013] 1. This invention combines the decoding concepts of autoregressive (AR) and non-autoregressive (NAR) methods. During the training phase, the translation is divided into blocks and trained using a semi-autoregressive method, with autoregressive methods implemented within blocks and non-autoregressive methods implemented between blocks, making the model more flexible and efficient. During the inference phase, editing strategies such as deleting incorrect words in historical translations, inserting prior knowledge terminology constraints, and predicting and retaining reasonable translation words are adopted. This allows sequence optimization to support dynamic length changes and also improves the accuracy of sequence generation.

[0014] 2. This method splits the translation into multiple chunks and generates the chunks in parallel. Incorporating domain-specific terminology into the chunks improves the translation performance and decoding speed of the translation model in specific domains.

[0015] 3. When generating each word, this method not only relies on the words that have been generated in the block, but also relies on the words that have been generated in other blocks, which better captures the dependency relationship on the target language side.

[0016] 4. This invention creates a mechanism for deleting erroneous information in historical translations. If the model finds that the content of a certain block of translation has already been translated in other blocks, it can dynamically delete the block, giving the model the ability to recover from repeated translation errors that are prone to occur.

[0017] 5. This paper proposes an NMT model that integrates a domain-specific terminology library and achieves significant improvements in indicators such as decoding speed (Time(s)) and translation quality (BLEU score).

[0018] 6. This invention can not only flexibly incorporate external prior knowledge and adapt to different translation models in multiple fields, but also significantly improve the decoding rate compared to the autoregressive translation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of a decoding method for introducing prior knowledge using a semi-autoregressive method in the method of the present invention;

[0020] Figure 2 The following is a comparative histogram of autoregressive and non-autoregressive methods on two translation tasks (Tibetan to Chinese);

[0021] Figure 3 The figure is a comparative bar chart of autoregressive and non-autoregressive methods on two translation tasks (Chinese to Tibetan). DETAILED DESCRIPTION

[0022] The method of the present invention is further described in detail below with reference to the accompanying drawings.

[0023] A low-resource machine translation method using semi-autoregressive fusion of domain terminology, comprising the following steps:

[0024] Step 1: Construct a decoding method based on a semi-autoregressive model to generate sequences in a semi-autoregressive manner.

[0025] Specifically, the semi-autoregressive model constructed in the present invention is consistent with the Transformer on the encoder side, and decodes in a semi-autoregressive manner on the decoder side.

[0026] like Figure 1The following is the working principle of the semi-autoregressive decoder. When generating the translation, the decoder divides the translation into blocks and decodes it synchronously. For example, a translation sequence S is divided into different blocks S 1 ,S 2 ,…,S k Within a block, the next word is predicted by combining source text information, prior knowledge, and generated historical translations through autoregressive (AR) decoding. At each stage, the decoder generates a corresponding word or symbol for the incomplete block. Specifically, as shown in formula (1):

[0027]

[0028] Where P(y|x) represents the conditional probability, x represents the input sequence, and y represents the output sequence; represents the tth word or symbol in the i-th block; The historical translation has been generated for the i-th block; L is the total length of the block, and K represents the number of blocks.

[0029] Calculate the i-th block S i The predicted word or symbol As shown in formula (2):

[0030]

[0031] Where V represents the vocabulary, <bos>and <eos>Represent the start symbol and end symbol respectively; P() represents the probability distribution of the corresponding expression; argmax represents the parameter set that maximizes the probability.

[0032] when When S i The block begins decoding, allowing the insertion of a constraint term base;

[0033] when When S i The block is not complete, allowing decoding to continue;

[0034] When S i = <eos>When S i Block completed, stopping decoding.

[0035] When the predicted words in the entire decoding process are <eos>, when the historical information remains unchanged and reaches the maximum length, it means that the decoding of this sequence has been completed.

[0036] Step 2: Build a domain-specific terminology database to incorporate external knowledge into the translated sentences.

[0037] Specifically, during decoding, prior knowledge is incorporated into the block to generate a sequence in an autoregressive manner. This is achieved by incorporating prior knowledge and removing redundant words.

[0038] In the formal introduction of prior knowledge and decoding stage, through the sentence boundary y 0 = <s>< / s> Perform the first iteration and fill y with the target constraint before removing the wrong words in the historical translation 0 Sequence, the target sequence is optimized through editing iterations. <s> Indicates the beginning of a sentence.< / s> Indicates the end of a sentence.

[0039] Among them, the method of integrating prior knowledge is:

[0040] The input of the decoder is the source language word embedding uniform mapping z, z = f(x; θ enc ), f() represents the mapping function, x represents the source language input sequence, and the given n sets of prior knowledge, where z k =ε(x i ), t=1,2,...,T y , n=P1,P2,...,P n θ enc Represents the relevant parameters of the encoder; z k represents the mapping result of the kth block; ε(x i ) represents the step function; t represents the current time step; T x represents the length of the source language sequence, T y represents the length of the target language sequence; P n Represents the nth pair of prior knowledge.

[0041] Each pair of prior knowledge P j By different words or phrases j constitute, j = 1, 2, ..., n. Before the formal decoding, the decoder combines the source language information in advance, selects the corresponding target language from the prior knowledge base, and inserts it into the sequence to be generated y 0 ,y 0 = <s>P1,P2,...,P n < / s> .

[0042] The method for deleting redundant words is: during formal decoding, the deletion operation is performed based on the constraints, source text information, and generated historical translations.

[0043] If the scale of prior knowledge does not include the entire source text information, or the entire prior knowledge constraint is deleted during the deletion operation, there will be no constraints in the translation generation process or the final translation will not contain prior knowledge. To solve this problem, the present invention introduces a constraint mask to indicate the position of the constraint marker in the sequence, and stipulates that the constraint marker specified by the constraint mask is prohibited from being deleted. By randomly adding a constraint mask placeholder, the position of the constraint mask will be recalculated and updated at each iteration, and finally a suitable candidate text will be selected from the prior knowledge base to replace the corresponding constraint mask. Specifically, the following method can be used:

[0044] The introduction of constrained mask operation consists of two stages: constrained mask prediction and candidate text prediction.

[0045] In the constrained mask prediction stage, in each iteration, for the decoder input sequence y, y = <s>y1,y2,...,y n < / s> , the model will be in every possible slot in y (y i ,y i+1 ), using a binary classifier To predict whether to add a constraint mask, the constraint mask is recorded as <plh>, as shown in formula (3):

[0046]

[0047] in, represents the prediction result of the constrained mask predictor for the i-th word in the sequence y, θ represents the model parameters, softmax represents the classification function, and h i represents the hidden state of the i-th word, h i+1 represents the hidden state of the i+1th word, concat represents the concatenation function, and n is the length of the sequence y.

[0048] In the candidate text prediction stage, for each constraint mask predicted above, the model trains a character predictor and selects actual characters from the prior knowledge base to replace the placeholder, as shown in formula (4).

[0049]

[0050] in, represents the prediction result of the candidate text predictor for the position of the mask in the sequence y, θ represents the model parameters, softmax represents the classification function, and h i Represents the hidden state of the i-th word, y i is the i-th word in the sequence; is a logical symbol, meaning "arbitrary"; C is a parameter shared with the word embedding matrix.

[0051] Step 3: Use a sentence-level knowledge distillation method, taking advantage of the autoregressive model's hints during training, to allow the non-autoregressive model to learn the distribution of latent variables and attention of the autoregressive model, thereby enhancing the model's capabilities and improving translation results.

[0052] Currently, most non-autoregressive translation methods use knowledge distillation during the training phase, allowing a smaller student model to learn from a larger teacher model and extract effective information from the feature distribution. This paper, however, uses sentence-level knowledge distillation to improve the model's translation performance.

[0053] Specifically, first, a priori knowledge base is added to the original training corpus, and an autoregressive translation model is trained as a teacher model (the training method can adopt the term-constrained NMT method);

[0054] Then, the source text is translated and the translation y′ is obtained;

[0055] Finally, a non-autoregressive translation model is trained using the pseudo-parallel corpus (x, y′) (for details, see "Non-autoregressive Translation with Enhanced Decoder Input"), where x represents the source language input sequence. This not only incorporates prior knowledge into the training corpus but also reduces the problem of duplicate or missed translations that non-autoregressive translation methods often encounter in the original corpus.

[0056] The translation model uses a greedy search algorithm to generate multiple candidate translations through parallel decoding, and then selects the translation sequence with the highest probability as the final translation.

[0057] Example verification

[0058] To demonstrate the effectiveness of this invention, we conducted experiments on a large-scale Tibetan-Chinese parallel corpus dataset and compared it with a mainstream baseline model. The experimental process is as follows:

[0059] 1. Experimental Data

[0060] This example constructs a separate domain-specific terminology library, covering new Tibetan and Chinese professional terms from 2015 to the first half of 2021. After cleaning and deduplication, there are 30,300 Tibetan-Chinese term pairs. The CCMT2019 and JudCorpus Tibetan-Chinese parallel corpora are combined as training corpora, with the CCMT2019 corpus containing 147,434 sentence pairs and the JudCorpus corpus containing 163,000 sentence pairs. To ensure a fair comparison, the Chinese corpus is uniformly processed using the BPE method, and the Tibetan corpus is uniformly processed using the phonetic-verbal fusion method. The Tibetan and Chinese vocabularies are 40,000 phonetic-verbal fusions and 40,000 subwords, respectively, and both vocabularies are shared throughout the training process. This invention combines the Test2018 and JudDev datasets as the development set, while the test set uses the Test2017, Dev2017, and JudTest datasets.

[0061] 2. Experimental Setup

[0062] This embodiment improves the Transformer model-based semi-autoregressive translation system RecoverSAT1 developed by Ran et al., and the experimental parameters are set according to the method of Gu et al.

[17] . In the actual experiment, due to the small size of the corpus, the corresponding parameters are changed, where dmodel = 278, dhidden = 507, nlayer = 5, nhead = 2, and pdropout = 0.1. This embodiment uses a sequence-level distillation method, which not only simulates each word but also simulates the distribution of all input sentences. The teacher's output information is sampled using a beam search algorithm, and then the student is trained by cross entropy, where the beam search size is set to 5. An AR translation model is pre-trained, and then the encoder parameters of the network of the present invention are initialized and the corresponding parameters are shared. The entire experiment is trained on a platform with two GeForce GTX1080Ti and 11GB of video memory.

[0063] 3. Baseline Method

[0064] For the sake of comparison, in the two translation tasks of Tibetan to Chinese and Chinese to Tibetan, in addition to the typical AR translation method Transformer-based model (AT-FT), this embodiment also selected the corresponding baseline system. The NAR translation model (NAT-FT) first proposed by Jiatao Gu et al.; the NMT method (AT-FT+Term) using term constraint training proposed by Georgiana Dinu et al. The present invention analyzes the above models on 5 test sets in two fields from the dimensions of translation rate of new words and terms Term%, decoding speed Time (s), BLEU score, etc.

[0065] 4. Experimental Data Analysis

[0066] a. Analysis of term usage and decoding rate

[0067] Scores were assessed based on the translation rate Term% and decoding speed Time(s) of new terms, and the performance of the AR and NAR models after the introduction of prior knowledge was analyzed. New terminology libraries were combined at both ends of each test set, and common words were replaced with new terms through semantic similarity matching to construct a test set containing new terms. The translation rate Term% of new terms refers to the ratio of the number of accurately translated new terms in the translation to the total number of new terms in the source end of the corresponding test set; the decoding rate Time(s) refers to the average time consumed in decoding each sentence. Table 1 shows the translation rate Term% and decoding rate Time(s) of new terms in the five test sets on the two translation tasks.

[0068] Table 1 Translation rate Term% and decoding rate Time(s) of new terms in five test sets on two translation tasks

[0069]

[0070]

[0071] When analyzing the translation rates of new terms, the AT-FT+Term baseline method achieved a minimum translation rate of 86.45% and a maximum of 98.32% in the Tibetan-to-Chinese translation task, with an average of 90.83%. The proposed method achieved a minimum translation rate of 88.21% and a maximum of 99.98%, with an average of 92.58%. In the Chinese-to-Tibetan translation task, the AT-FT+Term baseline method achieved a minimum translation rate of 86.12% and a maximum of 96.73%, with an average of 90.56%. The proposed method achieved a minimum translation rate of 90.77% and a maximum of 99.95%, with an average of 94.55%. Compared to the baseline method, the proposed model not only improved the translation rate of new terms in the Tibetan-to-Chinese translation task, but also in the Chinese-to-Tibetan translation task. This demonstrates that combining NAR training with a priori knowledge of new terms in NMT methods not only improves overall translation quality but also effectively translates new terms in specific fields, demonstrating strong application value in the industry. When analyzing decoding rates, our method significantly improved decoding rates across all five test sets in both Tibetan-to-Chinese and Chinese-to-Tibetan translation tasks compared to the baseline method. For example, in Tibetan-to-Chinese translation, our method improved decoding rates by 24.13% across all five test sets compared to the baseline AT-FT+Term method; and in Chinese-to-Tibetan translation, our method improved decoding rates by 46.17% across all five test sets compared to the baseline ATFT+Term method. This results in an average improvement of at least 11.11% in decoding efficiency per sentence, representing a significant performance improvement that could significantly benefit related work.

[0072] b. Analysis of translation quality of autoregressive and non-autoregressive

[0073] This embodiment further analyzes the effect of the autoregressive (AR) decoding method and the non-autoregressive (NAR) decoding method on improving the translation quality. The specific experiments are shown in Table 2. This table statistics the BLEU values ​​of the method of the present invention and other three baseline methods on five test sets in two translation tasks. In order to ensure the comparability between the respective models, the present invention strictly controls the experimental data scale, granularity, experimental platform and other related parameters. From the BLEU values ​​of the experimental results alone, it can be analyzed that in the translation models that incorporate the prior knowledge new word term library and the translation models that do not incorporate the prior knowledge new word term library, the translation quality score of AR is equivalent to the translation quality score of NAR. In the results for the Tibetan-to-Chinese and Chinese-to-Tibetan translation tasks on the Test2017, JudTest, JudDev, Dev2017, and JudTest test sets, AT-FT+Term achieved higher BLEU scores than NAT-FT+Term, with an average BLEU reduction of 0.07. Conversely, in the results for the Tibetan-to-Chinese and Chinese-to-Tibetan translation tasks on other test sets, AT-FT+Term achieved lower BLEU scores than NAT-FT+Term, with an average BLEU improvement of 0.43. As can be seen, the reduction in performance for both translation tasks across the various test sets is far less than the improvement. This demonstrates that incorporating prior knowledge into non-autoregressive translation models effectively improves translation performance.

[0074] Table 2 BLEU values ​​of the proposed method and other three baseline methods on five test sets in two translation tasks

[0075]

[0076] In order to further analyze the differences between the two translation modes, this paper compares the differences in BLEU values ​​of the two translation frameworks on the two translation tasks. Figure 2 As shown. It can be seen that in the two translation tasks, the present invention significantly improves the performance of autoregressive and non-autoregressive translation models by incorporating prior knowledge new word terminology libraries. For example, in the Chinese-Tibetan translation model trained with autoregression, the AT-FT-Term method on the test set Test2017 improved the BLEU value by (43.37-42.84=0.53) compared with the AT-FT method; in the Tibetan-Chinese translation model trained with non-autoregression, the NAT-FT+Term method on the test set Test2018 improved the BLEU value by (20.66-17.86=2.80) compared with the NAT-FT method. In addition, in the Tibetan-Chinese translation task, the NAT-FT+Term model on the test set Dev2017 improved the BLEU value by (42.34-41.89=0.450) compared with the AT-FT-Term model.

[0077] Through different cross-comparison experiments, it is proved that in Tibetan-Chinese and Chinese-Tibetan translation tasks, the incorporation of prior knowledge and new word terminology database can indeed improve the translation model indicators. In fact, in 50% of the tests, the quality of non-autoregressive translation is higher than that of autoregressive translation.

[0078] c. The impact of the number of segments on the quality of generated translations

[0079] By setting different block numbers K values ​​for training, the optimal segmentation K value is finally selected. In the experiment, the K values ​​are set to 2, 5, 8, and 10 respectively, and the newly constructed Test2018 and JuicialDev containing new words and terms are used as test sets. The BLEU values ​​of the translations under different K values, the translation rate Term% of the new words and terms, and the decoding rate Time(s) are statistically analyzed. The experimental results are shown in Table 3. When the K value is set to 2, the corresponding BLEU score and the translation rate of the new words and terms reach the highest, but the decoding rate is slow. At this time, the BLEU value of the translation of the method proposed in this invention is very close to that of the translation of the autoregressive method, and the decoding rate is improved by 40.16% compared with the autoregressive method. When the K value is set to 10, the corresponding BLEU score and the translation rate of the new words and terms decrease, but the decoding rate reaches the fastest. At this time, under the premise that the BLEU score of the translation decreases by less than 2.4, the decoding rate is significantly improved compared with the autoregressive method.

[0080] Table 3 Test results when setting different block numbers K

[0081]

[0082] In summary, the K value is negatively correlated with the translation's BLEU score and the translation rate of new terms, and positively correlated with the decoding rate. Selecting a K value of 2 not only maintains the translation's BLEU score and the translation rate of new terms, but also effectively improves the decoding rate.

[0083] d. Analysis of translation sequence generation and refinement methods

[0084] Compared with the insertion and deletion mechanism of LevT, the present invention introduces mechanisms such as deleting erroneous words in historical translations, inserting prior knowledge terminology constraint words, and predicting reasonable translation words and retaining them. In addition, the present invention assists the non-autoregressive translation model through a sentence-level knowledge distillation method, which improves the efficiency of the model in acquiring information and effectively solves problems such as information asymmetry. During training, the Transformer model updates the representation of each word through each layer, which leads to a lack of flexibility in the decoding model. The LevT method edits the sentence through each layer during training, which can not only flexibly generate translation sequences, but also continuously refine the sequence and change information such as sequence length, effectively improving the performance of the machine translation model. Compared with the Transformer model, the method of the present invention generally achieves better translation quality and a significant improvement in computing speed.

[0085] Although most of the existing NMTs can achieve satisfactory results in general fields, they all require large-scale parallel corpora in order to better train and achieve the best results. At this time, it is more important to be able to incorporate prior knowledge to meet the translation needs of specific fields with scarce resources. In addition, the current NMT system pays attention to building it with the AR translation model as the core framework. However, in the AR translation model, since decoding cannot be parallel, the efficiency of generating translations is limited. Therefore, by modifying the AR translation method, constructing a semi-autoregressive translation framework has higher application value. Based on the two problems mentioned above, the present invention proposes to build an NMT model by integrating a specific field term library.

[0086] The present invention compares the AR and NAR translation methods and adopts a semi-autoregressive method in the decoder in combination with the Transformer model. Specifically, the semi-autoregressive decoding method is implemented by block division, and the translation is generated by block division and synchronous decoding. At this time, the NAR method is used in parallel between each block, and the AR method is used in series within each block. Prior knowledge is integrated into the block. By adding mechanisms such as deleting erroneous information in historical translations, inserting prior knowledge terminology constraints, and predicting and retaining reasonable translation terms, an NMT model that integrates a specific domain terminology library is implemented. Finally, through experiments, it is compared with three baseline methods on five test sets of two translation tasks, and indicators such as the BLEU value of the translation, the translation rate of the terminology, and the decoding rate are analyzed. It can be seen that this method has significantly improved in all three indicators compared with other baseline methods. At the same time, without increasing the computational complexity, external discrete prior knowledge is effectively integrated. In addition, in order to ensure the accurate value of the number of blocks K, the method of the present invention separately sets K values ​​to 2, 5, 8, and 10, and tests the corresponding translation quality, terminology translation rate, and decoding rate. When the K value is 2, it can effectively translate new words and terms in specific fields while ensuring translation efficiency, and has strong application value in the industry.< / plh> < / eos> < / eos> < / eos> < / bos>

Claims

1. A low-resource machine translation method using semi-autoregressive fusion of domain terminology, characterized in that: The following steps are involved: Step 1: Construct a decoding method based on a semi-autoregressive model to generate sequences in a semi-autoregressive manner; The semi-autoregressive model constructed in step 1 is consistent with the Transformer on the encoder side, and decoded using a semi-autoregressive approach on the decoder side; When generating the translation, the decoder divides the translation into blocks and decodes them synchronously: A translation sequence S is divided into different blocks S 1 ,S 2 ,…,S k , within the block, the next word is predicted by combining the source text information, prior knowledge and the generated historical translation through autoregressive decoding. At each stage, the decoder generates a corresponding word or symbol for the incomplete block, as shown in formula (1): Among them, P(y|x) represents the conditional probability, x represents the input sequence, and y represents the output sequence; represents the tth word or symbol in the i-th block; The historical translation has been generated for the i-th block; L is the total length of the block, and K represents the number of blocks; Calculate the i-th block S i The predicted word or symbol As shown in formula (2): Where V represents the vocabulary, <bos>and <eos> Represent the start symbol and the end symbol respectively; P() represents the probability distribution of the corresponding expression; argmax represents the parameter set that maximizes the probability;< / eos> < / bos> when , indicating S i The block begins decoding, allowing the insertion of a constraint term base; when When S i The block is not complete, allowing decoding to continue; When S i = <eos>When S i The block is complete, stop decoding;< / eos> When the predicted words in the entire decoding process are <eos> ,When the historical information remains unchanged and reaches the maximum length, it means that the decoding of this sequence has been completed;< / eos> Step 2: Build a domain-specific terminology database, which integrates external knowledge into the translation sentences by removing incorrect words from historical translations, inserting prior knowledge terminology constraints, and predicting and retaining reasonable translation words. Step 3: Use a sentence-level knowledge distillation method to leverage the autoregressive model’s hints during training, allowing the non-autoregressive model to learn the distribution of the autoregressive model’s latent variables and attention. The translation model uses a greedy search algorithm to generate multiple candidate translations through parallel decoding, and then selects the translation sequence with the highest probability as the final translation.

2. The low-resource machine translation method using semi-autoregressive fusion of domain terminology according to claim 1, characterized in that: In step 2, when the prior knowledge is formally introduced and the decoding stage is performed, the sentence boundary y 0 = <s>< / s> Perform the first iteration and fill y with the target constraint before removing the wrong words in the historical translation 0 Sequence, the target sequence is optimized by editing iterations, where, <s> Indicates the beginning of a sentence.< / s> Indicates the end of a sentence; Among them, the method of integrating prior knowledge is: The input of the decoder is the source language word embedding uniform mapping z, z = f(x; θ enc ), f() represents the mapping function, x represents the source language input sequence, and the given n sets of prior knowledge, where z k =ε(x i ), t=1,2,...,T y ,n=P1,P2,…,P n θ enc Represents the relevant parameters of the encoder; z k Represents the mapping result of the kth block; ε(x i ) represents the step function; t represents the current time step; T x represents the length of the source language sequence, T y represents the length of the target language sequence; P n represents the nth pair of prior knowledge; Each pair of prior knowledge P j By different words or phrases j constitute, Before formal decoding, the decoder combines the source language information in advance, selects the corresponding target language from the prior knowledge base, and inserts the sequence to be generated y 0 , The method for deleting redundant words is: during formal decoding, the deletion operation is performed based on the constraints, source text information, and generated historical translations.

3. The low-resource machine translation method using semi-autoregressive fusion of domain terminology according to claim 1, characterized in that: In step 2, a constraint mask is introduced to indicate the position of the constraint marker in the sequence, and it is prohibited to delete the constraint marker specified by the constraint mask; by randomly adding a constraint mask placeholder, the position of the constraint mask is recalculated and updated in each iteration, and finally a suitable candidate text is selected from the prior knowledge base to replace the corresponding constraint mask.

4. The low-resource machine translation method using semi-autoregressive fusion of domain terminology as claimed in claim 3, characterized in that: The introduction of constrained mask operation includes two stages: constrained mask prediction and candidate text prediction; In the constrained mask prediction stage, in each iteration, for the decoder input sequence y, y = <s>y1,y2,…,y n < / s> ,in, <s> Indicates the beginning of a sentence.< / s> Indicates the end of the sentence, the model will find the end of the sentence in every possible slot in y (y i ,y i+1 ), using a binary classifier To predict whether to add a constraint mask, the constraint mask is recorded as <plh> , as shown in formula (3):< / plh> in, represents the prediction result of the constrained mask predictor for the i-th word in the sequence y, θ represents the model parameters, softmax represents the classification function, and h i represents the hidden state of the i-th word, h i+1 represents the hidden state of the i+1th word, concat represents the concatenation function, and n is the length of the sequence y; In the candidate text prediction stage, for each constraint mask predicted above, the model trains a character predictor to select actual characters from the prior knowledge base to replace the placeholder, as shown in formula (4): in, represents the prediction result of the candidate text predictor for the position of the mask in the sequence y, θ represents the model parameters, softmax represents the classification function, and h i Represents the hidden state of the i-th word, y i is the i-th word in the sequence; is a logical symbol, meaning "arbitrary"; C is a parameter shared with the word embedding matrix.

5. The low-resource machine translation method using semi-autoregressive fusion of domain terminology according to claim 1, characterized in that: In step 3, first, a priori knowledge base is added to the original training corpus, and an autoregressive translation model is trained as the teacher model; Then, the source text is translated and the translation y' is obtained; Finally, a non-autoregressive translation model is trained using the pseudo-parallel corpus (x, y'), where x represents the input sequence of the source language.

Images