Long text translation method and device, equipment, storage medium and product
By using a neural network model based on a decoder architecture and a generative training method, and training with parallel corpus pairs, combined with a relative position encoding mechanism, the problem of poor translation performance for long texts was solved, and efficient translation of long texts was achieved.
Patent Information
- Application Number
- CN202411110934.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-26
AI Technical Summary
Existing translation models cannot translate long texts correctly, resulting in poor translation performance, especially in the translation of long texts in low-resource, less common languages. Traditional sequence-to-sequence models are prone to memory shortages during training.
A neural network model based on a decoder architecture is adopted. It is trained using parallel corpus pairs through generative training, the encoder is removed, and the parallel corpus pairs are concatenated to predict the next character for translation. The relative position encoding mechanism is combined to expand the window size in the inference stage.
It effectively reduces the limitations on the length of text processed by the translation model, improves the translation effect of long texts, especially in the translation of long texts in low-resource minority languages, and improves the accuracy and stability of the translation.
Smart Images

Figure CN119026610A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a long text translation method, apparatus, device, storage medium and product. Background Art
[0002] With the development of computer technology and globalization, the application scope of computer program products has become more and more extensive, and the demand for cross-language services has also increased. When providing services to users in different countries and languages, especially in a multilingual environment, text translation has become an important part of ensuring user experience.
[0003] Existing translation models are mainly based on sequence-to-sequence (seq2seq) structures, which are mainly used for sentence-level translation. Traditional sequence-to-sequence translation models require the input of a source language, which is then encoded by an encoder and output by a decoder in the target language. Under this model structure, the text length of the input and output of the translation model needs to be set in advance, otherwise it is easy to cause insufficient video memory during the training process, resulting in the model being unable to correctly translate long texts and poor translation results for long texts. Summary of the invention
[0004] The embodiments of the present application provide a long text translation method, device, equipment, storage medium and product to solve the technical problem in the related art that the translation model cannot correctly translate the long text and the translation effect of the long text is poor, which can effectively improve the translation effect of the long text.
[0005] In a first aspect, an embodiment of the present application provides a long text translation method, comprising:
[0006] Get the text to be translated;
[0007] Inputting the text to be translated into a trained translation model, processing the text to be translated by the translation model and outputting a target translation text of the text to be translated;
[0008] The translation model is based on a neural network model of a decoder architecture and is trained using parallel corpus pairs in a generative training manner. The parallel corpus pairs are obtained by concatenating original training corpus and target training corpus.
[0009] In a second aspect, an embodiment of the present application provides a long text translation device, including a text acquisition module and a translation processing module, wherein:
[0010] The text acquisition module is configured to acquire the text to be translated;
[0011] The translation processing module is configured to input the text to be translated into a trained translation model, process the text to be translated through the translation model and output a target translation text of the text to be translated;
[0012] The translation model is based on a neural network model of a decoder architecture and is trained using parallel corpus pairs in a generative training manner. The parallel corpus pairs are obtained by concatenating original training corpus and target training corpus.
[0013] In a third aspect, an embodiment of the present application provides a long text translation device, including: a memory and one or more processors;
[0014] The memory is used to store one or more programs;
[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the long text translation method as described in the first aspect.
[0016] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the long text translation method as described in the first aspect when executed by a computer processor.
[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device performs the long text translation method as described in the first aspect.
[0018] In the embodiment of the present application, a neural network model based on a decoder architecture is used to obtain a target translation text by using a translation model trained using parallel corpus pairs in a generative training manner, wherein the parallel corpus pairs are obtained by concatenating the original training corpus and the target training corpus. After generative training, the translation model based on the decoder architecture can predict the target translation text by using the translation model to predict the next character of the text to be translated, which can effectively reduce the limitation on the text processing length of the translation model and effectively improve the translation effect of long texts. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a flow chart of a long text translation method provided in an embodiment of the present application;
[0020] Figure 2 It is a schematic diagram of a training process of a translation model provided in an embodiment of the present application;
[0021] Figure 3 It is a schematic diagram of a training corpus preparation process provided in an embodiment of the present application;
[0022] Figure 4 It is a structural schematic diagram of a long text translation device provided in an embodiment of the present application;
[0023] Figure 5 It is a structural schematic diagram of a long text translation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.
[0025] The long text translation method provided in the present application can be applied to the translation processing of long texts, such as the translation of long texts in minority languages. It aims to translate the text to be translated into the target translated text by using a translation model trained by parallel corpus pairs in a generative training manner through a neural network model based on a decoder architecture.
[0026] In the existing text translation model, it is generally concentrated on high-resource languages such as Chinese and English, and the training of these text translation models usually relies on high-quality parallel corpora. However, for low-resource minority languages, the long text data of low-resource minority languages is less, and it is difficult to train a reliable minority language translation model. In addition, the traditional text translation model is generally based on the neural network model of sequence to sequence (seq2seq) architecture to build and obtain, and the traditional text translation model is based on the encoder-decoder (encoder-decoder) structure to process data, wherein the encoder encodes the input data (the source text corresponding to the original language or the result of pre-processing the source text), and the decoder outputs the target text corresponding to the target language, and this model structure needs to set the length of the input and output text in advance during training, otherwise it is easy to have insufficient video memory during the training process, resulting in the situation of training failure, and this training mechanism will cause greater restrictions on the text processing length of the translation model, resulting in the existing text translation model being unable to translate long text. Based on this, a long text translation method of an embodiment of the present application is provided to solve the technical problem that the existing translation model cannot correctly translate long text and the translation effect of long text is poor.
[0027] Figure 1 A flowchart of a long text translation method provided in an embodiment of the present application is given. The long text translation method provided in an embodiment of the present application can be executed by a long text translation device, which can be implemented by hardware and / or software and integrated in a long text translation device.
[0028] The following description is made by taking a long text translation device executing a long text translation method as an example. Figure 1 , the long text translation method includes:
[0029] S110: Obtain the text to be translated.
[0030] When translation between set language types is required, obtain the text to be translated. Translation between set language types can be understood as text translation between the first language type and the second language type of the text to be translated. The text to be translated can be a text of the first language type or a text of the second language type. For example, when the text to be translated is a text of the first language type, translate the text to be translated into a text of the second language type (in this case, the first language and the second language are the original language and the target language, respectively). When the text to be translated is a text of the second language type, translate the text to be translated into a text of the first language type (in this case, the second language and the first language are the original language and the target language, respectively).
[0031] Optionally, the original language or target language may be a minority language with relatively few users, such as Arabic, Bengali, etc., and the target language or original language may be a language with relatively many users, such as Chinese, English, etc. In one embodiment, the text to be translated provided by the present solution may be obtained by text input or by text recognition processing of input speech.
[0032] S120: Input the text to be translated into the trained translation model, process the text to be translated through the translation model and output a target translation text of the text to be translated.
[0033] This solution uses a trained translation model to translate the text to be translated and outputs the corresponding target translation text. The translation model provided by this solution is obtained by training a neural network model based on a decoder architecture (such as an encoder-only architecture, a decoderonly architecture) using parallel corpus pairs in a generative training manner.
[0034] The parallel corpus pair provided by this scheme is obtained by splicing the original training corpus and the target training corpus, that is, the parallel corpus pair records the original training corpus and the corresponding target training corpus at the same time. In a parallel corpus pair, the corresponding original training corpus and the target training corpus are training corpora pointing to the same or similar meanings. The original training corpus corresponds to the original language, and the target training corpus corresponds to the target language. The parallel corpus pair can be recorded by including texts of the original training corpus and the target training corpus at the same time. In the process of training the translation model using parallel corpus pairs according to the generative training method, the translation model predicts the next character each time, so that the translation model can correctly learn the processing process from the input original training corpus to the output target training corpus, and there are fewer restrictions on the length of text processing.
[0035] Exemplarily, after obtaining the text to be translated, the text to be translated is input into a trained translation model, so that the translation model processes the text to be translated and outputs a target translation text in which the text to be translated is translated from the original language to the target language.
[0036] In one possible embodiment, Figure 2 As shown in the diagram of a training process of a translation model provided, the training steps of the translation model provided in this solution include:
[0037] S101: Acquire multiple original training corpora and target training corpora corresponding to each original training corpus, and concatenate each original training corpus with the corresponding target training corpus to obtain multiple parallel corpus pairs.
[0038] Exemplarily, multiple original training corpora for training the translation model are obtained, and target training corpora corresponding to these original training corpora are obtained. For example, each original training corpus can be translated (from the original language to the target language) to obtain the target training corpora corresponding to each original training corpus. Optionally, the original training corpus can be obtained by crawling data from websites corresponding to the original language (such as novel websites, news websites, etc.), or from open source databases (such as wiki databases), text data (such as novel data, book data, etc.). Optionally, each original training corpus can be a word, a phrase, etc.
[0039] In one embodiment, after obtaining a plurality of original training corpora and target training corpora corresponding to each original training corpora, the original training corpora and target training corpora with corresponding relationships are spliced to obtain parallel corpus pairs corresponding to each original training corpora. Optionally, the splicing position of the original training corpora can be before the target training corpora or after the target training corpora.
[0040] In one embodiment, this scheme concatenates each original training corpus with the corresponding target training corpus to obtain multiple parallel corpus pairs. It can be that each original training corpus and the corresponding target training corpus are concatenated and the corresponding original training corpus and the target training corpus are separated by a preset separator to obtain multiple parallel corpus pairs.
[0041] Exemplarily, the original training corpus and the corresponding target training corpus are concatenated using a preset separator, so that the separator separates the corresponding original training corpus and the target training corpus, and a parallel corpus pair including the separator separating the corresponding original training corpus, the target training corpus and the separator is obtained. Optionally, the original training corpus can be before the separator or after the separator.
[0042] Optionally, the separator can be "[SEP]", the original training corpus is identified as "source text", and the target training corpus is identified as "target text", then the corresponding parallel corpus pair can be represented as "[source text][SEP][targettext]". For example, the original training corpus is the English word "Hello", and the target training corpus is the Arabic word Then the corresponding parallel corpus pair can be expressed as By dividing the original training corpus and the target training corpus by delimiters, the translation model can correctly identify the original text and the target text during training and inference, thereby improving the accuracy of long text translation.
[0043] In one embodiment, the long text translation method provided by the present solution inputs the text to be translated into a trained translation model, processes the text to be translated through the translation model and outputs a target translation text of the text to be translated. A preset separator may be added to the text to be translated to obtain an intermediate processing text, and the intermediate processing text is input into the trained translation model, and the intermediate processing text is processed through the translation model and outputs a target translation text of the text to be translated.
[0044] Exemplarily, after obtaining the text to be translated, a preset separator is added to the text to be translated to obtain an intermediate processed text, and the intermediate processed text is processed using the translation model and a target translation text of the text to be translated is output. The adding of the preset separator to the text to be translated to obtain the intermediate processed text may be performed before inputting the text into the translation model or in the translation model.
[0045] For example, assuming the text to be translated is “Text to be translated”, the intermediate processed text for a may be: “[Text to be translated][SEP]”. The translation model may predict the subsequent target translation text in the corresponding target language based on the intermediate processed text “[Text to be translated][SEP]”. For example, the corresponding text in Arabic to “Text to be translated” in English is During the inference process of the translation model, only the intermediate processing text [source text][SEP] needs to be input into the translation model, and the decoder will complete the subsequent content, so that the translation model can learn the translation mode during training, and can effectively use the intermediate input text and the generated partial target results to assist in the prediction of the next word (token) during the inference stage. This solution obtains the intermediate processing text by adding preset delimiters to the text to be translated, so that the translation model can smoothly predict the corresponding words of the subsequent target language according to the text format of the parallel corpus pairs obtained based on splicing during the training process, and can correctly process the intermediate processing text and output the target translation text.
[0046] In one possible embodiment, Figure 3 As shown in the schematic diagram of a training corpus preparation process provided, this solution obtains multiple original training corpora and target training corpora corresponding to each original training corpus, including:
[0047] S1011: Acquire multiple first original corpora, and translate each first original corpus to obtain a corresponding first target corpus.
[0048] S1012: Concatenate the first original corpus and the first target corpus according to one or more preset concatenation lengths to obtain a plurality of second original corpora and a second target corpus, and obtain a plurality of original training corpora and corresponding target training corpora based on the first original corpus, the second original corpus, the first target corpus and the second target corpus.
[0049] Exemplarily, multiple first original corpora are obtained, and translation tools (such as Google Translate, Baidu Translate, Youdao Dictionary and other online and / or offline translation tools) are used to translate the multiple first original corpora to obtain first target corpora translated from the original language into the target language.
[0050] In one embodiment, the first original corpus and the first target corpus are spliced according to one or more set splicing lengths to obtain multiple second original corpora and second target corpora. After the multiple first original corpora are spliced according to the set splicing length to obtain the second original corpora, the multiple first target corpora that have a corresponding relationship with the multiple first original corpora (i.e., multiple first target corpora that express the same or similar meaning) are spliced to obtain the second target corpora. At this time, the second original corpora and the second target corpora have a corresponding relationship. For example, the splicing length is k=[32, 64, 128], that is, the splicing length can be 32, 64 and 128. Multiple first original corpora can be spliced according to the splicing length limit of 32 to obtain the corresponding second original corpora (for example, the length of the second original corpora is within 32), and the corresponding second target corpora are spliced according to the corresponding multiple first target corpora to obtain the corresponding second target corpora. After obtaining a set number of second original corpora with a splicing length of 32 and a second target corpus, the splicing process with splicing lengths of 64 and 128 is continued based on the above splicing method to obtain a combination of multiple first target corpora and second target corpora with different splicing lengths.
[0051] After obtaining multiple second original corpora and second target corpora, the first original corpora, the second original corpora, the first target corpora and the second target corpora are mixed, and the first original corpora, the second original corpora, the first target corpora and the second target corpora are used as multiple original training corpora for training processing of the translation model.
[0052] In a possible embodiment, after the first original corpus and the first target corpus are concatenated according to one or more preset concatenation lengths to obtain multiple second original corpora and second target corpora, the first original corpus, the second original corpus, the first target corpus, and the second target corpus can also be deduplicated. Exemplarily, the first original corpus, the second original corpus, the first target corpus, and the second target corpus are deduplicated, and when duplicate corpora are detected, one of the corpora is retained to reduce the occurrence of duplicate training corpora and improve the quality of training corpora.
[0053] This solution uses a back-translation multi-resolution training corpus generation method to obtain a large amount of long text parallel corpus in a single language, and then uses these long text parallel corpus to train the translation model, so that the model can obtain better long text translation capabilities. During the training process, the translation model can use short text fragments as an aid to better learn the translation of long texts. At the same time, using the first original corpus as the original corpus and the first target corpus as the translation generated corpus is equivalent to adding some noise to the first target corpus, which can effectively improve the robustness of the translation model training and improve the stability of the model translation.
[0054] S102: Train a neural network model based on a decoder-only architecture using parallel corpora in a generative training manner to obtain a translation model.
[0055] This solution builds a translation model through a neural network model based on a decoder-only architecture. Compared with the traditional translation model that adopts a sequence-to-sequence architecture (the traditional translation model includes an encoder and a decoder), the translation model provided by this solution is a decoder-only architecture, which removes the encoder in the traditional translation model. Only the decoder architecture can predict the next character each time during training. Even when the length of the inference text is longer than the length of the training text, the translation model can still guarantee a good translation level and can translate long texts more effectively.
[0056] Optionally, the translation model trained using the parallel corpus pairs may be a pre-trained decoder network model (for example, the pre-trained decoder network model may be a large language model such as Baichuan, LLama3, or Qwen).
[0057] In one embodiment, after obtaining a plurality of parallel corpus pairs, the parallel corpus pairs obtained above are used to train the neural network model based on the decoder-only architecture in a generative training manner until the number of training times reaches a set number of times and / or the model translation accuracy reaches a set requirement, thereby obtaining a trained translation model. This solution obtains a plurality of parallel corpus pairs by concatenating a plurality of original training corpora and corresponding target training corpora, and uses the parallel corpus pairs to train the neural network model based on the decoder-only architecture in a generative training manner to obtain a translation model, which can effectively reduce the limitation on the length of text processing by the translation model and effectively improve the translation effect of long texts.
[0058] In one possible embodiment, the long text translation method provided by the present solution uses parallel corpus pairs in a generative training manner to train a neural network model based on a decoder-only architecture to obtain a translation model. It can be based on a relative position encoding mechanism, and uses parallel corpus pairs in a generative training manner to train a neural network model based on a decoder-only architecture to obtain a translation model.
[0059] In the training of text translation models in related technologies, it is generally based on sinusoidal position coding (sinusoidal position coding) that uses sine and cosine functions to encode positions. The coding of each position consists of a set of sine and cosine functions, and the frequency changes logarithmically. This encoding method allows the model to capture the periodic changes of position information, but its encoding is absolute, that is, each position has a fixed encoding, which leads to a limit on the length of the model output text. This solution is based on a translation model with only a decoder architecture, which only predicts the next character each time during training (the existing sequence-to-sequence text translation model needs to predict the entire sentence). In the model reasoning stage, the relative position coding (YaRN, Yet Another RelativePosition) mechanism is combined to effectively solve the limitation of the translation model on the length of text output and improve the translation effect of long texts.
[0060] Exemplarily, based on the relative position encoding mechanism, a neural network model based on a decoder-only architecture is trained using parallel corpora in a generative training manner to obtain a translation model. Among them, the relative position encoding mechanism provided by this scheme uses the principle of rotational position encoding (RoPE) to rotate the word embedding at each position by a set angle, thereby encoding the position information. The inner product of two rotated word embeddings contains the relative position information of the two word embeddings. For example, for the word embedding at position m Positional word embeddings Based on the rotation angle θ, the two word embeddings are rotated respectively to obtain:
[0061]
[0062]
[0063] Calculating the inner product of two rotation vectors gives
[0064]
[0065] In the inner product result, we can see the relative position information of the two word embeddings, namely mn. Compared with the sine and cosine position encoding that only calculates the absolute position information, the rotation position encoding calculates the relative position information through the absolute position information. When processing long texts, it can better maintain the distance relationship and sequence information, so that the translation model can better handle the contextual relationship and output a more accurate target translation text.
[0066] Among them, in the inference stage of the rotation position coding mechanism, the position interpolation method can be used to scale the position coding, so that the rotation position coding mechanism has a better extrapolation effect. For the coding of different positions, the difference of the relatively close distance coding is mainly in the high-frequency area, and the difference of the relatively far distance coding is mainly in the low-frequency area. The relative position coding mechanism provided by this solution scales the position coding differently in different dimensions, for example, using the NKT-by-parts method to scale the position coding differently in different dimensions. First, calculate the wavelength corresponding to each dimension, where the wavelength of the dth dimension is:
[0067]
[0068] Among them, λ d is the wavelength of the dth dimension, θ d is the rotation angle of the dth dimension, b is the set reference value base, for example, base = 10000, and D is the total number of dimensions. When the wavelength is much smaller than the training sequence length (for example, a preset length of 512, 1024, 2048, or the parallel corpus is intercepted or padded to the preset length), the rotation angle of the dimension is scaled by a first scaling factor (for example, the first scaling degree is 1, that is, the rotation angle of the dimension is not adjusted). The wavelength is much smaller than the training sequence length, which can be a wavelength less than a set multiple (the set multiple can be 0-0.2 times, for example, 0.02 times) of the training sequence length, or a wavelength less than a set sequence length threshold. When the wavelength is equal to or greater than the training sequence length, the rotation angle of the dimension is scaled by a second scaling factor. When the wavelength is between the set multiple of the training sequence length or the set sequence length threshold, and the training sequence length, the rotation angle is adjusted between the first scaling factor and the second scaling factor.
[0069] Specifically, the ratio r(d) of the training sequence length to the wavelength of dimension d is calculated. When r(d) is less than the first reference value α, scaling is performed using the position interpolation method. When r(d) is greater than the second reference value β, no scaling is performed. Optionally, the scaling function of the rotation angle θ can be expressed as:
[0070]
[0071]
[0072] Among them, h(θ d ) is the scaling function of the rotation angle θ, s is the scaling factor s = L' / L, L' is the length of the sequence in the inference phase, L is the length of the sequence in the training phase, γ function is the ramp function, α is the preset first reference value, and β is the preset second reference value. By scaling different rotation angles to different degrees, the position encoding is scaled, and the translation model can effectively take into account the translation effects of both long and short texts.
[0073] In one embodiment, a temperature control factor can be added to the attention calculation of the model based on the relative position encoding YaRN to improve the translation effect. The temperature control factor can be expressed as:
[0074]
[0075] Where t is the temperature control factor, 1 / sqrt(t)=0.1ln(s)+1, q and k are the query matrix Query and key matrix Key corresponding to the self-attention mechanism in the translation model, is the transpose of the query matrix corresponding to position m, k n is the key matrix corresponding to the n position, and the temperature control factor t can be used to adjust the inner product result of the query matrix Query and the key matrix Key. Among them, in the traditional Decoder only model, the input length of the reasoning stage is consistent with the input length of the training stage. The role of Decoder only is mainly reflected in the output length of the reasoning stage. Because it is predicting the next character, if the end mark EOS (End of Sentence) is not output in the reasoning stage, the model can continue to output the next character. This solution implements the extension of the input length based on the relative position encoding YaRN, which enables the model to process longer input sequences and translate longer texts in the reasoning stage.
[0076] This solution is based on the relative position encoding mechanism. It uses parallel corpus to train a neural network model based on a decoder-only architecture in a generative training method to obtain a translation model. The relative position encoding mechanism is used to expand the window size in the reasoning stage, which alleviates the output length limitation in the sequence-to-sequence architecture of the existing translation model and can more effectively handle the needs of long text translation.
[0077] As mentioned above, through the neural network model based on the decoder architecture, the translation model obtained by training with parallel corpus pairs in a generative training manner translates the text to be translated to obtain the target translation text, wherein the parallel corpus pairs are obtained by concatenating the original training corpus and the target training corpus. After the generative training, the translation model based on the decoder architecture can predict the target translation text by using the translation model to predict the next character of the text to be translated, which can effectively reduce the restriction on the text processing length of the translation model and effectively improve the translation effect of long texts.
[0078] Figure 4 Schematic diagram of a long text translation device provided by an embodiment of the present application. Figure 4 The long text translation device includes a text acquisition module 41 and a translation processing module 42.
[0079] Among them, the text acquisition module 41 is configured to acquire the text to be translated; the translation processing module 42 is configured to input the text to be translated into a trained translation model, process the text to be translated through the translation model and output the target translation text of the text to be translated; wherein the translation model is a neural network model based on a decoder architecture, and is trained using parallel corpus pairs in a generative training manner, and the parallel corpus pairs are obtained by splicing the original training corpus and the target training corpus.
[0080] As mentioned above, through the neural network model based on the decoder architecture, the translation model obtained by training with parallel corpus pairs in a generative training manner translates the text to be translated to obtain the target translation text, wherein the parallel corpus pairs are obtained by concatenating the original training corpus and the target training corpus. After the generative training, the translation model based on the decoder architecture can predict the target translation text by using the translation model to predict the next character of the text to be translated, which can effectively reduce the restriction on the text processing length of the translation model and effectively improve the translation effect of long texts.
[0081] In a possible embodiment, a model training module is further included, and the model training module is configured to train the translation model. The steps of the model training module for training the translation model are configured as follows:
[0082] Acquire multiple original training corpora and target training corpora corresponding to each original training corpus, and concatenate each original training corpus with the corresponding target training corpus to obtain multiple parallel corpus pairs;
[0083] The neural network model based on the decoder-only architecture is trained using parallel corpus pairs in a generative training manner to obtain a translation model.
[0084] In a possible embodiment, the model training module concatenates each original training corpus with the corresponding target training corpus to obtain multiple parallel corpus pairs, including:
[0085] Each original training corpus is concatenated with the corresponding target training corpus, and the corresponding original training corpus and the target training corpus are separated by a preset separator to obtain a plurality of parallel corpus pairs.
[0086] In a possible embodiment, the model training module inputs the text to be translated into the trained translation model, processes the text to be translated through the translation model and outputs a target translation text of the text to be translated, including:
[0087] A preset separator is added to the text to be translated to obtain an intermediate processed text, and the intermediate processed text is input into the trained translation model. The intermediate processed text is processed by the translation model and a target translation text of the text to be translated is output.
[0088] In a possible embodiment, the model training module trains a neural network model based on a decoder-only architecture using parallel corpora in a generative training manner to obtain a translation model, including:
[0089] Based on the relative position encoding mechanism, a neural network model based on a decoder-only architecture is trained using parallel corpora in a generative training manner to obtain a translation model.
[0090] In a possible embodiment, the model training module obtains multiple original training corpora and target training corpora corresponding to each original training corpus, including:
[0091] Acquire multiple first original corpora, and translate each first original corpus to obtain a corresponding first target corpus;
[0092] The first original corpus and the first target corpus are concatenated according to one or more preset concatenation lengths to obtain a plurality of second original corpora and a second target corpus, and a plurality of original training corpora and corresponding target training corpora are obtained based on the first original corpus, the second original corpus, the first target corpus and the second target corpus.
[0093] In a possible embodiment, after the model training module concatenates the first original corpus and the first target corpus according to one or more preset concatenation lengths to obtain multiple second original corpora and second target corpora, it also deduplicates the first original corpus, the second original corpus, the first target corpus, and the second target corpus.
[0094] It is worth noting that in the above-mentioned embodiment of the long text translation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.
[0095] The embodiment of the present application also provides a long text translation device, which can integrate the long text translation apparatus provided in the embodiment of the present application. Figure 5 Schematic diagram of a long text translation device provided by an embodiment of the present application. Figure 5 The long text translation device includes: an input device 53, an output device 54, a memory 52 and one or more processors 51; the memory 52 is used to store one or more programs; when one or more programs are executed by one or more processors 51, the one or more processors 51 implement the long text translation method provided in the above embodiment. The long text translation device, equipment and computer provided above can be used to execute the long text translation method provided in any of the above embodiments, and have corresponding functions and beneficial effects.
[0096] The embodiment of the present application also provides a non-volatile storage medium storing computer executable instructions, which are used to execute the long text translation method provided in the above embodiment when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the long text translation method provided above, can also execute the related operations in the long text translation method provided in any embodiment of the present application. The long text translation device, equipment and storage medium provided in the above embodiment can execute the long text translation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiment, please refer to the long text translation method provided in any embodiment of the present application.
[0097] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the long text translation method provided in each embodiment of the present application.
Claims
1. A long text translation method, characterized in that: include: Get the text to be translated; Inputting the text to be translated into a trained translation model, processing the text to be translated by the translation model and outputting a target translation text of the text to be translated; The translation model is based on a neural network model of a decoder architecture and is trained using parallel corpus pairs in a generative training manner. The parallel corpus pairs are obtained by concatenating original training corpus and target training corpus.
2. The long text translation method according to claim 1, characterized in that: The training steps of the translation model include: Acquire multiple original training corpora and target training corpora corresponding to each of the original training corpora, and concatenate each of the original training corpora with the corresponding target training corpora to obtain multiple parallel corpus pairs; The parallel corpus pairs are used to train a neural network model based on a decoder-only architecture in a generative training manner to obtain a translation model.
3. The long text translation method according to claim 2, characterized in that: The step of concatenating each of the original training corpora with the corresponding target training corpora to obtain a plurality of parallel corpus pairs includes: Each of the original training corpora and the corresponding target training corpora are concatenated, and the corresponding original training corpora and the target training corpora are separated by a preset separator to obtain a plurality of parallel corpus pairs.
4. The long text translation method according to claim 3, characterized in that: The step of inputting the text to be translated into a trained translation model, processing the text to be translated by the translation model and outputting a target translation text of the text to be translated comprises: A preset separator is added to the text to be translated to obtain an intermediate processed text, and the intermediate processed text is input into a trained translation model, the intermediate processed text is processed by the translation model and a target translation text of the text to be translated is output.
5. The long text translation method according to claim 2, characterized in that: The method of training a neural network model based on a decoder-only architecture using the parallel corpus in a generative training manner to obtain a translation model includes: Based on the relative position encoding mechanism, the parallel corpus pairs are used to train a neural network model based on a decoder-only architecture in a generative training manner to obtain a translation model.
6. The long text translation method according to claim 2, characterized in that: The obtaining of a plurality of original training corpora and a target training corpus corresponding to each of the original training corpora includes: Acquire multiple first original corpora, and translate each of the first original corpora to obtain a corresponding first target corpus; The first original corpus and the first target corpus are concatenated according to one or more preset concatenation lengths to obtain a plurality of second original corpora and a second target corpus, and a plurality of original training corpora and corresponding target training corpora are obtained based on the first original corpus, the second original corpus, the first target corpus and the second target corpus.
7. The long text translation method according to claim 6, characterized in that: After the first original corpus and the first target corpus are spliced according to one or more preset splicing lengths to obtain a plurality of second original corpora and second target corpora, the method further includes: Deduplication processing is performed on the first original corpus, the second original corpus, the first target corpus, and the second target corpus.
8. A long text translation device, characterized in that: It includes a text acquisition module and a translation processing module, wherein: The text acquisition module is configured to acquire the text to be translated; The translation processing module is configured to input the text to be translated into a trained translation model, process the text to be translated through the translation model and output a target translation text of the text to be translated; The translation model is based on a neural network model of a decoder architecture and is trained using parallel corpus pairs in a generative training manner. The parallel corpus pairs are obtained by concatenating original training corpus and target training corpus.
9. A long text translation device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the long text translation method according to any one of claims 1 to 7.
10. A non-volatile storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the long text translation method according to any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the long text translation method according to any one of claims 1 to 7 is implemented.
