Translation post-editing method and apparatus, electronic device, and storage medium

Abstract

The embodiments of this disclosure relate to post-editing methods and apparatuses, electronic devices, and storage media, and pertain to the field of machine translation technology. The post-editing method includes: acquiring a target source text and a target machine-translated text, wherein the target machine-translated text is the machine-translated version of the target source text; inputting the target source text and the target machine-translated text into a pre-trained post-editing model, and correcting the target machine-translated text using the post-editing model; the correction methods of the post-editing model include deleting tokens, inserting placeholders, and replacing placeholders with tokens; and outputting the target post-edited text obtained by the post-editing model correcting the target machine-translated text.

Description

Technical Field

[0001] This disclosure relates to machine translation technology, and more specifically, to post-editing methods and apparatus, electronic devices, and storage media. Background Technology

[0002] Machine translation has wide applications in daily life and is an important research direction in the field of machine learning for language processing. After obtaining machine-translated text, users can further edit and adjust it using post-editing models. Currently, post-editing models suffer from significant time consumption and unsatisfactory editing results. Summary of the Invention

[0003] This disclosure provides post-editing methods and apparatus, electronic devices, and storage media to improve the accuracy of translated text.

[0004] According to a first aspect of the embodiments of this disclosure, a post-editing method is provided, the post-editing method comprising:

[0005] Obtain the target source text and the target machine-translated text, wherein the target machine-translated text is the machine-translated version of the target source text;

[0006] The target source text and the target machine-translated text are input into a pre-trained post-editing model, and the target machine-translated text is corrected by the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words;

[0007] Output the target post-edited text obtained by correcting the target machine-translated text using the post-editing model.

[0008] Optionally, the post-editing model includes an encoder network and a decoder network; the step of correcting the target machine-translated text using the post-editing model includes:

[0009] The target source text is encoded using the encoder network to obtain the encoded vector of the target source text.

[0010] The target machine-translated text is encoded using the decoder network to obtain an encoding vector for the target machine-translated text, and the target machine-translated text is corrected based on the encoding vector of the target source text and the encoding vector of the target machine-translated text.

[0011] Optionally, the decoder network includes an encoding unit, a decoding unit, a first prediction unit, a second prediction unit, and a third prediction unit;

[0012] The step of encoding the target machine-translated text using the decoder network to obtain the encoding vector of the target machine-translated text, and correcting the target machine-translated text based on the encoding vector of the target source text and the encoding vector of the target machine-translated text, includes:

[0013] The target machine-translated text is encoded using the encoding unit to obtain the encoded vector of the target machine-translated text.

[0014] The decoding unit performs matrix operations on the encoding vector of the target source text and the encoding vector of the target machine-translated text to obtain the attention distribution of the target machine-translated text based on the target source text.

[0015] The first prediction unit predicts whether to delete word units in the target machine-translated text based on the attention distribution of the target source text.

[0016] The second prediction unit predicts the number of placeholders that need to be inserted between words in the target machine-translated text, where the number is an integer greater than or equal to zero.

[0017] The third prediction unit predicts the lexical unit that will replace the placeholder.

[0018] Optionally, before inputting the target source text and the target machine-translated text into the pre-trained post-editing model, the method further includes:

[0019] Multiple sample data combinations are obtained, each sample data combination including sample source text, first sample translated text and second sample translated text, the first sample translated text is the machine translated text of the sample source text, and the second sample translated text is the post-edited text of the sample source text;

[0020] The post-editing model is trained using the combination of the multiple sample data.

[0021] Optionally, before acquiring the combination of multiple sample data, the method further includes:

[0022] Obtain bilingual text corpus, wherein the bilingual text corpus includes first language text and second language text corresponding to the same content;

[0023] The first language text is used as the sample source text in the sample data combination;

[0024] The second language text is used as the second sample translation text in the sample data combination;

[0025] Randomly replace and / or delete word units in the second language text to obtain the first sample translation text in the sample data combination.

[0026] Optionally, before acquiring the combination of multiple sample data, the method further includes:

[0027] Obtain bilingual text corpus, wherein the bilingual text corpus includes first language text and second language text corresponding to the same content;

[0028] The first language text is used as the sample source text in the sample data combination;

[0029] The second language text is used as the second sample translation text in the sample data combination;

[0030] The parameters of the pre-trained translation model are adjusted, the first language text is input into the translation model, the translation model is used to translate the first language text to obtain the translation result, and the translation result is used as the first sample translation text in the sample data combination.

[0031] Optionally, training the post-editing model using the combination of the multiple sample data includes:

[0032] The multiple sample data are combined into n training batches, and the unreliability probability of each training batch is determined.

[0033] If the unreliability probability of the i-th training batch is greater than or equal to a preset threshold, a first mode is used for training each of the sample data combinations in the i-th training batch; if the unreliability probability of the i-th training batch is less than the preset threshold, a second mode is used for training each of the sample data combinations in the i-th training batch; the i-th training batch is any one of the n training batches.

[0034] When training with the first mode for each sample data combination in the i-th training batch, the (j-1)-th word of the second sample translated text in the sample data combination is used as the next input of the post-editing model to predict the j-th word; when training with the second mode for each sample data combination in the i-th training batch, the (j-1)-th word predicted by the post-editing model is used as the next input of the post-editing model to predict the j-th word; where j is an integer and j≥2.

[0035] Optionally, the untrust probability of the i-th training batch is determined according to the following formula:

[0036] ∈ i=max(∈ min ,KC*i)

[0037] Where, ∈ i Let K be the unreliable probability of the i-th training batch, max() be the maximum value function, and K and C be positive constants. min The minimum unreliable probability is the preset value, ∈ min ≥0.

[0038] According to a second aspect of the embodiments of the present disclosure, a post-editing apparatus is provided, the post-editing apparatus comprising:

[0039] The acquisition module is used to acquire the target source text and the target machine-translated text, wherein the target machine-translated text is the machine-translated text of the target source text;

[0040] The correction module is used to input the target source text and the target machine-translated text into a pre-trained post-editing model, and to correct the target machine-translated text through the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words;

[0041] The output module is used to output the target post-edited text obtained by the post-editing model after correcting the target machine-translated text.

[0042] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including a processor and a memory, the memory storing computer instructions that, when executed by the processor, implement the post-editing method of the first aspect of the present disclosure.

[0043] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, implement the post-editing method of the first aspect of the present disclosure.

[0044] The post-editing method, apparatus, electronic device, and storage medium provided in this disclosure input the target source text and its machine-translated text into a pre-trained post-editing model. The machine-translated text of the target source text is then corrected by the post-editing model. In the correction process, the model mimics the human modification methods of deleting and inserting words, thereby improving the accuracy of the translation results.

[0045] The features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the embodiments of the present disclosure.

[0047] Figure 1 This is a schematic diagram of the structure of an electronic device that can be used to implement the embodiments of this disclosure, according to one embodiment of the present disclosure.

[0048] Figure 2 This is a flowchart of a post-translation editing method provided in one embodiment of this disclosure;

[0049] Figure 3 This is a schematic diagram of the architecture of a post-editing model provided in one embodiment of this disclosure;

[0050] Figure 4 This is a schematic diagram of the prediction process of a post-editing model provided in one embodiment of this disclosure;

[0051] Figure 5-6 This is a flowchart of the training process of a post-editing model provided in one embodiment of this disclosure;

[0052] Figure 7 This is a schematic diagram of the training process of a post-editing model provided in one embodiment of this disclosure;

[0053] Figure 8 This is a schematic diagram illustrating the generation of sample data provided in one embodiment of this disclosure;

[0054] Figure 9 This is a block diagram of a post-editing apparatus provided in one embodiment of this disclosure;

[0055] Figure 10 This is a schematic diagram of the structure of a post-editing device provided in one embodiment of this disclosure. Detailed Implementation

[0056] Various exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0057] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this disclosure or their application or use.

[0058] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0059] The translation scheme provided by the embodiments of this disclosure can incorporate user modifications to the translation during the machine-generated translation process, organically combining human translation and machine translation to improve translation accuracy.

[0060] <Hardware Configuration>

[0061] Figure 1 This is a schematic diagram of the structure of an electronic device that can be used to implement embodiments of the present disclosure. This electronic device can be used to implement the post-translation editing method of the embodiments of the present disclosure.

[0062] The electronic device 1000 can be a smartphone, laptop, desktop computer, tablet computer, server, etc., and is not limited thereto.

[0063] The electronic device 1000 may include, but is not limited to, a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, etc. The processor 1100 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MCU), etc., used to execute computer programs / instructions, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, SSE, etc. The memory 1200 may include, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. The interface device 1300 may include, for example, a USB interface, a serial interface, a parallel interface, etc. The communication device 1400 may be capable of wired communication using fiber optic cables or cables, or wireless communication, specifically including WiFi communication, Bluetooth communication, 2G / 3G / 4G / 5G communication, etc. The display device 1500 may be, for example, an LCD screen, a touch screen, etc. The input device 1600 may include, for example, a touch screen, a keyboard, motion input, etc. Speaker 1700 is used to output audio signals. Microphone 1800 is used to capture audio signals.

[0064] In the embodiments of this disclosure, the memory 1200 of the electronic device 1000 is used to store computer programs / instructions that control the processor 1100 to operate in order to implement the post-editing method according to the embodiments of this disclosure. Those skilled in the art can design such computer programs / instructions based on the scheme disclosed in this disclosure. How the computer program / instructions control the processor to operate is well known in the art and will not be described in detail here. The electronic device 1000 may be equipped with a smart operating system (e.g., Windows, Linux, Android, iOS, etc.) and application software.

[0065] Those skilled in the art should understand that, although in Figure 1 The present invention illustrates multiple devices of an electronic device 1000; however, the electronic device 1000 of the present invention may refer to only some of the devices, for example, only the processor 1100 and the memory 1200.

[0066] In this embodiment of the disclosure, the text to be translated is referred to as the source text (SRC), the translated text obtained by machine translation of the source text is referred to as the machine translation (MT), and the post-editing (PE) text obtained after correcting the machine translation text is referred to as the post-editing (PE) text. In this embodiment of the disclosure, the electronic device 1000 is used to correct the machine translation text (MT) to obtain the post-editing (PE) text.

[0067] The various embodiments and examples of the fundamental disclosure are described below with reference to the accompanying drawings.

[0068] <Example of Post-Translation Editing Method>

[0069] See Figure 2 The illustration shows a post-editing method provided by an embodiment of this disclosure. The post-editing method includes steps S102-S106.

[0070] Step S102: Obtain the target source text and the target machine-translated text. The target machine-translated text is the machine-translated version of the target source text.

[0071] Step S104: Input the target source text and the target machine-translated text into the pre-trained post-editing model, and correct the target machine-translated text through the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words.

[0072] Step S106: Output the target post-edited text obtained by correcting the target machine-translated text using the post-editing model.

[0073] In this embodiment, the post-editing model is used to simulate the way humans correct the translation by deleting and inserting words, and can be implemented based on a non-autoregressive model with an encoder-decoder architecture. Using a non-autoregressive model can significantly improve the model's inference speed and generation efficiency.

[0074] In the embodiments of this disclosure, the encoder and decoder can be implemented using neural networks, such as convolutional neural networks or attention-based neural networks. Convolutional Neural Networks (CNNs) are a type of feedforward neural network that includes convolutional computations and has a deep structure, offering advantages in fields such as Natural Language Processing (NLP), speech recognition, language modeling, text generation, and machine translation. Attention-based neural networks can select key information for processing during the handling of large amounts of information, thus improving the efficiency and accuracy of the neural network.

[0075] In a specific example, the post-editing model can be implemented using the Transformer model, which is a neural network model based on an encoder-decoder architecture.

[0076] The following is combined Figure 3 Explain the architecture and working process of the post-editing model:

[0077] Post-editing models can be non-autoregressive models based on an encoder-decoder architecture, including an encoder network and a decoder network. The encoder network encodes the source text (SRC) to obtain its encoded vector. The decoder network encodes the machine-translated text (MT) to obtain its encoded vector, and then corrects the machine-translated text (MT) based on both the source text (SRC) and machine-translated text (MT) encoded vectors.

[0078] The following is a further introduction:

[0079] The encoder network takes the source text (SRC) as input, encodes the SRC to obtain its encoded vector, and outputs this encoded vector to the decoder network. In one example, the encoder network can contain only one encoder. Using a non-autoregressive model with only a single encoder as the architecture of the post-editing model helps alleviate latency issues, reduces training time, and still achieves good results.

[0080] The first input of the decoder network is connected to the output of the encoder network to receive the encoded vector of the source text. The second input of the decoder network is used to input the machine-translated text (MT). The output of the decoder network is used to output the translated and edited text (PE).

[0081] The decoder network includes an encoding unit, a decoding unit, a first prediction unit, a second prediction unit, and a third prediction unit.

[0082] The encoding unit is used to encode the machine-translated text (MT) to obtain the encoded vector of the machine-translated text (MT).

[0083] The decoding unit performs matrix operations on the encoded vectors of the source text (SRC) and the machine-translated text (MT) to obtain the attention distribution of the machine-translated text (MT) based on the source text (SRC). Conventional matrix operations can include matrix addition, subtraction, scalar multiplication, transpose, conjugate, and conjugate transpose. In this embodiment, the attention distribution of the machine-translated text (MT) based on the source text (SRC) is obtained by performing matrix operations on the encoded vectors of the source text (SRC) and the machine-translated text (MT). An attention mechanism is a mechanism used in encoder-decoder structures, similar to the selective attention mechanism in humans. Its core objective is to identify the information most critical to the current task objective from a large amount of information. The attention distribution is the probability information of attention allocation determined by the attention mechanism. In this embodiment, the attention distribution of the machine-translated text (MT) based on the source text (SRC) characterizes the degree to which the words in the machine-translated text (MT) are influenced by the words in the source text (SRC).

[0084] The first prediction unit is used to predict whether to delete tokens in the machine-translated text (MT) based on the attention distribution of the machine-translated text (MT) on the source text (SRC).

[0085] The second prediction unit is used to predict the number of placeholders that need to be inserted between words in the machine-translated text (MT), where the number of placeholders is an integer greater than or equal to zero. In this embodiment of the disclosure, predicting the number of placeholders that need to be inserted between words in the machine-translated text (MT) refers to predicting the number of placeholders that need to be inserted before the first word, between any two adjacent words, and after the last word in the machine-translated text (MT). For any given position, if the predicted number of placeholders is 0, it means that no placeholders need to be inserted at that position; if the predicted number of placeholders is T, where T ≥ 1, it means that T placeholders need to be inserted at that position.

[0086] The third prediction unit is used to predict the lexical units that replace placeholders.

[0087] In other words, the first prediction unit first predicts the word segments that need to be deleted, then the second prediction unit predicts the position of the word segments that need to be inserted and the number of word segments that need to be inserted at that position, and the third prediction unit predicts what specific word segments these word segments need to be inserted are.

[0088] The first and third prediction units can be implemented based on classification networks, such as linear classification networks. The second prediction unit can be implemented based on regression analysis networks.

[0089] In step S104, the target source text is input to the encoding unit of the encoder network, and the target machine-translated text is input to the encoding unit of the decoder network. The encoder network encodes the target source text to obtain its encoding vector, which is then input to the decoding unit of the decoder network. The decoder network's encoding unit encodes the target machine-translated text to obtain its encoding vector. The decoder network's decoding unit performs matrix operations on the encoding vectors of the target source text and the target machine-translated text to obtain the attention distribution of the target machine-translated text based on the target source text. In this embodiment, the attention distribution of the target machine-translated text based on the target source text characterizes the degree to which each word in the target machine-translated text is influenced by each word in the target source text.

[0090] The system utilizes a first prediction unit, a second prediction unit, and a third prediction unit to mimic human actions of deleting and inserting lexical units. Specifically:

[0091] The first prediction unit predicts whether to delete words from the target machine-translated text based on the attention distribution of the target source text, thus obtaining the deleted target machine-translated text.

[0092] The first prediction unit may include a binary classification network, which classifies each word in the target machine-translated text into a word that needs to be deleted or a word that does not need to be deleted based on the attention distribution of the target source text. The word that needs to be deleted is then removed from the target machine-translated text to obtain the deleted target machine-translated text.

[0093] The second prediction unit predicts the number of placeholders that need to be inserted into the deleted target machine-translated text. This number is an integer greater than or equal to zero, resulting in the inserted target machine-translated text. Specifically, for the deleted target machine-translated text, it predicts the number of placeholders to be inserted before the first word, between any two adjacent words, and after the last word. For any given position, if the predicted number of placeholders is 0, then no placeholder is needed at that position. If the predicted number of placeholders is T, where T ≥ 1, then T placeholders are needed at that position.

[0094] The second prediction unit may include a regression analysis network, which uses regression analysis algorithms to predict the position and number of placeholders that need to be inserted in the target machine-translated text after deletion.

[0095] The third prediction unit predicts the lexical units that will replace the placeholders.

[0096] The third prediction unit may include a multi-classification network, the number of categories in which is consistent with the number of words in the vocabulary. For each placeholder inserted in the previous step, the word with the highest probability is predicted from the vocabulary as the target word, and the corresponding placeholder is replaced with the target word.

[0097] See below. Figure 4 As shown, a concrete example illustrates the working process of the post-editing model. Figure 4 middle, <s> Indicates the beginning of the text.< / s> The first prediction unit predicts whether to delete the tokens in the target machine-translated text "cat sit mat". In this example, the token "cat" does not need to be deleted, the token "sit" needs to be deleted, and the token "mat" does not need to be deleted. The second prediction unit predicts the number of placeholders to be inserted between the tokens in the target machine-translated text. The number of placeholders to be inserted should be an integer greater than or equal to zero. In this example, the result is that 1 placeholder needs to be inserted before the token "cat", 3 placeholders need to be inserted between "cat" and "mat", and 0 placeholders need to be inserted after "mat". That is, no placeholder needs to be inserted after "mat", thus obtaining the token sequence "[PLH]cat[PLH][PLH][PLH]mat". The third prediction unit is used to predict the tokens of the four placeholders in the substitution token sequence “[PLH]cat[PLH][PLH][PLH]mat”. It ultimately predicts that the token to replace the first placeholder is “a”, the token to replace the second placeholder is “sit”, the token to replace the third placeholder is “on”, and the token to replace the fourth placeholder is “the”, thus obtaining the translated edited text “a cat sit on the mat”.

[0098] The post-editing method provided in this disclosure inputs the target source text and its machine-translated text into a pre-trained post-editing model. The post-editing model then corrects the machine-translated text of the target source text, mimicking human methods of deleting and inserting words during the correction process, thereby improving the accuracy of the translation results.

[0099] The post-editing method provided in this disclosure utilizes a non-autoregressive generative model to mimic common operations in human post-editing—lexical deletion and lexical insertion—effectively capturing the potential differences between the post-edited text (PE) and the machine-translated text (MT), thereby predicting the post-edited text (PE) and effectively improving the accuracy of the translation results.

[0100] The post-editing method provided in this disclosure can improve the speed of post-editing and the quality of the final translation, greatly improving the work efficiency of translators and can be widely applied to various scenarios that require machine translation.

[0101] The following explains the process of obtaining the above-mentioned post-editing model through training:

[0102] See Figure 5 As shown in the embodiments of this disclosure, the process of training the original post-editing model includes steps S502-S504.

[0103] Step S502: Obtain multiple sample data combinations. Each sample data combination includes a sample source text, a first sample translated text, and a second sample translated text, wherein the first sample translated text is the machine-translated text of the sample source text, and the second sample translated text is the post-edited text of the sample source text.

[0104] Step S504: Train the post-editing model using a combination of multiple sample data.

[0105] For one of the sample data combinations, the specific training method can be as follows: input the sample source text and the first sample translated text into the post-editing model, and use the second sample translated text as supervision to train the post-editing model.

[0106] The post-editing model is trained using multiple sample data combinations to update its parameters until the loss of the post-editing model decreases to a certain level, indicating that training is complete and can be terminated. Specifically, during training, the prediction results of the post-editing model are compared with the second sample translated text, and the model's predictions are iteratively optimized towards the direction of the second sample translated text. During training, a loss function is used to calculate the loss of the post-editing model, which quantitatively measures the degree to which the prediction results differ from the second sample translated text. For example, the Levenshtein edit distance can be used to measure the degree of difference between the prediction results and the second sample translated text. During iteration, when the loss of the post-editing model is within a preset range over a preset time period, the model is considered to have converged, and training can be terminated.

[0107] In one example, the loss is calculated for each of the three editing decision processes (deleting a lexical, inserting a placeholder, and replacing a placeholder). The sum of the three losses is taken as the total loss. When the total loss is within a preset range over a preset time period, the post-editing model is considered to have converged, and training can end at this point. In another example, the Levenshtein edit distance can be used to measure the loss generated by each editing decision process.

[0108] In one example, a combination of Teacher-forcing and non-Teacher-forcing modes is used for model training. In the field of text generation, model training typically employs the Teacher-forcing mode, which uses the output from the prior time step as input and requires the model's prediction to perfectly correspond to the standard answer. However, this training method has certain problems, leading to discrepancies between the generated data during training and inference. Specifically, when training a model using the Teacher-forcing mode, the model is required not to use the output of the previous step as input for the next step, but instead uses the standard answer from the previous step in the training data. This results in a different data distribution during inference compared to training, leading to poorer inference performance—a problem known as exposure bias. In this embodiment, a combination of Teacher-forcing and non-Teacher-forcing modes is used to mitigate the potential exposure bias problem.

[0109] Specifically, in step S504, training is performed using a combination of teacher-forcing and non-teacher-forcing modes. See [link / reference] Figure 6 As shown, step S504 includes steps S602-S606.

[0110] Step S602: Divide multiple sample data into n training batches and determine the unreliability probability of each training batch.

[0111] A sample data set includes a source text and its corresponding first and second translated texts. Multiple sample data sets are divided into n training batches, and the post-editing model is trained batch by batch.

[0112] Step S604: If the unreliability probability of the i-th training batch is greater than or equal to a preset threshold, determine that the first mode is used for training each sample data combination in the i-th training batch. If the unreliability probability of the i-th training batch is less than the preset threshold, determine that the second mode is used for training each sample data combination in the i-th training batch. The i-th training batch is any one of the n training batches.

[0113] In this embodiment, the first mode corresponds to the Teacher-forcing mode, and the second mode corresponds to the non-teacher-forcing mode. As the training batch size increases, the unreliability probability tends to decrease, and the model tends to use the previous prediction result of the post-editing model itself as the input for the current prediction.

[0114] In one example, the untrust probability of the i-th training batch is determined using the following formula:

[0115] ∈ i =max(∈ min ,KC*i)

[0116] Where ∈1 is the unreliability probability of the i-th training batch, max() is the maximum value function, ∈ min The minimum unreliable probability is the preset value, ∈ min ≥0. K represents the offset of the decay, and C represents the slope of the decay. K and C are positive constants that depend on the expected convergence rate and can be determined based on the amount of training data and other parameters / experience.

[0117] In this way, the n training batches are divided into two stages. In the first stage, the unreliability probability of the training batches is greater than or equal to a preset threshold, and the Teacher-forcing mode is used for training. In the second stage, the unreliability probability of the training batches is less than the preset threshold, and the non-Teacher-forcing mode is used for training.

[0118] Step S606: When training with the first mode for each sample data combination in the i-th training batch, the (j-1)-th word of the second sample translated text in the sample data combination is used as the next input of the post-editing model to predict the j-th word. When training with the second mode for each sample data combination in the i-th training batch, the (j-1)-th word predicted by the post-editing model is used as the next input of the post-editing model to predict the j-th word. Here, j is an integer and j≥2.

[0119] See Figure 7 As shown, <s>This indicates the start of the text. Softmax is the normalized exponential function, and h1,…,h j-1 ,h j ... are intermediate feature vectors. For a sample dataset, the tokens in the translated text of the second sample are denoted as y1,...,y1 in order. i-1 ,y j The lexical units predicted by the post-editing model are denoted as… If the current training batch uses the first mode, when predicting j-th words, the (j-1)th word y of the translated text from the second sample is used. j-1 As input. If the current training batch uses the second mode, when predicting the j-th word, use the (j-1)-th word already predicted by the post-editing model. As input.

[0120] This embodiment employs a course-based learning strategy, arguing that learning should be gradual, transitioning from one state to another. At the beginning of training, due to insufficient model training, using model predictions as input would result in very slow convergence. Therefore, in the early stages of model training, second-sample translated text is used as input to improve training efficiency. In the later stages of model training, the model's own predictions are preferred as input to mitigate potential exposure bias issues.

[0121] In this embodiment, the model training is divided into two stages, with different prediction inputs used in the two stages. On the other hand, the corresponding loss is calculated for each of the three editing decision processes (corresponding to deleting lexical units, inserting placeholders, and replacing placeholders). Through these two aspects, the model can fully learn the editing decision process, enabling the post-editing model to learn the best editing operation.

[0122] The process of obtaining sample data is explained below:

[0123] A sample data set includes a source text sample, its corresponding first sample translation text, and a second sample translation text. For learning tasks requiring a large number of training samples, it is necessary to collect a large number of samples in advance and manually annotate them, which is time-consuming and labor-intensive. In this embodiment of the disclosure, a method for easily obtaining sample data is provided.

[0124] See Figure 8 As shown, bilingual data is acquired, cleaned, and erroneous data is removed to obtain bilingual text corpora corresponding to the same content. For example, if the desired bilingual text corpus for training is Chinese-English, but the bilingual data also includes Chinese-German data, then the Chinese-German data needs to be excluded. Similarly, if the same bilingual data does not correspond to the same content, then that bilingual data needs to be excluded.

[0125] After obtaining bilingual corpora including first-language texts and second-language texts with corresponding content, sample data is constructed using the bilingual corpus texts.

[0126] In the first construction method: the first language text is used as the source text, and the second language text is used as the second sample translation text. Words in the second language text are randomly replaced and / or deleted to obtain the first sample translation text. For example, if the first language text is "a cat sits on the carpet," and the second language text is "a cat sits on the mat," randomly replacing words in the second language text yields "a cat stands on the mat." Using the first language text "a cat sits on the carpet" as the source text, the second language text "a cat sits on the mat" as the second sample translation text, and the replaced "a cat stands on the mat" as the first sample translation text, this method allows for the simple and rapid construction of a large amount of sample data.

[0127] In the second construction method, a translation model is used to construct sample data. Specifically, the first language text is used as the source sample text, and the second language text is used as the second sample translation text. The first sample translation text is constructed by translating the source sample text using a translation model.

[0128] In one example, a pre-trained translation model is used to translate the source text to construct the first sample translated text. The process involves: fine-tuning the parameters of the pre-trained translation model, inputting the first language text into the model, translating the first language text using the model, and using this translation result as the first sample translated text. Since the pre-trained translation model's parameters are already optimal, fine-tuning the parameters can shift the model from its optimal state to a worse state, resulting in a poorer translation of the first language text, which is then used as the first sample translated text.

[0129] The translation model could be, for example, a beam search algorithm-based model. The beam search algorithm works by selecting the m words with the highest probabilities from the vocabulary each time, generating multiple search paths, and determining the search path with the highest overall probability as the final search path. Here, m is an adjustable parameter; degrading m from its optimal value yields a relatively poor translation result, which is then used as the first sample translated text. In this embodiment, m can be adjusted from its optimal value upwards or downwards, making it no longer optimal, thus degrading m. For example, after degrading m from its optimal value, the translation model is used to translate the sample source text, and the resulting translation is used as the first sample translated text. The beam search algorithm is a heuristic graph search algorithm, typically used when the solution space of the graph is large. To reduce the space and time occupied by the search, some nodes with lower quality are pruned at each depth expansion step. In this embodiment, using the beam search algorithm to construct the first sample translated text can reduce space consumption and improve time efficiency.

[0130] In one example, a translation model based on the sampling algorithm can be used to construct the first sample translation text. The principle of the sampling algorithm is to randomly sample from the vocabulary. Translation models based on the sampling algorithm themselves have poor translation performance, and it is easy to obtain poor translation results as the first sample translation text.

[0131] In one example, a translation model based on the Top-K Sampling algorithm can be used to construct the first sample translation text. The Top-K Sampling algorithm is similar to the beam search algorithm, but after selecting the K most probable words from the vocabulary each time, it normalizes the probabilities of these K words. This causes a shift in the optimal probability words, resulting in a deviation in the final search path and a poor translation result. Using a translation model based on the Top-K Sampling algorithm, it is easy to obtain a poor translation result as the first sample translation text.

[0132] After obtaining multiple first sample translated texts through any of the above methods, further filtering can be performed. In this embodiment of the disclosure, the text with the lowest confidence level is selected from the multiple first sample translated texts and included in the sample data.

[0133] For example, the translation model obtains a predicted probability for each word in the output. By averaging the probabilities of each word in the translation model's output, the confidence level of the translation result can be obtained, which is the confidence level of the first sample translated text. The confidence levels of multiple first sample translated texts are then clustered, dividing them into two categories: high confidence and low confidence. The first sample translated text belonging to the low confidence category is then selected as the final first sample translated text.

[0134] After obtaining the sample data, the sample data is fed into the original post-editing model for training.

[0135] <Example of Post-Editing Device>

[0136] Figure 9 This is a schematic diagram of a post-editing apparatus 20 provided in one embodiment of the present disclosure. The post-editing apparatus 20 includes the following modules:

[0137] The acquisition module 21 is used to acquire the target source text and the target machine-translated text, wherein the target machine-translated text is the machine-translated text of the target source text.

[0138] Correction module 22 is used to input the target source text and the target machine-translated text into a pre-trained post-editing model, and to correct the target machine-translated text through the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words.

[0139] Output module 23 is used to output the target post-edited text obtained by the post-editing model after correcting the target machine-translated text.

[0140] In one example, the post-editing model includes an encoder network and a decoder network. The post-editing model corrects the target machine-translated text by: encoding the target source text using the encoder network to obtain an encoding vector; encoding the target machine-translated text using the decoder network to obtain an encoding vector; and correcting the target machine-translated text based on the encoding vectors of the source and machine-translated texts.

[0141] In one example, the decoder network includes an encoding unit, a decoding unit, a first prediction unit, a second prediction unit, and a third prediction unit. The decoder network encodes the target machine-translated text to obtain its encoded vector, and then modifies the target machine-translated text based on the encoded vectors of the source and source texts. This includes: the encoding unit encoding the target machine-translated text to obtain its encoded vector; the decoding unit performing matrix operations on the encoded vectors of the source and source texts to obtain the attention distribution of the target machine-translated text based on the source text; the first prediction unit predicting whether to delete words in the target machine-translated text based on the attention distribution; the second prediction unit predicting the number of placeholders to be inserted between words in the target machine-translated text, where the number is an integer greater than or equal to zero; and the third prediction unit predicting the words to replace the placeholders.

[0142] In one example, the post-editing device also includes a training module.

[0143] The training module is used to: acquire multiple combinations of sample data before inputting the target source text and the target machine-translated text into the pre-trained post-editing model. Each combination of sample data includes a sample source text, a first sample translated text, and a second sample translated text, where the first sample translated text is the machine-translated text of the sample source text and the second sample translated text is the post-edited text of the sample source text; and to train the post-editing model using the multiple combinations of sample data.

[0144] In one example, the post-editing device also includes a first sample construction module.

[0145] The first sample construction module acquires bilingual corpus text, which includes first-language text and second-language text with corresponding content. The first-language text is used as the source text in the sample data combination. The second-language text is used as the second sample translation text in the sample data combination. Words in the second-language text are randomly replaced and / or randomly deleted to obtain the first sample translation text in the sample data combination.

[0146] In one example, the post-editing device also includes a first sample construction module.

[0147] The second sample construction module is used to acquire bilingual corpus text, which includes first-language text and second-language text with corresponding content. The first-language text is used as the source text in the sample data combination. The second-language text is used as the second sample translation text in the sample data combination. The parameters of the pre-trained translation model are adjusted, the first-language text is input into the translation model, and the translation model translates the first-language text to obtain the translation result. This translation result is used as the first sample translation text in the sample data combination.

[0148] In one example, the training module trains the post-editing model using multiple combinations of sample data, including: dividing the multiple combinations of sample data into n training batches and determining the unreliability probability of each training batch. If the unreliability probability of the i-th training batch is greater than or equal to a preset threshold, a first mode is used for training each combination of sample data in the i-th training batch. If the unreliability probability of the i-th training batch is less than the preset threshold, a second mode is used for training each combination of sample data in the i-th training batch. The i-th training batch is any one of the n training batches. When training with the first mode for each combination of sample data in the i-th training batch, the (j-1)-th word of the second sample translated text in the sample data combination is used as the next input of the post-editing model to predict the j-th word; when training with the second mode for each combination of sample data in the i-th training batch, the (j-1)-th word predicted by the post-editing model is used as the next input of the post-editing model to predict the j-th word; j is an integer and j≥2.

[0149] In one example, the training module determines the untrust probability of the i-th training batch according to the following formula:

[0150] ∈ i =max(∈ min ,KC*i)

[0151] Where, ∈ i Let K be the unreliable probability of the i-th training batch, max() be the maximum value function, and K and C be positive constants. min The minimum unreliable probability is the preset value, ∈ min ≥0.

[0152] The post-editing apparatus provided in this embodiment inputs the target source text and its machine-translated text into a pre-trained post-editing model. The model then corrects the machine-translated text of the target source text, mimicking human methods of deleting and inserting words during the correction process, thereby improving the accuracy of the translation results.

[0153] <Electronic Device Examples>

[0154] Figure 10 This is a schematic diagram of an electronic device 30 provided in one embodiment of the present disclosure. The electronic device 30 includes a processor 31 and a memory 32. The memory 32 stores computer instructions, which, when executed by the processor 31, implement the post-editing method disclosed in any of the foregoing embodiments.

[0155] In a specific example, the electronic device 30 may be equipped with translation software. The electronic device 30 may be, for example, an electronic device running a smart operating system (such as Android, iOS, Windows, Linux, etc.), including but not limited to laptops, desktop computers, mobile phones, tablets, etc., and may have similar... Figure 1 The hardware configuration of the electronic device 1000 shown.

[0156] <Example of Computer-Readable Storage Medium>

[0157] This disclosure also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the post-editing method disclosed in any of the foregoing embodiments.

[0158] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and apparatus embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0159] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0160] Embodiments of this disclosure may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the embodiments of this disclosure.

[0161] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0162] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0163] Computer program instructions used to perform the operations of embodiments of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of embodiments of this disclosure.

[0164] Various aspects of embodiments of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0165] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0166] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0167] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0168] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.< / s>

Claims

1. A post-editing method, characterized by, include: Obtain the target source text and the target machine-translated text, wherein the target machine-translated text is the machine-translated version of the target source text; The target source text and the target machine-translated text are input into a pre-trained post-editing model, and the target machine-translated text is corrected by the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words; Output the target post-edited text obtained by the post-editing model after correcting the target machine-translated text; The post-editing model includes an encoder network and a decoder network; the decoder network includes an encoding unit, a decoding unit, a first prediction unit, a second prediction unit, and a third prediction unit; the step of correcting the target machine-translated text using the post-editing model includes: The target source text is encoded using the encoder network to obtain the encoded vector of the target source text. The target machine-translated text is encoded using the encoding unit to obtain the encoded vector of the target machine-translated text; The decoding unit performs matrix operations on the encoding vector of the target source text and the encoding vector of the target machine-translated text to obtain the attention distribution of the target machine-translated text based on the target source text. The attention distribution characterizes the degree to which the lexical units in the target machine-translated text are influenced by the lexical units in the target source text. The first prediction unit predicts whether to delete word units in the target machine-translated text based on the attention distribution of the target source text. The second prediction unit predicts the number of placeholders that need to be inserted between words in the target machine-translated text, where the number is an integer greater than or equal to zero. The third prediction unit predicts the lexical unit that will replace the placeholder.

2. The method of claim 1, wherein, Before inputting the target source text and the target machine-translated text into the pre-trained post-editing model, the method further includes: Multiple sample data combinations are obtained, each sample data combination including sample source text, first sample translated text and second sample translated text, the first sample translated text is the machine translated text of the sample source text, and the second sample translated text is the post-edited text of the sample source text; The post-editing model is trained using the combination of the multiple sample data.

3. The method of claim 2, wherein, Before acquiring multiple combinations of sample data, the method further includes: Obtain bilingual text corpus, wherein the bilingual text corpus includes first language text and second language text corresponding to the same content; The first language text is used as the sample source text in the sample data combination; The second language text is used as the second sample translation text in the sample data combination; Randomly replace and / or delete word units in the second language text to obtain the first sample translation text in the sample data combination.

4. The method of claim 2, wherein, Before acquiring multiple combinations of sample data, the method further includes: Obtain bilingual text corpus, wherein the bilingual text corpus includes first language text and second language text corresponding to the same content; The first language text is used as the sample source text in the sample data combination; The second language text is used as the second sample translation text in the sample data combination; The parameters of the pre-trained translation model are adjusted, the first language text is input into the translation model, the translation model is used to translate the first language text to obtain the translation result, and the translation result is used as the first sample translation text in the sample data combination.

5. The method of claim 2, wherein, The step of training the post-editing model using the combination of the multiple sample data includes: The multiple sample data are combined into n training batches, and the unreliability probability of each training batch is determined. If the unreliability probability of the i-th training batch is greater than or equal to a preset threshold, a first mode is used for training each of the sample data combinations in the i-th training batch; if the unreliability probability of the i-th training batch is less than the preset threshold, a second mode is used for training each of the sample data combinations in the i-th training batch; the i-th training batch is any one of the n training batches. When training with the first mode for each sample data combination in the i-th training batch, the (j-1)-th word of the second sample translated text in the sample data combination is used as the next input of the post-editing model to predict the j-th word; when training with the second mode for each sample data combination in the i-th training batch, the (j-1)-th word predicted by the post-editing model is used as the next input of the post-editing model to predict the j-th word; where j is an integer and j≥2.

6. The method according to claim 5, characterized in that, The untrust probability of the i-th training batch is determined according to the following formula: in, Let be the unreliable probability of the i-th training batch, and max() be the function to find the maximum value. and It is a positive constant. This is the preset minimum probability of being unreliable. ≥0.

7. A post-editing device, characterized in that, include: The acquisition module is used to acquire the target source text and the target machine-translated text, wherein the target machine-translated text is the machine-translated text of the target source text; The correction module is used to input the target source text and the target machine-translated text into a pre-trained post-editing model, and to correct the target machine-translated text through the post-editing model; the correction methods of the post-editing model include deleting words, inserting placeholders, and replacing placeholders with words; The output module is used to output the target post-edited text obtained by the post-editing model after correcting the target machine-translated text; The post-editing model includes an encoder network and a decoder network; the decoder network includes an encoding unit, a decoding unit, a first prediction unit, a second prediction unit, and a third prediction unit; the step of correcting the target machine-translated text using the post-editing model includes: The target source text is encoded using the encoder network to obtain the encoded vector of the target source text. The target machine-translated text is encoded using the encoding unit to obtain the encoded vector of the target machine-translated text; The decoding unit performs matrix operations on the encoding vector of the target source text and the encoding vector of the target machine-translated text to obtain the attention distribution of the target machine-translated text based on the target source text. The attention distribution characterizes the degree to which the lexical units in the target machine-translated text are influenced by the lexical units in the target source text. The first prediction unit predicts whether to delete word units in the target machine-translated text based on the attention distribution of the target source text. The second prediction unit predicts the number of placeholders that need to be inserted between words in the target machine-translated text, where the number is an integer greater than or equal to zero. The third prediction unit predicts the lexical unit that will replace the placeholder.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer instructions, characterized in that, when the computer instructions are executed by the processor, they implement the post-editing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed by a processor, implement the post-editing method according to any one of claims 1-6.

Images