Data generation method, device and readable storage medium

Abstract

The present disclosure relates to a data generation method, device and readable storage medium, the method comprising: performing noise adding processing on initial first language text to obtain noise-added first language text; processing the noise-added first language text according to a pre-trained language model to obtain target first language text; performing back translation processing on the target first language text to obtain second language text; and obtaining training data for training a translation model based on the target first language text and the second language text. The method of the present disclosure can improve the diversity of the training data for training the translation model and solve the problem of training data shortage.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and particularly relates to a data generation method and device and readable storage medium. BACKGROUND

[0002] Machine translation refers to a process of converting a source language into a target language by a computer. With the development of machine learning and deep learning technologies, machine translation gradually moves from statistical machine translation to neural machine translation era. For a neural machine translation model, training data used for training the translation model has a very important influence on the prediction of the model, therefore, how to better construct the training data of the translation model to improve the translation accuracy of the translation model is a problem to be solved. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides a data generation method, device and readable storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a data generation method is provided, comprising:

[0005] performing noise adding processing on the initial first language text to obtain a first language text after noise adding;

[0006] processing the first language text after noise adding according to a pre-trained language model to obtain a target first language text;

[0007] performing back translation processing on the target first language text to obtain a second language text;

[0008] obtaining training data for training a translation model based on the target first language text and the second language text.

[0009] In some embodiments, the first language text is a chapter-level text, and the performing back translation processing on the target first language text to obtain a second language text comprises:

[0010] splitting the target first language text to obtain a plurality of target first language subtexts; wherein the first language subtext is a sentence-level text;

[0011] performing the back translation processing on the plurality of target first language subtexts according to a back translation model to obtain a plurality of second language subtexts;

[0012] performing fusion processing on the plurality of second language subtexts to obtain the second language text.

[0013] In some embodiments, the performing noise adding processing on the initial first language text to obtain a first language text after noise adding comprises:

[0014] The initial first language text is subjected to a deletion replacement and noise addition process to obtain a first language text with noise added, the first language text with noise added being missing a preset number or preset position of words relative to the initial first language text.

[0015] In some embodiments, the noise addition process on the initial first language text to obtain the first language text with noise added includes:

[0016] The initial first language text is subjected to a noise addition process of random reordering to obtain a first language text with noise added, the first language text with noise added being different in sentence order or word order relative to the initial first language text.

[0017] In some embodiments, the pre-trained language model is a bidirectional autoregressive transformer model.

[0018] In some embodiments, the back translation model is trained in the following manner:

[0019] A plurality of training samples are obtained; each of the training samples includes a sample first language subtext and a sample second language subtext; the sample first language subtext and the sample second language subtext are the sentence-level text;

[0020] The parameters of the initial back translation model are iteratively updated based on the plurality of training samples to reduce the loss function value corresponding to each training sample, to obtain a trained back translation model;

[0021] The loss function value corresponding to each training sample is determined in the following manner:

[0022] The sample first language subtext is processed by the back translation model to obtain a predicted second language subtext;

[0023] The loss function value is determined based on at least the difference between the predicted second language subtext and the sample second language subtext.

[0024] In some embodiments, the back translation model is a transformer model.

[0025] According to a second aspect of the embodiments of the present disclosure, a data generation apparatus is provided, including:

[0026] A noise addition module configured to perform a noise addition process on an initial first language text to obtain a first language text with noise added;

[0027] A processing module configured to process the first language text with noise added according to a pre-trained language model to obtain a target first language text;

[0028] a back-translation module configured to perform a back-translation process on the target first language text to obtain a second language text;

[0029] a training data determination module configured to obtain training data for training a translation model based on the target first language text and the second language text.

[0030] In some embodiments, the first language text is a document-level text, and the back-translation module is further configured to:

[0031] split the target first language text to obtain a plurality of target first language subtexts; wherein the first language subtext is a sentence-level text;

[0032] perform the back-translation process on the plurality of target first language subtexts according to the back-translation model to obtain a plurality of second language subtexts;

[0033] fuse the plurality of second language subtexts to obtain the second language text.

[0034] In some embodiments, the noise adding module is further configured to perform a deletion and replacement noise adding process on the initial first language text to obtain a noise-added first language text, the noise-added first language text missing a preset number or a preset position of words relative to the initial first language text.

[0035] In some embodiments, the noise adding module is further configured to perform a disordering noise adding process on the initial first language text to obtain a noise-added first language text, the noise-added first language text having a different sentence order or word order relative to the initial first language text.

[0036] In some embodiments, the pre-trained language model is a bidirectional autoregressive transformer model.

[0037] In some embodiments, the apparatus further comprises a training module configured to:

[0038] obtain a plurality of training samples; wherein each of the training samples comprises a sample first language subtext and a sample second language subtext; wherein the sample first language subtext and the sample second language subtext are the sentence-level texts;

[0039] update the parameters of the initial back-translation model based on the plurality of training samples to reduce the loss function value corresponding to each training sample to obtain a trained back-translation model;

[0040] wherein the loss function value corresponding to each training sample is determined by the following process:

[0041] processing the sample first language text through a back translation model to obtain a predicted second language subtext;

[0042] determining a loss function value based at least on a difference between the predicted second language subtext and the sample second language subtext.

[0043] In some embodiments, the back translation model is a Transformer model.

[0044] According to a third aspect of embodiments of the present disclosure, a data generation apparatus is provided, comprising:

[0045] a processor;

[0046] a memory for storing processor-executable instructions;

[0047] The processor is configured to execute the computer program in the memory to implement the steps of the method of any one of the first aspect of the present disclosure.

[0048] According to a fourth aspect of embodiments of the present disclosure, a computer readable storage medium is provided, having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the method of any one of the first aspect of the present disclosure.

[0049] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects: by processing the noise-added first language text through the pre-trained language model, a target first language text different from the initial first language text can be generated, and then, based on the back translation result of the target first language text and the target first language text, training data can be constructed, which can reflect different scores, implement diversification of the training data, and improve the training effect of the translation model; and without additional training of the pre-trained model, the efficiency of generating the training data is greatly improved.

[0050] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0052] Figure 1 is a flowchart of a data generation method according to an exemplary embodiment of the present disclosure;

[0053] Figure 2 is a flowchart of training a back translation model according to an exemplary embodiment of the present disclosure;

[0054] Figure 3 is a block diagram of a data generation apparatus according to an example embodiment of the present disclosure;

[0055] Figure 4 is a block diagram of a data generation apparatus according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] The example embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, unless otherwise indicated, like numbers in the different drawings represent the same or similar elements. The following description of example embodiments is not representative of all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] Before introducing a data generation method provided by the present disclosure, first, the application scenarios involved in each embodiment of the present disclosure are described, and the present disclosure can be applied to the process of training a translation model, wherein the trained translation model can be used for translating a chapter-level text.

[0058] In some embodiments, in the scenario of chapter translation, the original chapter-level text can be split to obtain multiple sentence-level texts, so as to construct training data for training a chapter-level translation model according to the translation results corresponding to the multiple sentence-level texts and the original chapter-level text. However, this method has the following characteristics: there is a complex dependency relationship between the sentences in the chapter, and the multiple sentence-level texts obtained after simple splitting of the chapter-level text do not reflect this dependency relationship, resulting in inconsistent context and weak coherence of the final generated translation results. Further, the training data constructed in this way reduces the training effect of the model. In addition, in the scenario of chapter translation, the training data required for training the translation model is very scarce, which limits the widespread use of the chapter-level translation model.

[0059] Therefore, the present disclosure proposes a data generation method, apparatus and readable storage medium, which processes the first language text with noise through a pre-trained language model, so as to generate a target first language text different from the initial first language text, i.e., a target first language text with translation errors, and then constructs training data based on the back-translation result of the target first language text and the target first language text, so as to obtain training data reflecting different scores, realize the diversification of the training data, and improve the training effect of the translation model; and without additional training of the pre-trained model, the efficiency of generating the training data is greatly improved.

[0060] Figure 1 is a flowchart of a data generation method according to an example embodiment, as shown in Figure 1As shown, the method can include the following steps.

[0061] In step S11, the initial first language text is subjected to noise adding processing to obtain a first language text after noise adding.

[0062] In some embodiments, the initial first language text is a first language text that has not been subjected to any processing. The first language text can be a text obtained after translation of a text that needs to be translated, i.e., a source language text. The first language text can be a text in any language. For example, English, Chinese, German, etc. In some embodiments, the initial first language text can be a chapter-level text. A chapter can be a whole unit of language composed of a series of continuous words, phrases, clauses, sentences or paragraphs, and the chapter-level text can be a text formed by the whole unit of language. For example, a thesis, a book, a journal, etc. It can be understood that the chapter-level text can include a large number of (e.g., four thousand, one million, etc.) characters.

[0063] In some embodiments, the first language text after noise adding can be a text obtained after noise adding processing of the initial first language text. In some embodiments, the noise adding processing can include at least one of the following: replacing at least one word in the initial first language text, deleting at least one word in the initial first language text, deleting at least one word segment in the initial first language text, changing the order of at least two sentences in the initial first language text, and rotating the initial first language text.

[0064] In some embodiments, the noise adding processing of the initial first language text to obtain the first language text after noise adding can include: performing deletion and replacement noise adding processing on the initial first language text to obtain the first language text after noise adding, the first language text after noise adding missing a preset number or a preset position of words relative to the initial first language text.

[0065] It can be understood that the deletion and replacement noise adding processing can be used to obtain the first language text after noise adding missing a preset number or a preset position of words relative to the initial first language text. The preset number and the preset position can be specifically set according to actual conditions. In some embodiments, the deletion and replacement noise adding processing can include at least one of the following: deleting at least one word in the initial first language text, deleting at least one word segment in the initial first language text.

[0066] In some embodiments, when the noise adding process is a replacement process, a preset marker can be used to replace at least one word in the initial first language text. For example, the preset marker is MASK. In this case, the replacement can be a random replacement. For example, taking a sentence A included in the initial first language text and a word sequence of the sentence A as {a1, a2, a3, a4, a5} as an example, if the word a3 in the sentence A is replaced, the word sequence of the sentence A in the first language text after the noise adding process changes to {a1, a2, MASK, a4, a5}. The method of replacing other words is similar to the foregoing replacement process, and will not be described here again.

[0067] In some embodiments, replacing at least one word in the initial first language text can include replacing at least one word in the initial first language text except for conjunction words and entity words. The conjunction words can refer to words used to connect sentences. For example, words related to causality, contrast, summary, inference, etc. The entity can be any object that can be described, for example, services, names, and place names, etc. The entity word can be a word corresponding to the entity. By replacing the words other than the conjunction words and the entity words, the overall structure of the initial first language text after the noise adding process can be ensured to change as little as possible, so that the training data generated subsequently is more matched with the field of the real text data, and thus the training effect of the translation model is better.

[0068] In some embodiments, at least one word in the initial first language text can be randomly deleted. For example, taking a sentence A included in the initial first language text and a word sequence of the sentence A as {a1, a2, a3, a4, a5} as an example, if the words a4 and a5 are deleted, the word sequence of the sentence A in the first language text after the noise adding process changes to {a1, a2, a3}.

[0069] In some embodiments, the word segment can be a segment composed of a plurality of continuous words. In some embodiments, the word segment can be a plurality of continuous words in a sentence included in the initial first language text. It should be noted that the plurality of word segments can be a plurality of word segments included in the same sentence or different sentences in the initial first language text. For example, still taking the word sequence of the sentence A as {a1, a2, a3, a4, a5} as an example, the plurality of word segments can include a1-a2, a1-a2-a3, a2-a3-a4-a5, etc.

[0070] In some embodiments, the at least one word piece in the first language text can be randomly deleted. For example, still taking the foregoing example, the word piece a1-a2 or a1-a2-a3 can be deleted, and then the word sequence of sentence A in the first language text after noise addition changes to {a3, a4, a5} or {a4, a5}. In some embodiments, the length of each word piece in the at least one word piece obeys a Poisson distribution with λ = 3.

[0071] In some embodiments, the noise addition processing on the initial first language text to obtain the first language text after noise addition can include: performing disorder noise addition processing on the initial first language text to obtain the first language text after noise addition, the first language text after noise addition being different from the initial first language text in sentence order or word order.

[0072] It can be understood that the disorder noise addition processing can be used to obtain the first language text after noise addition which is different from the initial first language text in sentence order or word order. In some embodiments, the disorder noise addition processing can include at least one of the following: replacing at least one word in the initial first language text, changing the order of at least two sentences in the initial first language text, and rotating the initial first language text.

[0073] In some embodiments, the at least one word in the initial first language text can be replaced by a preset marker. For example, the preset marker is MASK. Wherein, the replacement can be random replacement. For example, still taking the word sequence {a1, a2, a3, a4, a5} of sentence A included in the initial first language text as an example, if the word a3 in sentence A is replaced, then the word sequence of sentence A in the first language text after noise addition changes to {a1, a2, MASK, a4, a5}. The method of replacing other words is similar to the foregoing replacement process, and will not be described here.

[0074] In some embodiments, replacing the at least one word in the initial first language text can include: replacing at least one word in the initial first language text except for conjunction words and entity words. The conjunction word can refer to a word used to link sentences. For example, words related to causality, contrast, summary, inference, etc. The entity can be any object that can be described, for example, services, names and place names, etc. The entity word can be a word corresponding to the entity. By replacing the words other than the conjunction words and the entity words, the overall structure of the initial first language text after noise addition can be ensured to change as little as possible, so that the training data generated subsequently is more matched with the field of the real text data, and thus the training effect of the translation model is better.

[0075] In some embodiments, the order of at least two sentences in the initial first language text can be randomly changed. In some embodiments, the initial first language text can be divided into multiple sentences with a period as a delimiter, and the order of the multiple sentences can be randomly changed. For example, taking the sentence sequence of the initial first language text as {A, B, C, D, E}, the order of the sentences A to E can be randomly changed to obtain the sentence sequence {D, E, A, B, C} of the first language text after the noise processing.

[0076] In some embodiments, rotating the initial first language text can mean randomly selecting a sentence, taking the sentence as the center, and rotating the initial first language text with the selected sentence as the new beginning. For example, taking the sentence sequence of the initial first language text as {A, B, C, D, E}, if the sentence D is taken as the center and the initial first language text is rotated, the sentence sequence of the first language text after the noise processing is {D, E, A, B, C}.

[0077] In some embodiments, multiple noise processing can be performed on the initial first language text to obtain the first language text after the noise processing. In some embodiments, the multiple noise processing can include the deletion and replacement noise processing and the disorder noise processing. The execution order of the multiple noise processing can be set according to actual conditions, and the present disclosure does not make any limitation on this. For example, taking the sentence sequence of the initial first language text as {A, B, C, D, E} and the word sequence of the sentence A as {a1, a2, a3, a4, a5}, if the noise processing includes: changing the order of the sentences A and B → rotating the initial first language text with the sentence D as the center → deleting the word fragments a1-a2 in the sentence A → deleting the word a4 in the sentence A → replacing the word a3 in the sentence A, the sentence sequence of the first language text after the noise processing is {D, E, B, A, C}, and the word sequence of the sentence A is {MASK, a5}. It should be noted that, for the purpose of simplifying the examples of the present disclosure, the number of sentences included in the initial first language text and the number of words included in the sentences are not exhaustively listed, and the present disclosure does not make any limitation on the number of sentences and the number of words.

[0078] In step S12, the pre-trained language model is used to process the first language text after the noise processing to obtain a target first language text.

[0079] In some embodiments, the target first language text can be a text obtained by the pre-trained language model reconstructing the first language text after the noise processing. Since the pre-trained language model processes the noise data, in some embodiments, the target first language text is different from the initial first language text, that is, the reconstructed text is different from the real text, and there is a restored error content.

[0080] In some embodiments, the pre-trained language model can be a neural network model pre-trained for text generation. In some embodiments, the pre-trained language model can be a Bidirectional and Auto-Regressive Transformers (BART) model, which is a pre-trained language model using the overall structure of the Transformer model. When pre-training the BART model, first, the input text is damaged using a plurality of noises, and then the input text is reconstructed by a sequence-to-sequence model to obtain an output sample.

[0081] In some embodiments, the BART model can include an encoder and a decoder, wherein the encoder uses the encoder component of the Bidirectional Encoder Representations from Transformers (BERT) model, which can encode the input text from two directions to obtain more context information. The decoder uses the decoder component from the Generative Pre-Training (GPT), which is used to reconstruct the input text.

[0082] In some embodiments, the BART model can process the first language text with noise including a preset number of characters, and thus the initial first language text can be a language text including a preset number of characters. For example, two thousand characters. In some embodiments, the initial first language text can be a small chapter language text (e.g., a language text of 2,000 characters) obtained by splitting a large chapter language text (e.g., a language text of 10,000 characters). In some embodiments, a large amount of training data can be constructed by performing the data generation method of the present disclosure on a plurality of initial first language texts including a preset number of characters, thereby improving the accuracy of training the translation model.

[0083] Since the BART model uses the context information of the text, i.e., the bidirectional semantic information, in the encoding process of pre-training, the target first language text reconstructed by the BART model is more consistent with the semantics of the real language text (i.e., the initial first language text), and has stronger fluency and coherence. Further, the second language text translated from the target first language text has stronger fluency and coherence, which is beneficial for subsequent generation of training data.

[0084] In step S13, the target first language text is back-translated to obtain a second language text.

[0085] In step S14, based on the target first language text and the second language text, training data for training a translation model is obtained.

[0086] In some embodiments, the back translation process can refer to a process of translating the language text obtained by translation into the source language text. In some embodiments, the second language text can be the source language text. For example, the target first language text is an English text, and the second language text can be a Chinese text. For another example, the target first language text is a German text, and the second language text can be an English text. The first language text and the second language text correspond to different languages. It should be noted that the first language text and the second language text can be any language translation corresponding text, and the present disclosure does not make any limitation on this.

[0087] In some embodiments, the back translation process can be performed according to the model. As described above, the first language text is a chapter-level text, and in some embodiments, the chapter-level text can be split to obtain a sentence-level text, so as to perform the back translation process based on the sentence-level text.

[0088] In some embodiments, the back translation process of the target first language text to obtain the second language text can include: splitting the target first language text to obtain a plurality of target first language subtexts; wherein the first language subtext is a sentence-level text; performing the back translation process on the plurality of target first language subtexts according to the reverse translation model to obtain a plurality of second language subtexts; and performing fusion processing on the plurality of second language subtexts to obtain the second language text.

[0089] In some embodiments, the plurality of target first language subtexts can be a plurality of sentence-level texts obtained by splitting the chapter-level target first language text. In some embodiments, the reverse translation model can be a pre-trained machine learning model, and the trained reverse translation model can output a second language subtext according to an input first language subtext. The second language subtext can be a source language text of the first language subtext. It can be understood that the second language subtext can be a sentence-level text. The training process of the reverse translation model can be referred to Figure 2 and related descriptions, which will not be repeated here.

[0090] In some embodiments, the plurality of second language subtexts can be fused to obtain the second language text. In some embodiments, the fusion processing can include splicing. In some embodiments, the plurality of second language subtexts can be spliced according to the sentence structure of the target first language text. For example, still taking the sentence sequence of the target first language text as {A, B, C, D, E} as an example, if the plurality of second language subtexts are A', B', C', D' and E', the splicing order of the plurality of second language subtexts A', B', C', D' and E' is A'→B'→C'→D'→E'.

[0091] In some embodiments, the training data can be obtained based on target first language text and second language text. The target first language text is the translated language text, and the second language text is the source language text. A parallel data set can be constructed from the target first language text and the second language text, and the parallel data set can be used to train the translation model. In some embodiments, the translation model can be a model for paragraph-level text translation.

[0092] The target first language text at the paragraph level is split into multiple target first language subtexts at the sentence level. The multiple target first language subtexts are back translated by the back translation model to obtain multiple second language subtexts. The multiple second language subtexts are fused to obtain the second language text. Since the back translation model is used for sentence-level text translation, the training method is simple and easy to apply, which can improve the efficiency of training data for the translation model. In addition, since the target first language text is more consistent with the semantics of real language text, it has strong fluency and coherence. Therefore, based on the multiple sentence-level target first language subtexts obtained by splitting the target first language text, the context semantic information is considered, and the fluency and coherence of the paragraph-level second language text obtained based on the multiple sentence-level target first language subtexts are stronger.

[0093] In addition, the present disclosure adopts a BART model pre-trained on a large amount of corpus, without the need to retrain additional models. At the same time, the first language text with noise obtained by the BART model not only retains the overall structure and style of the initial first language text, but also increases the diversity of the generated language text through noise addition, thereby increasing the diversity of the finally generated paragraph-level second language text, effectively solving the problem of shortage of training data for the translation model in the paragraph translation scenario, and helping the subsequent training of the translation model.

[0094] Figure 2 is a flowchart for training a back translation model according to an exemplary embodiment. As shown in Figure 2 , the flow includes:

[0095] In step S21, a plurality of training samples are obtained; each of the training samples includes a sample first language subtext and a sample second language subtext; the sample first language subtext and the sample second language subtext are sentence-level texts.

[0096] In some embodiments, the training samples can be data input into the initial back translation model for training the back translation model. In some embodiments, the sample first language subtext can be a sentence-level translated target language text, and the sample second language subtext can be a sentence-level source language text. More details about the sample first language subtext and the sample second language subtext can be found in the above step S14 and its related description, which will not be repeated here.

[0097] In some embodiments, the plurality of training samples can be obtained through a database or by calling a related interface.

[0098] In step S22, the parameters of the initial back translation model are iteratively updated based on the plurality of training samples to reduce the loss function value corresponding to each training sample, to obtain a trained back translation model.

[0099] In some embodiments, the back translation model can be a Transformer model. During the training of the back translation model, the parameters of the initial back translation model can be iteratively updated based on the plurality of training samples. Specifically, the parameters of the initial back translation model can be continuously adjusted to reduce the loss function value corresponding to each training sample, so that the loss function value meets a preset condition. For example, the loss function value converges, or the loss function value is less than a preset value. When the loss function meets the preset condition, the model training is completed, and a trained back translation model is obtained. The trained back translation model can perform back translation processing on the target first language subtext to obtain the second language subtext.

[0100] In some embodiments, the loss function value corresponding to each training sample is determined by the following process: processing the sample first language subtext through the back translation model to obtain a predicted second language subtext; and determining the loss function value based at least on the difference between the predicted second language subtext and the sample second language subtext.

[0101] Figure 3 is a block diagram of a data generation apparatus 300 according to an exemplary embodiment. Referring to Figure 3 The apparatus includes a noise adding module 310, a processing module 320, a back translation module 330, and a training data determination module 340.

[0102] The noise adding module 310 is configured to perform noise processing on the initial first language text to obtain a noise-added first language text.

[0103] The processing module 320 is configured to process the noise-added first language text according to a pre-trained language model to obtain a target first language text.

[0104] The back-translation module 330 is configured to perform back-translation processing on the target first language text to obtain a second language text.

[0105] The training data determination module 340 is configured to determine training data for training a translation model based on the target first language text and the second language text.

[0106] In some embodiments, the first language text is a document-level text, and the back-translation module 330 is further configured to:

[0107] split the target first language text to obtain a plurality of target first language subtexts; wherein the first language subtext is a sentence-level text;

[0108] perform the back-translation processing on the plurality of target first language subtexts according to the back-translation model to obtain a plurality of second language subtexts;

[0109] fuse the plurality of second language subtexts to obtain the second language text.

[0110] In some embodiments, the noise adding module 310 is further configured to perform deletion and replacement noise processing on the initial first language text to obtain a noise-added first language text, the noise-added first language text missing a preset number or a preset position of words relative to the initial first language text.

[0111] In some embodiments, the noise adding module 310 is further configured to perform disorder noise processing on the initial first language text to obtain a noise-added first language text, the noise-added first language text having different sentence order or word order relative to the initial first language text.

[0112] In some embodiments, the pre-trained language model is a bidirectional autoregressive transformer model.

[0113] In some embodiments, the apparatus further comprises a training module configured to:

[0114] obtain a plurality of training samples; wherein each training sample comprises a sample first language subtext and a sample second language subtext; wherein the sample first language subtext and the sample second language subtext are the sentence-level texts;

[0115] iteratively update parameters of the initial back-translation model based on the plurality of training samples to reduce a loss function value corresponding to each training sample to obtain a trained back-translation model;

[0116] wherein the loss function value corresponding to each training sample is determined by the following process:

[0117] processing the sample first language text through a back translation model to obtain predicted second language subtext;

[0118] determining a loss function value based at least on a difference between the predicted second language subtext and the sample second language subtext.

[0119] In some embodiments, the back translation model is a Transformer model.

[0120] With regard to the apparatus in the above-described embodiments, a specific manner in which each module performs operations has been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0121] Figure 4 is a block diagram of a data generation apparatus 400 according to an exemplary embodiment. The apparatus 400 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like, for example.

[0122] Referring to Figure 4 , the apparatus 400 can include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0123] The processing component 402 generally controls the overall operations of the apparatus 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 can include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned data generation method. In addition, the processing component 402 can include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 can include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0124] The memory 404 is configured to store various types of data to support the operations of the apparatus 400. Examples of these data include instructions for any application or method operating on the apparatus 400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 404 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0125] Power component 406 provides power to various components of device 400. Power component 406 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for device 400.

[0126] Multimedia component 408 includes a screen providing an output interface between device 400 and a user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, multimedia component 408 includes a front camera and / or a rear camera. When device 400 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0127] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when device 400 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0128] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, which can be a keyboard, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0129] The sensor component 414 includes one or more sensors to provide the status of various aspects of the device 400. For example, the sensor component 414 can detect the open / closed status of the device 400, the relative positioning of components, such as the display and keypad of the device 400, the sensor component 414 can also detect changes in position of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration / g-force and the temperature changes of the device 400. The sensor component 414 can include an optical sensor for detecting ambient light, a proximity sensor configured to detect proximity of an object, a motion sensor configured to detect movement of the device 400, a temperature sensor configured to detect temperature changes of the device 400, a humidity sensor configured to detect humidity changes of the device 400, an acceleration sensor configured to detect the acceleration of the device 400, a gyroscope sensor configured to detect the orientation of the device 400, a magnetometer sensor configured to detect the orientation of the device 400, a pressure sensor configured to detect the pressure of the device 400, or a combination thereof.

[0130] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and another device. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-WideBand (UWB) technology, Bluetooth (BT) technology and other technologies.

[0131] In an exemplary embodiment, the device 400 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components to perform the data generation method described above.

[0132] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 404 including instructions, is also provided, which can be executed by the processor 420 of the device 400 to perform the data generation method described above. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0133] In another exemplary embodiment, there is also provided a computer program product comprising a computer program being executable by a programmable apparatus, the computer program having code portions for performing the data generating method described above when the computer program is executed by the programmable apparatus.

[0134] In another exemplary embodiment, there is also provided a computer program product comprising a computer program being executable by a programmable apparatus, the computer program having code portions for performing the data generating method described above when the computer program is executed by the programmable apparatus.

[0135] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure. It is intended that the present disclosure cover any and all variations of the present disclosure including those variations contained within the scope of the present disclosure, as well as those adaptations implementing features that are presently or are expected to be developed in the future as part of the present technology. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0136] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data generating method characterized by comprising: The method comprises: performing noise addition on an initial first language text to obtain a first language text after noise addition; reconstructing the first language text after noise addition according to a pre-trained language model to obtain a target first language text, the pre-trained language model being a bidirectional autoregressive transformer model; performing back translation on the target first language text to obtain a second language text; obtaining training data for training a translation model based on the target first language text and the second language text; the first language text is a chapter-level text; the back translation on the target first language text to obtain a second language text comprises: splitting the target first language text to obtain a plurality of target first language subtexts, wherein the first language subtext is a sentence-level text; performing the back translation on the plurality of target first language subtexts according to a back translation model to obtain a plurality of second language subtexts; performing fusion processing on the plurality of second language subtexts to obtain the second language text.

2. The method of claim 1, wherein, The noise addition on the initial first language text to obtain the first language text after noise addition comprises: performing deletion and replacement noise addition on the initial first language text to obtain the first language text after noise addition, the first language text after noise addition being missing a preset number or a preset position of words relative to the initial first language text.

3. The method of claim 1, wherein, The noise addition on the initial first language text to obtain the first language text after noise addition comprises: performing random order noise addition on the initial first language text to obtain the first language text after noise addition, the first language text after noise addition being different in sentence order or word order relative to the initial first language text.

4. The method of claim 1, wherein, The back translation model is trained in the following manner: obtaining a plurality of training samples; wherein each training sample comprises a sample first language subtext and a sample second language subtext; wherein the sample first language subtext and the sample second language subtext are the sentence-level text; iteratively updating parameters of an initial back translation model based on a plurality of training samples to reduce a loss function value corresponding to each training sample to obtain a trained back translation model; wherein the loss function value corresponding to each training sample is determined in the following process: processing the sample first language subtext through the back translation model to obtain a predicted second language subtext; determining the loss function value based on at least the difference between the predicted second language subtext and the sample second language subtext.

5. The method of claim 4, wherein, The back translation model is a transformer model.

6. A data generating apparatus characterized by comprising: The method comprises: a noise addition module configured to perform noise addition on an initial first language text to obtain a first language text after noise addition; a processing module configured to reconstruct the first language text after noise addition according to a pre-trained language model to obtain a target first language text, the pre-trained language model being a bidirectional autoregressive transformer model; a back translation module configured to perform back translation on the target first language text to obtain a second language text; a training data determination module configured to obtain training data for training a translation model based on the target first language text and the second language text; the first language text is a chapter-level text; the back translation module is further configured to: split the target first language text to obtain a plurality of target first language subtexts; wherein the first language subtext is a sentence-level text; perform the back translation processing on the plurality of target first language subtexts according to the back translation model to obtain a plurality of second language subtexts; fuse the plurality of second language subtexts to obtain the second language text.

7. A data generating apparatus characterized by comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the processor-executable instructions in the memory to implement the steps of the method of any one of claims 1-5.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, the program instructions, when executed by the processor, implement the steps of the method of any one of claims 1-5.

Images