A text style transfer model training and text style transfer method and device

Abstract

The embodiment of the present invention provides a text style transfer model training and a text style transfer method and device, which relates to the field of text style transfer technology. The above-mentioned training method includes: obtaining each first corpus pair and each second corpus pair; wherein the first corpus pair includes an initial style text and a first text, and the second corpus pair includes a target style text and a second text, the initial style text and the target style text belong to a target language, and the first text and the second text belong to a specified language, and the specified language is different from the target language; using the first corpus pair and the second corpus pair, establish a correspondence between each target style text and a third text belonging to the initial style; using the correspondence, train the initial text style transfer model to obtain a trained target text style transfer model. Compared with the prior art, the solution provided by the embodiment of the present invention can improve the accuracy of the text transfer results of the trained text style transfer model.

Description

Technical Field

[0001] The present invention relates to the technical field of text style transfer, and in particular to a text style transfer model training and a text style transfer method and device. Background Art

[0002] With the continuous development of text style transfer models, in many cases, users hope to use text style transfer models to convert original text of one style into text of another style, for example, converting spoken Chinese text into written classical Chinese.

[0003] In the related art, the text style of the text to be transferred is usually converted by directly using the trained text style transfer model.

[0004] However, due to the small amount of text style transfer corpus used in training, that is, the small number of training samples and the poor diversity of samples, the text transfer effect of the trained text style transfer model is not good, for example, the accuracy of the output results is poor, etc., and establishing text style transfer corpora one by one will make the training efficiency of the text style transfer model low.

[0005] Based on this, how to improve the text transfer effect of the trained text style transfer model while ensuring the high training efficiency of the text style transfer model has become a problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a text style transfer model training and text style transfer method and device, so as to improve the text transfer effect of the trained text style transfer model on the basis of ensuring high training efficiency of the text style transfer model. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a text style transfer model training method, the method comprising:

[0008] Acquire each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, and each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, and each first text and each second text belong to a specified language, and the specified language is different from the target language;

[0009] Using each first corpus pair and each second corpus pair, establishing a correspondence between each target style text and a third text belonging to the initial style;

[0010] The initial text style transfer model is trained using the corresponding relationship to obtain a trained target text style transfer model.

[0011] Optionally, in a specific implementation, the step of using each first corpus pair and each second corpus pair to establish a correspondence between each target style text and a third text belonging to the initial style includes:

[0012] Using each first corpus pair to train a target translation model for translating the designated language into the target speech;

[0013] Using the target translation model to translate the second texts in each second corpus pair, to obtain third texts belonging to the initial style corresponding to each second text;

[0014] For each second expected pair, a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text is established.

[0015] Optionally, in a specific implementation, the step of using the corresponding relationship to train the initial text style transfer model to obtain a trained target text style transfer model includes:

[0016] Each third text is taken as input, and the target style text corresponding to each third text is taken as output, and the initial text style transfer model is trained to obtain a trained target text style transfer model.

[0017] Optionally, in a specific implementation manner, each of the initial style texts does not include each of the target style texts.

[0018] Optionally, in a specific implementation, the initial text style transfer model is: a Transformer model, or a paragraph-level Transformer model.

[0019] In a second aspect, an embodiment of the present invention provides a text style transfer method, the method comprising:

[0020] Get the text to be migrated;

[0021] Inputting the text to be transferred into a preset target text style transfer model; wherein the target text style transfer model is trained based on any one of the methods provided in the first aspect above;

[0022] The output result of the target text style transfer model is obtained as the target text after the text style transfer is performed on the text to be transferred.

[0023] In a third aspect, an embodiment of the present invention provides a text style transfer model training device, the device comprising:

[0024] A corpus pair acquisition module is used to acquire each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, and each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, and each first text and each second text belong to a specified language, and the specified language is different from the target language;

[0025] A corresponding relationship determination module, used to establish a corresponding relationship between each target style text and a third text belonging to the initial style by using each first corpus pair and each second corpus pair;

[0026] The model training module is used to train the initial text style transfer model using the corresponding relationship to obtain a trained target text style transfer model.

[0027] Optionally, in a specific implementation manner, the corresponding relationship determination module is specifically used to:

[0028] Using each first corpus pair to train a target translation model for translating the designated language into the target speech;

[0029] Using the target translation model to translate the second texts in each second corpus pair, to obtain third texts belonging to the initial style corresponding to each second text;

[0030] For each second expected pair, a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text is established.

[0031] Optionally, in a specific implementation, the model training module is specifically used to:

[0032] Each third text is taken as input, and the target style text corresponding to each third text is taken as output, and the initial text style transfer model is trained to obtain a trained target text style transfer model.

[0033] Optionally, in a specific implementation manner, each of the initial style texts does not include each of the target style texts.

[0034] Optionally, in a specific implementation, the initial text style transfer model is: a Transformer model, or a paragraph-level Transformer model.

[0035] In a fourth aspect, an embodiment of the present invention provides a text style transfer device, the device comprising:

[0036] A text acquisition module, used to acquire the text to be migrated;

[0037] A text output module, used for inputting the text to be transferred into a preset target text style transfer model; wherein the target text style transfer model is trained based on any one of the methods provided in the first aspect above;

[0038] The result acquisition module is used to obtain the output result of the target text style transfer model as the target text after the text style transfer is performed on the text to be transferred.

[0039] In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0040] Memory, used to store computer programs;

[0041] The processor is used to implement the steps of any text style transfer model training method provided in the first aspect and / or the steps of any text style transfer method provided in the second aspect when executing the program stored in the memory.

[0042] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the computer program implements the steps of any text style transfer model training method provided in the first aspect, and / or the steps of any text style transfer method provided in the second aspect.

[0043] In the seventh aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps of any text style transfer model training method provided in the first aspect above, and / or the steps of any text style transfer method provided in the second aspect above.

[0044] Beneficial effects of the embodiments of the present invention:

[0045] As can be seen from the above, by applying the solution provided by the embodiment of the present invention, in the process of training the text style transfer model, firstly, each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text are obtained, wherein each initial style text and each target style text belong to the target language, each first text and each second text belong to the specified language, and the above specified language is different from the above target language. Then, using each first corpus pair and each second corpus pair, a corresponding relationship between each target style text and a third text belonging to the initial style text is established, and finally, using the above corresponding relationship, the initial text style transfer model is trained, thereby obtaining a trained target text style transfer model.

[0046] Based on this, by applying the solution provided by the embodiment of the present invention, the corresponding relationship between each target style text and the third text belonging to the initial style text can be established by using each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text, so that each group of corresponding target style text and third text can be used as a corpus pair for training the style transfer model. In this way, when training the text style transfer model, there is no need to establish each corpus pair for model training one by one, which improves the efficiency of establishing the corpus pairs for model training, and further, on the basis of ensuring the high training efficiency of the text style transfer model, the accuracy of the text transfer result of the trained text style transfer model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0048] Figure 1 A flowchart of a text style transfer model training method provided by an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the model structure of the chapter-level Transformer model;

[0050] Figure 3 A diagram of a specific embodiment of a text style transfer model training method is provided for an embodiment of the present invention;

[0051] Figure 4 A flowchart of a text style transfer method provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the structure of a text style transfer model training device provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the structure of a text style transfer device provided by an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0056] In the related art, due to the small number of paired text style transfer corpora used for training, the text transfer effect of the trained text style transfer model will be poor, for example, poor accuracy, etc., and the small number of paired text style transfer corpora established one by one will make the training efficiency of the text style transfer model low. Based on this, how to improve the text transfer effect of the trained text style transfer model while ensuring the high training efficiency of the text style transfer model has become a problem that needs to be solved urgently.

[0057] In order to solve the above technical problems, an embodiment of the present invention provides a text style transfer model training method.

[0058] The method is applicable to various application scenarios that require text style transfer, for example, converting colloquial Chinese text into classical Chinese, or converting colloquial English text into written language, etc. The method can also be applied to various electronic devices such as laptops, tablet computers, and mobile phones, hereinafter referred to as the first electronic device. Based on this, the embodiment of the present invention does not limit the application scenario and execution subject of the method.

[0059] A text style transfer model training method provided by an embodiment of the present invention may include the following steps:

[0060] Acquire each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, and each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, and each first text and each second text belong to a specified language, and the specified language is different from the target language;

[0061] Using each first corpus pair and each second corpus pair, establishing a correspondence between each target style text and a third text belonging to the initial style;

[0062] The initial text style transfer model is trained using the corresponding relationship to obtain a trained target text style transfer model.

[0063] As can be seen from the above, by applying the solution provided by the embodiment of the present invention, in the process of training the text style transfer model, firstly, each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text are obtained, wherein each initial style text and each target style text belong to the target language, each first text and each second text belong to the specified language, and the above specified language is different from the above target language. Then, using each first corpus pair and each second corpus pair, a corresponding relationship between each target style text and a third text belonging to the initial style text is established, and finally, using the above corresponding relationship, the initial text style transfer model is trained, thereby obtaining a trained target text style transfer model.

[0064] Based on this, by applying the solution provided by the embodiment of the present invention, the corresponding relationship between each target style text and the third text belonging to the initial style text can be established by using each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text, so that each group of corresponding target style text and third text can be used as a corpus pair for training the style transfer model. In this way, when training the text style transfer model, there is no need to establish each corpus pair for model training one by one, which improves the efficiency of establishing the corpus pairs for model training, and further, on the basis of ensuring the high training efficiency of the text style transfer model, the accuracy of the text transfer result of the trained text style transfer model can be improved.

[0065] In the following, a text style transfer model training method provided by an embodiment of the present invention is specifically described in conjunction with the accompanying drawings.

[0066] Figure 1 A text style transfer model training method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps S101-S103.

[0067] S101: Acquire each first corpus pair and each second corpus pair;

[0068] Each first corpus pair includes: an initial style text and a corresponding first text, each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, each first text and each second text belong to a specified language, and the specified language is different from the target language.

[0069] When training a text style transfer model, we can first determine the text style of the text to be transferred and the text style of the desired text. The text style of the text to be transferred can be called the initial style, and the text style of the desired text can be called the target style. Then, we can determine the language type of the text to be transferred as the target language.

[0070] In some embodiments, the text corresponding to the initial style and the text corresponding to the target style belong to the same language type, that is, the language type of the text to be migrated remains unchanged before and after the text style changes, and both can be the target language.

[0071] For the sake of clarity, the text belonging to the initial style and the target language may be referred to as the initial style text, and the text belonging to the target style and the target language may be referred to as the target style text.

[0072] In many cases, in the target language, when using everyday language and professional terms to express the same object or the same meaning, the texts used are different. For example, the cylindrical metal part that supports the inner contour of a car tire and is centrally mounted on an axle is usually called a rim in everyday language, while the professional term in the automotive industry is: wheel hub.

[0073] Based on this, in many fields, users want to convert text composed of daily language into text composed of professional language in the field. Therefore, the text composed of daily language can be used as the initial style text, and the text composed of professional language in the field can be used as the target style text.

[0074] Furthermore, another language of a different type from the above-mentioned target language may be determined as the designated language.

[0075] In this way, each first corpus pair and each second corpus pair can be obtained. Each first corpus pair includes: an initial style text and a corresponding first text, wherein the initial style text and the first text included in each first corpus pair have the same meaning; each second corpus pair includes: a target style text and a corresponding second text, wherein the target style text and the second text included in each second corpus pair have the same meaning.

[0076] For example, the target language is Chinese, the initial style is spoken language, the target style is written language, and the designated language is English. Then each first corpus pair includes: an initial style text belonging to spoken Chinese, and an English text with the same Chinese meaning as represented by the initial style text (the English text is the first text), for example: dad and father; then each second corpus pair includes: a target style text belonging to written Chinese, and an English text with the same Chinese meaning as represented by the target style text (the English text is the second text), for example: father and father.

[0077] For another example, the target language is Chinese, the initial style is spoken language, the target style is written language, and the designated language is English. Then each first corpus pair includes: an initial style text belonging to spoken Chinese, and an English text with the same Chinese meaning as represented by the initial style text (the English text is the first text), for example: dad and dad; then each second corpus pair includes: a target style text belonging to written Chinese, and an English text with the same Chinese meaning as represented by the target style text (the English text is the second text), for example: father and father.

[0078] It can be seen from the above that the first text and the second text may be the same or different, and this embodiment of the present invention does not specifically limit this.

[0079] It should be noted that, in the embodiment of the present invention, the first corpus pairs and the second corpus pairs mentioned above may be obtained in a variety of ways, which are not specifically limited in the embodiment of the present invention.

[0080] For example, the first corpus pairs and the second corpus pairs are obtained from translation dictionaries of the target language and the specified language, for example, Chinese-English dictionaries, etc.; for another example, the first corpus pairs and the second corpus pairs are extracted from publicly published content such as newspapers and web pages.

[0081] Optionally, in a specific implementation, each initial text style in each of the first corpus pairs does not include each target text style in each of the second corpus pairs.

[0082] S102: Using each first corpus pair and each second corpus pair, establish a correspondence between each target style text and a third text belonging to the initial style.

[0083] After obtaining the above-mentioned first corpus pairs and second corpus pairs, the correspondence between the initial style text and the first text in the first corpus pair and the correspondence between the target style text and the second text in the second corpus pair can be used to establish the correspondence between each target style text and the third text belonging to the initial style.

[0084] Optionally, in a specific implementation, the above step S102 may include the following steps 11-13:

[0085] Step 11: using each first corpus pair to train a target translation model for translating a specified language into a target language;

[0086] Step 12: using the target translation model to translate the second texts in each second corpus pair to obtain a third text corresponding to each second text and belonging to the initial style;

[0087] Step 13: For each second expected pair, establish a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text.

[0088] In this specific implementation, after obtaining the above-mentioned first corpus pairs and the above-mentioned second corpus pairs, the above-mentioned first corpus pairs can be used to train a target translation model, and the target translation model is used to translate the above-mentioned specified language into the above-mentioned target language.

[0089] For example, if the target language is Chinese and the specified language is English, the trained target translation model is a model for translating English into Chinese.

[0090] In the process of training to obtain the above-mentioned target translation model, the first electronic device used for model training can pre-build an initial translation model, and then input the above-mentioned first corpus pairs into the initial translation model for training, thereby obtaining the target translation model. During the training process, the initial translation model can learn the semantic features and language conversion of each first corpus pair. After learning a large number of first corpus pairs, the initial translation model gradually establishes a corresponding relationship between the initial style text and the first text, which includes the conversion from the specified language to the target language, thereby obtaining the target translation model.

[0091] The initial translation model may be any model that can realize text translation between different languages, such as a Seq2Seq (Sequence to sequence) translation model with an attention mechanism, a domain adaptation translation model, a Transformer-based translation model, etc.

[0092] Taking the Transformer-based translation model as an example, the model may include: an encoder and a decoder. When using the model for training, each first corpus pair may be input into the model for training, and then a target translation model is obtained. During the training process, the model may include the ability to learn the semantic features of each first corpus pair. After learning a large number of first corpus pairs, the model may gradually establish a corresponding relationship between the initial style text and the first text, thereby obtaining a target translation model.

[0093] When the initial translation model satisfies the first preset condition, the training can be stopped to obtain the target translation model. Optionally, the first preset condition can be that the number of iterations of each first corpus pair reaches the first preset number. Optionally, the first preset condition can be that the error between the true value and the predicted value of the first text corresponding to the initial style text in each first corpus pair is less than the first preset error.

[0094] In this way, after obtaining the above-mentioned target translation model, since the language type of the second text included in each second corpus pair belongs to the specified language, the above-mentioned target translation model can be used to translate the second text in each second corpus pair, wherein, for each second text, the obtained translation result is: a text with the same meaning as that represented by the second text and belonging to the initial style of the target language.

[0095] That is, for each second text, a third text corresponding to the second text and belonging to the initial style can be obtained, and the language type of the third text belongs to the target language.

[0096] In this way, since the target style text corresponding to the same second text and the third text belonging to the initial style have the same meaning, and the language types of the target style text and the third text both belong to the target language, thus, for each second corpus pair, the target style text of the second text included in the second corpus pair and the third text belonging to the initial style corresponding to the second text included in the second corpus pair obtained by using the target translation model can be obtained, and then, the correspondence between the target style text corresponding to the second text included in the second corpus pair and the third text belonging to the initial style can be established.

[0097] That is, for each second text included in the second corpus pair, a correspondence relationship between a second text belonging to the target style of the target language corresponding to the second text and a third text belonging to the initial style of the target language can be established.

[0098] Therefore, after obtaining the third text of the initial style corresponding to the second text included in each second corpus pair by using the target translation model, the corresponding relationship between the target style text and the third text in each second corpus pair can be established. Furthermore, the target style text and the third text included in each established corresponding relationship can constitute a corpus pair for training the style transfer model.

[0099] Optionally, each initial style text included in each first corpus pair obtained above may not include each target style text in each second corpus pair obtained above.

[0100] S103: Using the corresponding relationship, the initial text style transfer model is trained to obtain a trained target text style transfer model.

[0101] The initial text style transfer model is trained using the correspondence between each target style text and the third text belonging to the initial style established above, thereby obtaining a target text style transfer model.

[0102] Optionally, in a specific implementation, the above step S103 may include the following step 22:

[0103] Step 22: Take each third text as input, and take the target style text corresponding to each third text as output, train the initial text style transfer model, and obtain a trained target text style transfer model.

[0104] In this specific implementation, each third text can be used as input, and the target style text corresponding to each third text can be used as output to train the initial text style transfer model. When the initial text style transfer model reaches the second preset condition, the training is stopped to obtain the target text style transfer model.

[0105] Among them, the above-mentioned initial text style transfer model can be various text transfer models that can realize switching between different text styles of the same language, for example, an unsupervised text style transfer model based on a reinforcement learning (Reinforcement Learning) framework, a text style transfer model based on a generative adversarial network (GAN, Generative Model and Discriminative Model), etc., wherein Generative Model is a generative model and Discriminative Model is a discriminative model.

[0106] In the process of training to obtain the above-mentioned target text style transfer model, the first electronic device used for model training can pre-build an initial text style transfer model, and then use each of the above-mentioned third texts as input and the target style text corresponding to each third text as output to train the initial text style transfer model, thereby obtaining the target text style transfer model.

[0107] During the training process, the initial text style transfer model can learn the semantic features of each third text and the target style text corresponding to each third text. After learning the semantic features between a large number of third texts and the target style text corresponding to each third text, the initial text style transfer model gradually establishes the correspondence between the target style text and the third text belonging to the initial style.

[0108] Optionally, the second preset condition may be that the number of iterations of each third text and the target style text corresponding to each third text reaches a second preset number.

[0109] Optionally, the second preset condition may be that the error between the true value and the predicted value of each third text and the target style text corresponding to each third text is less than a second preset value.

[0110] Optionally, the first preset condition and the second preset condition may be the same or different.

[0111] Optionally, the first preset number and the second preset number may be the same or different.

[0112] Optionally, the first preset value and the second preset value may be the same or different.

[0113] In one embodiment, taking an unsupervised text style transfer model based on a reinforcement learning framework as an example, since the style transfer task generally has two style transfer directions, namely, transfer from the target style to the initial style and transfer from the initial style to the target style, a loop can be established in the above two style transfer directions, and the correspondence between each target style text and the third text belonging to the initial style established above is used to train a first sub-model for transferring from a text belonging to the target style to a text belonging to the initial style, and a second sub-model for transferring from a text belonging to the initial style to a text belonging to the target style, respectively. Thus, the output of the first sub-model is used to guide the training of the second sub-model, and the output of the second sub-model is used to guide the training of the first sub-model, until the loss values ​​of the first sub-model and the second sub-model reach predetermined values, and then the training is terminated.

[0114] In this way, the trained first sub-model can be used to convert text of the target style of the target speech into text of the initial style of the target speech, and the trained second sub-model can be used to convert text of the initial style of the target speech into text of the target style of the target speech.

[0115] In another embodiment, taking the text style transfer model based on the generative adversarial network as an example, two encoders are trained, and the two encoders are respectively: a content encoder that extracts the features that determine the text content template and a style encoder that extracts the features that determine the text style. Then, a decoder that can use the feature vector of the above-mentioned text content template and the feature vector that determines the text style to restore the original text is trained. During the training process, the text style transfer model based on the generative adversarial network can be trained as a self-decoder. Furthermore, when performing text style transfer, the text can be input into the above-mentioned content encoder to obtain the feature vector of the text content template, so that the text after style transfer can be obtained by using the decoder to the feature vector of the text content template and the style feature vector given by the above-mentioned style encoder.

[0116] Among them, optionally, in a specific implementation method, the initial text style transfer model can also be: a Transformer model, or a paragraph-level Transformer model.

[0117] Among them, the Transformer model is a conversion model that relies entirely on self-attention to calculate the representation of input and output without using sequence-aligned recursive neural networks or convolutional neural networks.

[0118] Furthermore, since the text is not composed of isolated, unrelated elements, but of juxtaposed and structured groups of sentences, which are bound together by complex language elements, ignoring the relationship between the elements will result in problems such as incoherent sentences and poor logic in the resulting text. Therefore, in order to further improve the effect of text migration, based on the Transformer model, a paragraph-level Transformer model that can introduce the contextual information of each word in the training sample during model training can be used as the initial text migration model.

[0119] Among them, the chapter-level Transformer model is a variant of the Transformer model, which can introduce contextual information of training samples during the training process.

[0120] That is to say, for the paragraph-level Transformer model, it uses a context-aware encoder based on the Transformer model and uses sentence-based pre-trained embedding as the input of the context encoder. In the second stage of training, the model only learns document-level parameters without fine-tuning the sentence-level parameters of the model.

[0121] Among them, when using the paragraph-level Transformer model to train the text style transfer model, for the training text used for model training, it is not necessary to perform cyclic and sequential operations, but all words in the sequence can be processed in parallel, and the self-attention mechanism is used to combine the contextual content of each word with the more distant words. Thus, by processing all words in parallel and letting each word pay attention to other words in the training text in multiple training steps. In this way, by introducing the contextual information of each word in the training text as input, and combining the style-transferred word corresponding to the word as output, the text transfer effect of the trained text style transfer model can be improved.

[0122] That is to say, in this specific implementation method, when the paragraph-level Transformer model is trained by using the correspondence between the target style text corresponding to the second text in each corpus pair established above and the third text to obtain the target text style transfer model, for each set of correspondences, the third text in the set of correspondences can be used as the input of the paragraph-level Transformer model, and, for each word in the third text, the contextual information of the word in the third text is also introduced as input to the paragraph-level Transformer model, and then the target style text in the set of correspondences is used as output.

[0123] In this way, when the paragraph-level Transformer model establishes the correspondence between the target style text and the third text belonging to the initial style by learning the semantic features of the third text and the target style text in the above-mentioned groups of correspondence, it can pay attention to the order of each word included in each third text and the logical relationship between them, so as to improve the accuracy of the text migration result of the text style transfer model finally obtained.

[0124] For example, Figure 2The model structure of the chapter-level Transformer model is shown, wherein the chapter-level Transformer model includes: context embedding layer, self-attention mechanism (Self-Attention) layer, feed-forward layer, source embedding (Source-Embedding) layer, softmax layer, encoder-decoder attention (Encoder-Decoder Attention) layer and target embedding (TargetEmbedding) layer.

[0125] For example, Figure 3 The specific training method of the text style transfer model is shown, wherein the translation corpus A is the first corpus pair, each first corpus pair includes: an initial style text and a corresponding first text, the translation corpus B is the second corpus pair, each second corpus pair includes: a target style text and a corresponding second text, language a is a specified language, language b is a target language, each initial style text and each target style text belong to the target language b, each first text and each second text belong to the specified language a, the specified language a is different from the target language b, the training translation model is a translation model about the specified language a, the translation model Model-A is the target translation model, the domain translation corpus B is the second corpus pair, the back-translation corpus B' is the third text, the paragraph-level Transformer model is the initial text style transfer model, and the text style transfer model Model-B is the target text style transfer model.

[0126] The translation model is trained using translation corpus A to obtain translation model Model-A. The domain translation corpus B is input into translation model Model-A to obtain back-translation prediction B'. Then, the paragraph-level Transformer model is trained using domain corpus B and back-translation prediction B' to obtain text style transfer model Model-B.

[0127] As can be seen from the above, by applying the solution provided by the embodiment of the present invention, the first corpus pairs including the initial style text and the corresponding first text, and the second corpus pairs including the target style text and the corresponding second text can be used to establish the correspondence between each target style text and the third text belonging to the initial style text, so that each group of corresponding target style text and third text can be used as a corpus pair for training the style transfer model. In this way, when training the text style transfer model, there is no need to establish each corpus pair for model training one by one, which improves the efficiency of establishing the corpus pairs for model training, and further improves the training efficiency of the text style transfer model.

[0128] Corresponding to the text style transfer model training method provided by the above-mentioned embodiment of the present invention, the embodiment of the present invention also provides a text style transfer method.

[0129] Among them, the method is applicable to various application scenarios that require text style transfer, for example, converting colloquial Chinese text into classical Chinese, and for example, converting text composed of daily language into text composed of professional language, etc. And the method can be applied to various electronic devices such as laptops, tablets, and mobile phones, hereinafter referred to as the second electronic device. Based on this, the embodiment of the present invention does not limit the application scenario and execution subject of the method.

[0130] It should be noted that the first electronic device and the second electronic device may be the same electronic device or different electronic devices.

[0131] Figure 4 A text style transfer method provided by an embodiment of the present invention, such as Figure 4 As shown, the method includes the following steps S401-S403.

[0132] S401: Obtain the text to be migrated;

[0133] S402: Inputting the text to be transferred into a preset target text style transfer model;

[0134] The target text style transfer model is obtained based on any of the above text style transfer model training methods;

[0135] S403: Obtain the output result of the target text style transfer model as the target text after the text style transfer is performed on the text to be transferred.

[0136] In this specific implementation method, the text to be transferred is first obtained, and then the above text to be transferred is input into the target text style transfer model obtained in advance by any of the above text style transfer model training methods, so that the target text style transfer text model can process the above text to be transferred to obtain the semantic features of the text to be transferred, and then, by using the established correspondence between each target style text and the third text belonging to the initial style, the output result corresponding to the text to be transferred is obtained as the target text after the style transfer of the text to be transferred.

[0137] Based on this, by applying the solution provided by the embodiment of the present invention, the text to be transferred can be input into the target text style transfer model, and the correspondence between each target style text and the third text belonging to the initial style text can be used to directly obtain the target text after the style transfer of the text to be transferred, thereby realizing text style transfer.

[0138] Corresponding to the text style transfer model training method provided by the above-mentioned embodiment of the present invention, the embodiment of the present invention provides a text style transfer model training device.

[0139] Figure 5 A structural diagram of a text style transfer model training device provided by an embodiment of the present invention, such as Figure 5 As shown, the device may include the following modules:

[0140] The corpus pair acquisition module 510 is used to acquire each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, and each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, and each first text and each second text belong to a specified language, and the specified language is different from the target language;

[0141] A correspondence determination module 520, for establishing a correspondence between each target style text and a third text belonging to the initial style by using each first corpus pair and each second corpus pair;

[0142] The model training module 530 is used to train the initial text style transfer model using the corresponding relationship to obtain a trained target text style transfer model.

[0143] As can be seen from the above, by applying the solution provided by the embodiment of the present invention, in the process of training the text style transfer model, firstly, each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text are obtained, wherein each initial style text and each target style text belong to the target language, each first text and each second text belong to the specified language, and the above specified language is different from the above target language. Then, using each first corpus pair and each second corpus pair, a corresponding relationship between each target style text and a third text belonging to the initial style text is established, and finally, using the above corresponding relationship, the initial text style transfer model is trained, thereby obtaining a trained target text style transfer model.

[0144] Based on this, by applying the solution provided by the embodiment of the present invention, the corresponding relationship between each target style text and the third text belonging to the initial style text can be established by using each first corpus pair including the initial style text and the corresponding first text, and each second corpus pair including the target style text and the corresponding second text, so that each group of corresponding target style text and third text can be used as a corpus pair for training the style transfer model. In this way, when training the text style transfer model, there is no need to establish each corpus pair for model training one by one, which improves the efficiency of establishing the corpus pairs for model training, and further, on the basis of ensuring the high training efficiency of the text style transfer model, the accuracy of the text transfer result of the trained text style transfer model can be improved.

[0145] Optionally, in a specific implementation, the corresponding relationship determination module 520 is specifically used to:

[0146] Using each first corpus pair to train a target translation model for translating the designated language into the target speech;

[0147] Using the target translation model to translate the second texts in each second corpus pair, to obtain third texts belonging to the initial style corresponding to each second text;

[0148] For each second expected pair, a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text is established.

[0149] Optionally, in a specific implementation, the model training module 530 is specifically used to:

[0150] Each third text is taken as input, and the target style text corresponding to each third text is taken as output, and the initial text style transfer model is trained to obtain a trained target text style transfer model.

[0151] Optionally, in a specific implementation manner, each of the initial style texts does not include each of the target style texts.

[0152] Optionally, in a specific implementation, the initial text style transfer model is: a Transformer model, or a paragraph-level Transformer model.

[0153] Corresponding to the text style transfer method provided by the above-mentioned embodiment of the present invention, the embodiment of the present invention provides a text style transfer device.

[0154] Figure 6 A structural diagram of a text style transfer device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device may include the following modules:

[0155] A text acquisition module 610 is used to acquire the text to be migrated;

[0156] A text output module 620 is used to output the text to be transferred to a preset target text style transfer model; wherein the target text style transfer model is obtained by training based on the text style transfer method described above;

[0157] The result acquisition module 630 is used to obtain the output result of the target text style transfer model as the target text after the text style transfer is performed on the text to be transferred.

[0158] Based on this, by applying the solution provided by the embodiment of the present invention, the text to be transferred can be input into the target text style transfer model, and the correspondence between each target style text and the third text belonging to the initial style text can be used to directly obtain the target text after the style transfer of the text to be transferred, thereby realizing text style transfer.

[0159] Corresponding to the text style transfer model training and text style transfer device provided in the above embodiment of the present invention, the embodiment of the present invention also provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0160] Memory 703, used for storing computer programs;

[0161] The processor 701 is used to implement the steps of any text style transfer model training method provided by the above-mentioned embodiments of the present invention and / or the steps of any text style transfer method provided by the above-mentioned embodiments of the present invention when executing the program stored in the memory 703.

[0162] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0163] The communication interface is used for communication between the above electronic device and other devices.

[0164] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0165] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0166] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any text style transfer model training method provided by the above-mentioned embodiments of the present invention and / or the steps of any text style transfer method provided by the above-mentioned embodiments of the present invention are implemented.

[0167] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided. When the computer program product is run on a computer, the computer executes the steps of any text style transfer model training method provided by the above-mentioned embodiments of the present invention, and / or the steps of any text style transfer method provided by the above-mentioned embodiments of the present invention.

[0168] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0169] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0170] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, electronic device embodiment, computer-readable storage medium embodiment, and computer program product embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A text style transfer model training method, characterized in that: The method comprises: Obtain each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, each first text and each second text belong to a specified language, and the specified language is different from the target language; the initial style text and the first text included in each first corpus pair have the same meaning; the target style text and the second text included in each second corpus pair have the same meaning; Using each first corpus pair and each second corpus pair, establishing a correspondence between each target style text and a third text belonging to the initial style; Using the corresponding relationship, the initial text style transfer model is trained to obtain a trained target text style transfer model; The step of establishing a correspondence between each target style text and a third text belonging to the initial style by using each first corpus pair and each second corpus pair comprises: Using each first corpus pair to train a target translation model for translating the designated language into the target language; Using the target translation model to translate the second texts in each second corpus pair, to obtain third texts belonging to the initial style corresponding to each second text; For each second corpus pair, a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text is established.

2. The method according to claim 1, characterized in that The step of using the corresponding relationship to train the initial text style transfer model to obtain a trained target text style transfer model includes: Each third text is taken as input, and the target style text corresponding to each third text is taken as output, and the initial text style transfer model is trained to obtain a trained target text style transfer model.

3. The method according to claim 1, characterized in that The respective initial style texts do not include the respective target style texts.

4. The method according to any one of claims 1 to 3, characterized in that: The initial text style transfer model is: a Transformer model, or a paragraph-level Transformer model.

5. A text style transfer method, characterized in that: The method comprises: Get the text to be migrated; Inputting the text to be transferred into a preset target text style transfer model; wherein the target text style transfer model is trained based on the method according to any one of claims 1 to 4; The output result of the target text style transfer model is obtained as the target text after the text style transfer is performed on the text to be transferred.

6. A text style transfer model training device, characterized in that: The device comprises: A corpus pair acquisition module is used to acquire each first corpus pair and each second corpus pair; wherein each first corpus pair includes: an initial style text and a corresponding first text, and each second corpus pair includes: a target style text and a corresponding second text, each initial style text and each target style text belong to a target language, and each first text and each second text belong to a specified language, which is different from the target language; the initial style text and the first text included in each first corpus pair have the same meaning; the target style text and the second text included in each second corpus pair have the same meaning; A corresponding relationship determination module, used to establish a corresponding relationship between each target style text and a third text belonging to the initial style by using each first corpus pair and each second corpus pair; A model training module, used to train the initial text style transfer model using the corresponding relationship to obtain a trained target text style transfer model; The corresponding relationship determination module is specifically used for: Using each first corpus pair to train a target translation model for translating the designated language into the target language; Using the target translation model to translate the second texts in each second corpus pair, to obtain third texts belonging to the initial style corresponding to each second text; For each second corpus pair, a corresponding relationship between the target style text corresponding to the second text in the second corpus pair and the third text is established.

7. A text style transfer device, characterized in that: The device comprises: A text acquisition module, used to acquire the text to be migrated; A text output module, used for inputting the text to be transferred into a preset target text style transfer model; wherein the target text style transfer model is trained based on the method according to any one of claims 1 to 4; The result acquisition module is used to obtain the output result of the target text style transfer model as the target text after the text style transfer is performed on the text to be transferred.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 4 and / or the method steps described in claim 5 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 and / or the method steps described in claim 5 are implemented.

Images