Text multi-style transfer method based on style representation and multi-task learning

Abstract

The present invention proposes a text multi-style transfer method based on style representation and multi-task learning, comprising the following steps: (1) constructing a multi-corpus text graph network, (2) using graph network technology and graph embedding algorithm to obtain graph node representation, (3) using label embedding technology to obtain text style representation, (4) using Transformer to construct an encoder to obtain sentence representation that integrates text style information, (5) using Transformer to construct a decoder to perform style transfer on the target text, and (6) using the parameter hard sharing method in the multi-task learning method to construct a text multi-style transfer model. The present invention uses graph neural network and label embedding technology to obtain a text style representation with better effect; constructs a text style transfer model based on the combination of Transformer and style representation to obtain a better text style transfer effect; uses multi-task technology to construct a text multi-style transfer model to reduce the influence between different styles and obtain a better text multi-style transfer effect.

Description

Technical Field

[0001] The present invention belongs to the field of text processing technology and relates to a text multi-style transfer method based on deep learning. More specifically, the present invention proposes a text multi-style transfer method based on style representation and multi-task learning. Background Art

[0002] The goal of text style transfer is to change the expressive style of a text while maintaining the subject matter. Current research on text style transfer primarily focuses on emotional style, scene style, and gender-specific style. The style transfer task originated in the field of image processing, where commonly used approaches include reinforcement learning and pseudo-aligned data generation. However, research on multi-style transfer in text often uses model concatenation and controllable text generation.

[0003] The reinforcement learning paradigm assumes that the content and style of a text can be separated. Therefore, the reinforcement learning text style transfer model consists of two main parts: the first is a style discrimination module, and the second is a content reconstruction module. The reinforcement learning method uses a style classification discriminator to determine the extent to which the transferred text expresses opposing styles and thereby derives a style score. The content reconstruction module, on the other hand, uses a text generation model to reconstruct the source text to ensure that the transferred text is thematically consistent with the source text. The style scores are then integrated into the model update process to complete the generation of the transferred text. However, the separation of text style and content is often difficult, resulting in poor performance of reinforcement learning methods.

[0004] A key reason for using reinforcement learning is the lack of parallel corpora for text style transfer tasks. To alleviate this problem, methods based on pseudo-aligned data generation have also gained attention. Some researchers have proposed a deletion and retrieval-based approach, using the machine learning algorithm TFIDF to model text based on word co-occurrence. Stylistic keywords are obtained from texts of two different styles and replaced or deleted based on semantic similarity to generate pseudo-parallel corpora. Other researchers have also used back-translation methods to obtain pseudo-aligned data and used Seq2Seq structures on the transfer generation side to generate texts in opposing styles. However, style transfer methods based on pseudo-aligned data are highly dependent on the quality of the pseudo-aligned data. In the case of poor dataset quality, these methods perform unstably.

[0005] Based on existing single-style text transfer methods, different text style transfer targets can be trained separately through model concatenation, and then used in tandem when applied. However, this concatenation approach requires multiple training sessions, which consumes training resources. Furthermore, because different style transfer models are trained separately, each style becomes independent and unconnected, which in turn reduces the overall multi-style transfer performance. Therefore, some researchers have used controllable text generation methods, using LSTM as a base model, combining input text and style label information to construct a DAE architecture, and using back-translation to control the properties of text generation to achieve multi-style text transfer. Other researchers have used the reinforcement learning paradigm to address the multi-style transfer problem, utilizing multiple style discriminators to score the generated text on different style dimensions, and combining the style loss with the generative model loss to achieve multi-style text transfer. However, the reinforcement learning paradigm assumes that text content and style are separate, but in reality, the two are often difficult to separate. Furthermore, existing multi-style text transfer methods do not effectively address the mutual influence of different styles, resulting in a decline in the quality of multi-style transfer.

[0006] In summary, how to solve the problem of difficulty in separating style and content in style transfer; how to reduce the influence between different styles in the study of multi-style transfer of text are important issues in current style transfer research. Summary of the Invention

[0007] To overcome the shortcomings of the existing technology, the present invention provides a text multi-style transfer method based on style representation and multi-task learning. The method first constructs a multi-corpus text graph network, placing text information and style information in the same space for representation; then uses graph network technology and the graph embedding algorithm Node2Vec to obtain graph node representations, and further uses label embedding technology to obtain word representations and text style representations; further uses Transformer to construct an encoder to obtain sentence representations that incorporate text style information, and also uses Transformer to construct a decoder to transfer the style of the target text; finally, uses the parameter hard sharing method in the multi-task learning method to construct a text multi-style transfer model. On the basis of strengthening the underlying connection between different tasks, it reduces the coupling between different tasks and constructs a text style transfer model for multiple styles.

[0008] In order to achieve the above-mentioned purpose and solve the problems existing in the prior art, the technical solution adopted by the present invention is: a text multi-style transfer method based on style representation and multi-task learning, comprising the following steps:

[0009] Build a multi-lingual text graph network to strengthen the correlation between style information and text information;

[0010] Through graph network technology and label embedding methods, Node2Vec is used to obtain graph node representations and extract text representations and style representations;

[0011] Use Transformer to build a generation model and integrate style representation to build a text style transfer model;

[0012] By utilizing the hard sharing of parameters in the multi-task learning method, different text style transfer targets are regarded as different tasks, the underlying parameters of text style transfer models of different styles are shared, and multi-task is used to reduce the coupling relationship between different tasks to construct a text multi-style transfer model.

[0013] Furthermore, a text graph network is constructed from multiple dimensions to strengthen the connection between text information and text information, and between text information and style information. Step 1, constructing a text graph network of multiple corpora, specifically includes the following steps:

[0014] Step 1.1: Construct a local text graph, as shown in formulas (1) to (5):

[0015] Data={S,L} (1)

[0016] S={Sentence1…Sentence n} (2)

[0017] L={label1…label m} (3)

[0018] Sentence={word1…word p} (4)

[0019] G local (Dataset) = {(word i ,word j )|left≤i,j≤right} (5)

[0020] Among them, Data is the style dataset used, which consists of two parts, S and L. S is the sentence set of the dataset, the number of sentences is n, and L is the style label set of the dataset, the number of style label labels is m. In the text style transfer task, m is often 2. Sentence is a sentence in the dataset, consisting of multiple words, the number of words is p, where word represents a word; the local text graph construction method is shown in formula (5), using a sliding window to slide on all sentences in the dataset, where left is the left boundary of the sliding window, and right is the right boundary of the sliding window. The words in the window are represented as graph nodes and connected in pairs. After completing the sliding and connecting operations on all sentences in the dataset, the local text graph G is obtained. local;

[0021] Step 1.2: Construct a global text graph, as shown in formula (6):

[0022] G global ={(word u ,label v )|word u ∈Sentence,label v ∈L} (6)

[0023] Among them, word u is the word in the sentence Sentence, and label v The style label corresponding to the sentence is represented as a node in the graph, and the word node is connected with its corresponding style label node. After completing the connection operation on all sentences in the dataset, the global text graph G is obtained. global ;

[0024] Step 1.3: Construct a multi-corpus text graph, as shown in formula (7):

[0025] G multi ={G local (Data1),G local (Data2),G global (Data1),G global (Data2)} (7)

[0026] Among them, Data1 and Data2 are datasets of different styles. In the same graph network, the local text graph G of Data1 is constructed at the same time. local (Data1) and the global text graph G global (Data1), and construct the local text graph G of Data2 local (Data2) and the global text graph G global (Data2), that is, the multi-corpus text graph G is obtained multi .

[0027] Furthermore, in step 2, the method of obtaining style representation using graph network technology and label embedding method is specifically as follows:

[0028] In step 2.1, the multi-corpus text graph constructed in claim 1 is used as the input of the style representation method, and its structure is shown in formulas (8) and (9):

[0029] G multi ={E,N} (8)

[0030] N={N word ,N label} (9)

[0031] Where E represents the edge set of the multi-corpus text graph, N represents the node set of the multi-corpus text graph, and N consists of two parts: the word node set N word and style tag node set N label ;

[0032] In step 2.2, the Node2Vec deep walk algorithm is used to sample the node set N in the multi-corpus text graph structure in step 2.1 to obtain the node sampling set K. The sampling method is shown in formulas (10) and (11):

[0033] N={node1…node r} (10)

[0034] K={(node1…node d ) j |node i+1 ∈Neighbour(node i ),1≤i <d,1≤j≤t} (11)

[0036] Among them, t is the number of sampling times, d is the maximum depth of a single sampling, Neighbour (node i ) represents the node i The set of connected neighbor nodes; in the jth sampling process, a node node is randomly selected from the node set N i As the starting node, and select node node from its neighbor node set i+1 , and loop through the nodes i+1 As the starting node for sampling, after reaching the maximum depth, the word sampling result (node1…node d ) j , after random sampling t times, the sampling set K can be obtained.

[0037] In step 2.3, the SkipGram algorithm is used to train the node sampling set obtained in step 2.2 to obtain the graph node representation. The static representation is separated to obtain the word representation and style representation respectively. The acquisition method is described by formulas (12) to (14):

[0038] SkipGram={Emb word ,Emb label} (12)

[0039] K={(node 11 …node 1d )1,…(nodet1 …node td ) t} (13)

[0040]

[0041] Among them, SkipGram is the training model, and Emb word is the representation matrix of the word node set, and Emb label is the representation matrix of the label node. Let the window size during training be w, and the node sampling set K consists of multiple nodes. Let the j-th node in the i-th sampling be node ij , and the j±g-th node be node ij±g , where g is a value and 1≤g<w; during the training phase, the central word node ij predicts other words node ij±g within the window. After training on all sampling examples, the trained SkipGram model is obtained, where Emb word is the trained word representation, and Emb label is the trained style representation.

[0042] Furthermore, the method of combining the style representation with Transformer to construct a style transfer model is specifically as follows:

[0043] Combine the style representation obtained in step 2.3 with Transformer to construct a style transfer model, and its construction method is described by formulas (15) to (20):

[0044] Model={Emb m , Encoder, Decoder} (15)

[0045] Emb m ={Emb word , Emb label} (16)

[0046] Emb label ={Emb source , Emb target} (17)

[0047] Context = Encoder(Sentence source ) (18)

[0048] P h-1 = W * Concat(Y h-1 , Emb target ) + B (19)

[0049] Yh =Decoder(P h-1 ,Context) (20)

[0050] Among them, Model is the model, and is represented by the matrix Emb m , encoder Encoder and decoder Decoder, the representation matrix Emb in the model m The word representation matrix Emb word and style representation matrix Emb label Composition, of which Emb label Contains source style representation Emb source Embed with target style representation target ; During the model training process, the encoder is used to input the text Sentence source The context vector Context is obtained by encoding; in the style fusion process of formula (19), the randomly initialized weight W and bias vector B are defined, and the output Y of the h-1 time step in the decoding process is converted to h-1 Embed with target style representation target After concatenation by the Concat method, the time step output P that integrates the style representation is obtained. h-1 , and in the next h-th time step decoding process, the context vector Context and P h-1 Get the word Y predicted at the current time step h .

[0051] Furthermore, the text multi-style transfer method based on multi-task learning is specifically as follows:

[0052] Constructing multiple text style transfer models for different styles strengthens the underlying connection between different tasks through parameter hard sharing, while reducing the degree of coupling between different style transfer tasks. The construction method is described by formulas (21) to (23):

[0053] Model multi ={Model1,Model2} (21)

[0054] Model1={Emb word ,Emb label ,Weight enc1 ,Weight dec1} (twenty two)

[0056] Model2={Emb word ,Emb label ,Weight enc2 ,Weight dec2} (twenty three)

[0058] Among them, Model multi The constructed text multi-style transfer model consists of multiple text style transfer models Model1 and Model2, where Model1 and Model2 share the same word representation matrix Emb word and style representation matrix Emb label , and the encoder parameters Weight in different models enc and decoder weights dec They are independent of each other.

[0059] Furthermore, the training method of the text multi-style transfer model is as follows:

[0060] In step 5.1, the model is preheated and trained to make different task models available. The training method is described by formulas (24) and (25):

[0061]

[0062]

[0063] Among them, for the models Model1 and Model2 of different tasks, the text pairs related to their tasks are used respectively. source1 ,Sentence target1 ) and (Sentence source2 ,Sentence target2 ) are trained respectively;

[0064] In step 5.2, the model connects different task models through joint training to achieve the goal of text multi-style transfer. The training method is described by formula (26);

[0065] Sentence source →Model1→C→Model2→Sentence target (26)

[0067] Among them, the input text Sentence source By cascading, after passing through Model1, a high-dimensional vector C containing the target style information is obtained, and then further input into Model2 to obtain the output text Sentence containing multiple style information target .

[0068] A device includes a processor and a memory, wherein the processor executes code in the memory to implement the above method.

[0069] A computer storage medium stores a computer program, wherein the computer program is executed by hardware to implement the above method.

[0070] The beneficial effects of the present invention are: using graph neural network and label embedding technology to obtain better text style representation; building a text style transfer model based on the combination of Transformer and style representation to obtain better text style transfer effect; using multi-task technology to build a text multi-style transfer model to reduce the influence between different styles and obtain better text multi-style transfer effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flow chart of the steps of the method of the present invention.

[0072] Figure 2 This is the framework diagram of the text style transfer model proposed in this invention. DETAILED DESCRIPTION

[0073] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0074] Example 1:

[0075] The Data style datasets used in this example are Yelp and Gender, standard datasets for English text style transfer. Both are non-parallel datasets. The Yelp data is a sentiment-positive style dataset, containing 260,000 positive style samples and 180,000 negative style samples. The Gender dataset is a gendered style dataset, containing 130,000 masculine style samples and 130,000 feminine style samples. Due to the large data volume, the datasets were sampled during the experiments. The dataset information is shown in Tables 1 and 2 below.

[0076] Table 1

[0077]

[0078] Table 2

[0079]

[0080] The evaluation indicators used in the method of this embodiment are based on the average BLEU value of the multi-language model, the perplexity (PPL) under the multi-language model, and the style transfer score ACC. In the style transfer of the embodiment, two methods are used to evaluate the quality of the transferred text. First, the accuracy index (Accuracy) of the style discriminator on the test set is used to evaluate its style transfer degree, and the BLEU index is used to measure its content preservation. The calculation formula is shown in formula (27). In the multi-style transfer research, this embodiment uses Self-BLEU to measure its content preservation. The calculation method is the same as BLEU. At the same time, this embodiment uses the unsupervised (PPL) perplexity index to measure the fluency of the generated text. The calculation formula is shown in formula (28).

[0081]

[0082]

[0083] The indicators of the model results proposed in the present invention were compared with the five models of CA-series, DRG-series, ASE-Series, MA and OTM. Among them, CA-series was proposed in 2017 to construct a text style transfer model using the cross-alignment method, and multiple style models were connected in series to construct a multi-style transfer model. DRG-series was proposed in 2018 to construct a text style transfer model using the method of style keyword deletion and reconstruction, and multiple style models were connected in series to construct a multi-style transfer model. ASE-Series was proposed in 2020 to obtain style embedding and construct a style transfer model through a classification model, and multiple style models were connected in series to construct a multi-style transfer model. MA was proposed in 2018 to use LSTM as the base model, combine the input text and style label information to construct a DAE architecture, and use the back-translation method to control the attributes of text generation, thereby completing text multi-style transfer. OTM was proposed in 2021 to construct a text multi-style transfer model by utilizing multiple style discriminators and combining reinforcement learning methods.

[0084] Table 3

[0085]

[0086] The experimental results in Table 3 show that the model proposed in this embodiment achieves better results in text multi-style transfer overall. The style ACC on the Yelp dataset is 2% higher than the original model, and the style ACC on the Gender dataset is roughly the same as the existing model. The best result is achieved in the multi-data average style ACC, indicating that the model is superior to the existing model in style transfer capabilities. The SBLEU metric achieves the best content preservation score, which is 10% higher than the existing model, indicating that the model retains the source text theme better than the existing model.

[0087] Table 4

[0088]

[0089]

[0090] This example further validates the effectiveness of a single style transfer model. The results, shown in Table 4, show that the model of this example significantly outperforms existing models in terms of stylization metrics across different datasets, achieving a 5% improvement on the Yelp dataset and a 4% improvement on the Gender dataset. Furthermore, based on parallel test sets within the dataset, the content preservation of text generated by different models is compared. As shown in the table, the BLEU metric on the Yelp dataset is close to that of the ASE model, while achieving the best content preservation on the Gender dataset, with an improvement of 17%. In summary, the model proposed in this example generally outperforms existing models in terms of content preservation and style scores, and also achieves good results in terms of text fluency scores.

[0091] Table 5

[0092]

[0093] Furthermore, this example compares the performance of a multi-style transfer model with a single-style model on the same dataset and calculates the reduction ratio of their style metrics to measure the influence of different styles and their negative impact on the model's final results. The results are shown in Table 5. The model proposed in this chapter achieves the smallest reduction ratio, indicating that compared with other multi-style transfer models, the use of text methods can reduce the interaction between multiple styles.

[0094] Table 6

[0095]

[0096] This example further conducted an ablation experiment to verify the effectiveness of the hard parameter sharing model. By building a multi-style transfer model with independent embedding matrices and comparing it with the original model, the results are shown in Table 6. As can be seen from the results in the table, the model using independent parameters lacks the connection between text information and different style information, and its performance in multi-style transfer is generally inferior to that of the hard parameter sharing model.

[0097] Table 7

[0098]

[0099] Furthermore, this embodiment experimentally verifies the two-stage training method. The original model first undergoes warm-up training to warm up the two style transfer models, and then performs joint training. The comparison model only performs warm-up training or joint training, and the results are shown in Table 7. The effect of using warm-up training or joint training alone is not as good as the two-stage training method. When using warm-up training alone, the two style transfer models are not fused. In the prediction stage, it is more similar to a series model. Although there is a certain transfer effect, the effect is obviously inferior to the two-stage training method. When using joint training alone, the parameters of the different style transfer models on the multi-task end are randomly initialized, which leads to an unstable model state in the early stage of training. Therefore, the model requires more training time to obtain a stable transfer effect. It can be seen that using the joint training method alone is inferior to the two-stage training method. This embodiment further provides an example of text multi-style transfer, and the results are shown in Table 8.

[0100] Table 8

[0101]

[0102] In summary, the method of the present invention has the following advantages: a more effective style representation is obtained by using graph neural network and label embedding technology; a text style transfer model is constructed based on the combination of Transformer and style representation, which achieves a better transfer effect; a text multi-style transfer model is constructed using multi-task technology, which reduces the influence between different styles and achieves a better text multi-style transfer effect.

Claims

1. A text multi-style transfer method based on style representation and multi-task learning, characterized by: The following steps are involved: Step 1: Build a multi-language text graph network, which includes the following steps: Step 1.1, construct a local text graph, as shown in formulas (1) to (5): （1） （2） （3） （4） （5） Where Data is the style dataset used, which consists of two parts, S and L. S is the sentence set of the dataset, the number of sentences is n, L is the style label set of the dataset, the number of style label labels is m, Sentence is a sentence in the dataset, consisting of multiple words, the number of words is p, where word represents a word; the local text graph construction method is shown in formula (5), using a sliding window to slide on all sentences in the dataset, where left is the left boundary of the sliding window, and right is the right boundary of the sliding window. The words in the window are represented as graph nodes and connected in pairs. After completing the sliding and connecting operations on all sentences in the dataset, the local text graph is obtained. ; Step 1.2, construct the global text graph, as shown in formula (6): （6） in, For Sentences The words in The style label corresponding to the sentence is represented as a node in the graph, and the word node is connected to its corresponding style label node. After completing the connection operation on all sentences in the dataset, the global text graph is obtained. ; Step 1.3, construct a multi-corpus text graph, as shown in formula (7): （7） in, and Different styles of data sets are constructed simultaneously in the same graph network. Local text map of With global text map , and build Local text map of With global text map , that is, we get the multi-corpus text graph ; Step 2: Using graph network technology and label embedding method, use Node2Vec to obtain the graph node representation of the text graph network in step 1, and extract the text representation and style representation; Step 3: Use Transformer to build a generation model, and integrate the style representation and word representation in step 2 to build a text style transfer model; Step 4: Utilize the hard sharing of parameters in the multi-task learning method, treat different text style transfer targets as different tasks, share the underlying parameters of text style transfer models of different styles, and use multi-task to reduce the coupling relationship between different tasks to build a text multi-style transfer model.

2. The text multi-style transfer method based on style representation and multi-task learning as claimed in claim 1, characterized in that: The method of obtaining style representation using graph network technology and label embedding method in step 2 is specifically as follows: Step 2.1, using the multi-corpus text graph constructed in claim 1 as the input of the style representation method, the structure of which is shown in formulas (8) and (9): （8） （9） in Represents the edge set of the multi-corpus text graph, N represents the node set of the multi-corpus text graph, where N consists of two parts, namely the word node set A collection of nodes with style tags ; Step 2.2: Use the Node2Vec deep walk algorithm to sample the node set N in the multi-corpus text graph structure in step 2.1 to obtain the node sampling set K. The sampling method is shown in formulas (10) and (11): （10） （11） Among them, t is the number of sampling times, d is the maximum depth of a single sampling, Representatives and The set of connected neighbor nodes; in the jth sampling process, a node is randomly selected from the node set N As the starting node, select a node from its neighbor node set , and cyclically As the starting node for sampling, after reaching the maximum depth, the result of word sampling can be obtained , after random sampling t times, the sampling set K can be obtained.

3. Step 2.3, use the SkipGram algorithm to train based on the node sampling set obtained in step 2.2 to obtain the graph node representation, and separate the static representation to obtain the word representation and style representation respectively. The acquisition method is described by formulas (12) to (14): （12） （13） （14） in, SkipGram is the training model, is the representation matrix of the word node set, is the representation matrix of the label node, the window size during training is set to w, the node sampling set K consists of multiple nodes, and the jth node in the i-th sampling is set to , and the The nodes are , where g is a numerical value and ; In the training phase, the central word Other words in the prediction window After training on all sample examples, the trained SkipGram model is obtained, where is the trained word representation, is the trained style representation.

4. The text multi-style transfer method based on style representation and multi-task learning as claimed in claim 2, characterized in that: The specific method of building a text style transfer model is: The style representation obtained in step 2.3 is combined with Transformer to construct a style transfer model, which is described by formulas (15) to (20): （15） （16） （17） （18） （19） （20） in, is the model, and is represented by the matrix , encoder Encoder and decoder Decoder, the representation matrix in the model Representation matrix by word and the style representation matrix Composition, among which Contains source style representation With the target style representation ; During the model training process, the input text is processed by the encoder Encode to get the context vector ; In the style fusion process of formula (19), the randomly initialized weight W and bias vector B are defined to convert the output of h-1 time step in the decoding process With the target style representation After concatenation through the Concat method, the operation obtains the time step output that integrates the style representation , and in the next h-th time step decoding process, the context vector Context is used with Get the word predicted at the current time step .

5. The text multi-style transfer method based on style representation and multi-task learning as claimed in claim 1, characterized in that: The text multi-style transfer method based on multi-task learning is as follows: Constructing multiple text style transfer models for different styles, through hard parameter sharing, strengthens the underlying connection between different tasks and reduces the degree of coupling between different style transfer tasks. The construction method is described by formulas (21) to (23): （21） （22） （23） in, The constructed text multi-style transfer model consists of multiple text style transfer models and Composition, among which and Share the same word representation matrix and the style representation matrix , while the encoder parameters in different models and decoder weights They are independent of each other.

6. The text multi-style transfer method based on style representation and multi-task learning as claimed in claim 4, characterized in that: The training method of the text multi-style transfer model is as follows: Step 5.1: The model is preheated and trained to make different task models available. The training method is described by formulas (24) and (25): （24） （25） Among them, for different tasks, the models and , using the text pairs related to their tasks, and Train them separately; Step 5.2: The model connects different task models through joint training to achieve the goal of text multi-style transfer. The training method is described by formula (26); （26） The input text By connecting in series, first After that, we get a high-dimensional vector containing the target style information. , and further input Get output text containing multiple style information .

7. A text multi-style transfer device based on style representation and multi-task learning, characterized in that: The method comprises a processor and a memory, wherein the processor executes the code in the memory to implement the method according to any one of claims 1 to 5.

8. A computer storage medium, characterized in that A computer program is stored, and the computer program is executed by hardware to implement the method according to any one of claims 1 to 5.

Images