Method and apparatus for generating text

Abstract

The embodiments of the present disclosure disclose a method and device for generating text. A specific implementation of the method includes: receiving original text and element labels; based on the original text, using a pre-trained encoder to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text; based on the hidden state vector of the decoder at the current moment and the encoded text feature vector, determining the word attention weight of each word in the original text at the current moment; based on the hidden state vector, the encoded sentence feature vector and the element label, determining the sentence attention weight of each sentence in the original text; based on the sentence attention weight and the word attention weight, determining the normalized word attention weight of each word at the current moment; based on the encoded text feature vector and the normalized word attention weight of each word, estimating the target word at the current moment; based on the target word output by the decoder at each moment, generating the target text. The accuracy and controllability of text generation are improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, specifically to the field of deep learning, and more particularly to a method and device for generating text. Background Art

[0002] Text summarization refers to converting the original text into a refined text containing only key information, which mainly includes two types: extractive summarization and generative summarization. Extractive summarization mainly extracts key words or short sentences from the source document through algorithms such as clustering or sequence labeling. Generative summarization mainly adopts deep learning methods, using the sequence-to-sequence (sequence2sequence, sea2seq) framework. The source document is first encoded into a fixed-dimensional latent vector using an encoder, and then input into the decoder for step-by-step text generation.

[0003] In related technologies, the Pointer Generator Network introduces a copy mechanism and an overwrite mechanism based on the seq2seq model. The copy mechanism calculates the attention weights of all words in the source document at each decoding step, and can selectively copy parts of the original document into the summary, thereby generating words that have not appeared in the vocabulary; the overwrite mechanism accumulates the attention weights of the current step and all previous steps at each decoding step to avoid continuously copying parts of the original document that have already obtained high weights. Summary of the invention

[0004] Embodiments of the present disclosure provide a method and apparatus for generating text.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating text, the method comprising: receiving original text and element labels; based on the original text, using a pre-trained encoder to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text; using a pre-trained decoder to perform the following decoding steps: based on the hidden state vector of the decoder at the current moment and the encoded text feature vector, determining the word attention weights of each word in the original text at the current moment; based on the hidden state vector of the decoder at the current moment, the encoded sentence feature vector and the element label, determining the sentence attention weights of each sentence in the original text at the current moment; based on the sentence attention weights and the word attention weights, determining the normalized word attention weights of each word in the original text at the current moment; based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment, estimating the target word at the current moment; the method also comprises: generating a target text based on the target words output by the decoder at each moment.

[0006] In some embodiments, based on the original text, a pre-trained encoder is used to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text, including: generating a text feature matrix based on the original text; inputting the text feature matrix into a pre-trained first encoder to obtain an encoded text feature vector; inputting the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector for each word in the original text; determining a sentence feature vector for each sentence in the original text based on the encoded word feature vectors for each word in each sentence in the original text; inputting the sentence feature vector for each sentence in the original text into a pre-trained third encoder to obtain an encoded sentence feature vector for each sentence in the original text.

[0007] In some embodiments, based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment, the target word at the current moment is estimated, including: based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment, a context feature vector at the current moment is generated; based on the context feature vector and the encoded text feature vector, the target word at the current moment is estimated.

[0008] In some embodiments, the element label is generated through the following steps: clustering the original text to obtain a cluster label set; determining a target cluster label from the cluster label set; and determining the target cluster label as the element label.

[0009] In some embodiments, clustering the original text to obtain a cluster label set includes: using preset punctuation marks as segmentation boundaries to segment the original text to obtain a short sentence set; deleting short sentences in the short sentence set that do not meet preset conditions to obtain an updated short sentence set; based on the updated short sentence set, using a pre-trained feature extraction model to extract word features of each word in each short sentence, and based on the word features, determining the sentence features of each short sentence; based on the sentence features of each short sentence in the updated short sentence set, using a pre-trained clustering model to determine the cluster label of each short sentence in the updated short sentence set to obtain a cluster label set.

[0010] In a second aspect, an embodiment of the present disclosure provides a device for generating text, the device comprising: a receiving unit, configured to receive an original text and an element label; an encoding unit, configured to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text based on the original text using a pre-trained encoder; a decoding unit, configured to use a pre-trained decoder to perform the following decoding steps: determining the word attention weights of each word in the original text at the current moment based on the hidden state vector of the decoder at the current moment and the encoded text feature vector; determining the sentence attention weights of each sentence in the original text at the current moment based on the hidden state vector of the decoder at the current moment, the encoded sentence feature vector and the element label; determining the normalized word attention weights of each word in the original text at the current moment based on the sentence attention weights and the word attention weights; estimating the target word at the current moment based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment; a generating unit, configured to generate a target text based on the target words output by the decoder at each moment.

[0011] In some embodiments, the encoding unit further includes: a feature matrix generation module, configured to generate a text feature matrix based on the original text; a first encoding module, configured to input the text feature matrix into a pre-trained first encoder to obtain an encoded text feature vector; a second encoding module, configured to input the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector of each word in the original text; a sentence feature vector generation module, configured to determine the sentence feature vector of each sentence in the original text based on the encoded word feature vector of each word in each sentence in the original text; a third encoding module, configured to input the sentence feature vector of each sentence in the original text into a pre-trained third encoder to obtain an encoded sentence feature vector of each sentence in the original text.

[0012] In some embodiments, the decoding unit further includes: a context feature vector generation module, configured to generate a context feature vector at the current moment based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment; a target word output module, configured to estimate the target word at the current moment based on the context feature vector and the encoded text feature vector.

[0013] In some embodiments, the device also includes an element label generation unit configured to: cluster the original text to obtain a cluster label set; determine a target cluster label from the cluster label set; and determine the target cluster label as the element label.

[0014] In some embodiments, the element label generation unit further includes: a segmentation module, configured to use preset punctuation marks as segmentation boundaries to segment the original text and obtain a short sentence set; a screening module, configured to delete short sentences in the short sentence set that do not meet preset conditions to obtain an updated short sentence set; a feature extraction module, configured to extract word features of each word in each short sentence based on the updated short sentence set using a pre-trained feature extraction model, and determine the sentence features of each short sentence based on the word features; a label generation module, configured to determine the clustering label of each short sentence in the updated short sentence set based on the sentence features of each short sentence in the updated short sentence set using a pre-trained clustering model to obtain a clustering label set.

[0015] The method and device for generating text provided by the embodiments of the present disclosure compress the original text through an encoder, determine the word attention weight of each word through a decoder, and determine the sentence attention weight of each sentence based on the correlation between each sentence and element label in the original text, and then determine the normalized word attention weight based on the word attention weight and the sentence attention weight, and estimate the target word based on the normalized word attention weight, and finally generate the target text based on the target word, so that the sentences and words in the target text have a high correlation with the element labels, thereby improving the accuracy and controllability of text generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0017] Figure 1 is an exemplary system architecture diagram in which some embodiments of the present disclosure may be applied;

[0018] Figure 2 is a flow chart of an embodiment of a method for generating text according to the present disclosure;

[0019] Figure 3 yes Figure 2 A schematic diagram of an application scenario of an embodiment of a method for generating text is shown;

[0020] Figure 4 is a flowchart of another embodiment of a method for generating text according to the present disclosure;

[0021] Figure 5 is a structural schematic diagram of an embodiment of an apparatus for generating text according to the present disclosure;

[0022] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] Figure 1 An exemplary system architecture 100 is shown to which a method for generating text or an apparatus for generating text according to an embodiment of the present disclosure can be applied.

[0026] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0027] The user can use the terminal devices 101 , 102 , 103 to interact with the server 105 via the network 104 to receive or send messages, etc. For example, the user can send an original text to the server and receive a target text from the server.

[0028] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be electronic devices with communication functions, including but not limited to smart phones, tablet computers, e-book readers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules for providing distributed services, or they can be implemented as a single software or software module. No specific limitation is made here.

[0029] The server 105 may be a server that provides various services, such as a background data server that processes the original text uploaded by the terminal devices 101, 102, and 103 (e.g., generates a target text based on the original text). The background data server may perform encoding, decoding, and other processing on the received original text, and feed back the processing result (e.g., the target text) to the terminal device.

[0030] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules for providing distributed services, or it can be implemented as a single software or software module. No specific limitation is made here.

[0031] It should be noted that the method for generating text provided in the embodiment of the present disclosure can be executed by the terminal device 101, 102, 103, or by the server 105. Accordingly, the device for generating text can be set in the terminal device 101, 102, 103, or in the server 105. No specific limitation is made here.

[0032] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for generating text according to the present disclosure. The method for generating text comprises the following steps:

[0033] Step 201, receiving original text and element labels.

[0034] In this embodiment, the element tag may represent the user's expectation of the target text content and may be preset by the user according to actual needs.

[0035] As an example, the original text may be the detailed information of a product, and the target text may be the information summary of the product. If the user expects the target text to contain content related to "appearance", the element tag may be set to "appearance", and then the detailed information of the product and the element tag may be sent to the execution subject of this embodiment (for example, it may be Figure 1 For another example, if the user expects the target text to include content related to "sales volume", the element tag can be set to "sales volume".

[0036] In some optional implementations of this embodiment, the element label is generated through the following steps: clustering the original text to obtain a cluster label set; determining a target cluster label from the cluster label set; and determining the target cluster label as the element label.

[0037] In this implementation, the clustering labels can represent the content features of the original text. The execution subject can use a clustering algorithm to extract the clustering labels from the original text, and then select element labels therefrom, so as to ensure the relevance of the element labels to the original text content.

[0038] As an example, the execution subject may input the original text into a pre-trained text clustering model, such as TF-IDF, word2vec, etc., to obtain a clustering label set of the original text. Afterwards, the execution subject may present the clustering label set to the user, and the user may select a target clustering label therefrom, thereby determining the clustering label selected by the user as the feature label.

[0039] Step 202: Based on the original text, a pre-trained encoder is used to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text.

[0040] In this embodiment, the encoder is used to convert the original text into an encoded text feature vector of a preset dimension, and to convert each sentence in the original text into an encoded sentence feature vector. The encoded text feature vector includes the word vectors of all words in the original text, which are used to characterize the content characteristics of the original text. The encoded sentence feature vector includes the word vectors of all words in the sentence, which are used to characterize the content characteristics of the sentence. The word vector can characterize the position of a word in a preset vocabulary.

[0041] The encoder can use various types of deep learning models, such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network) or GRU (Gated Recurrent Unit).

[0042] As an example, the execution subject may use an LSTM (Long Short-Term Memory) network as an encoder. The execution subject may input the original text into the encoder to obtain an encoded text feature vector of a preset dimension, and then input each sentence in the original text into the encoder to obtain an encoded sentence feature vector of each sentence.

[0043] In some optional implementations of the present embodiment, the following steps may be used to obtain an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text: based on the original text, generate a text feature matrix; input the text feature matrix into a pre-trained first encoder to obtain an encoded text feature vector; input the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector for each word in the original text; based on the encoded word feature vectors for each word in each sentence in the original text, determine the sentence feature vector for each sentence in the original text; input the sentence feature vector for each sentence in the original text into a pre-trained third encoder to obtain an encoded sentence feature vector for each sentence in the original text.

[0044] In this implementation, the execution subject can use an embedding matrix to convert the original text into a text feature matrix, where the embedding matrix represents the correspondence between words and word feature values, and the word feature value represents the position of the word in the vocabulary. Afterwards, the execution subject obtains an encoded text feature vector through the first encoder, and obtains an encoded sentence feature vector of each sentence in the original text through the second encoder and the third encoder.

[0045] As an example, the first encoder can be an LSTM encoder, and the second encoder and the third encoder can be GRU encoders. The execution subject inputs the text feature matrix into the first encoder respectively, and encodes the text feature matrix into an encoded text feature vector of a first preset dimension. The text feature matrix is ​​input into the second encoder, and each word vector in the text feature matrix is ​​encoded into an encoded word vector of a second preset dimension. After that, the execution subject uses the mean of the encoded word vectors of each word in each sentence as the sentence feature vector of the sentence, and inputs the sentence feature vector into the third encoder to obtain the encoded sentence feature vector of each sentence in the original text. For another example, the execution subject can also use the maximum value of the encoded word vectors of each word in each sentence as the sentence feature vector of the sentence, and can also use the encoded word vector of the word at the end of each sentence as the sentence feature vector of the sentence.

[0046] Step 203, using the pre-trained decoder, performs decoding steps: step 2031 to step 2034.

[0047] In this embodiment, the decoder is used to estimate the target word from the encoded text feature vector output by the encoder. The decoder can use the same deep learning model as the encoder.

[0048] Step 2031, based on the hidden state vector of the decoder at the current moment and the encoded text feature vector, determine the word attention weight of each word in the original text at the current moment.

[0049] Usually, the decoder outputs the hidden state vector at the current moment for each decoding step. The hidden state vector is used to represent the state parameters of each hidden layer of the decoder in the previous decoding step.

[0050] In this embodiment, the word attention weight determined by the hidden state vector at the current moment and the encoded text feature vector can represent the probability that the decoder selects the word at the current moment.

[0051] As an example, the execution subject inputs the encoded text vector B obtained in step 202 into the pre-trained LSTM decoder, and the LSTM decoder gradually decodes the encoded text feature vector. Assume that the LSTM completes the nth step of decoding, and the hidden state vector at the current moment is the hidden state vector A corresponding to the nth step of decoding n , the execution subject is based on the hidden state vector An The word attention weights of each word at the current moment are calculated with the encoded text feature vector B Among them, C represents the word attention weight, and i represents the i-th word in the original text. After that, LSTM completes the n+1th step decoding, and the hidden state vector at the current moment is the hidden state vector A corresponding to the n+1th step decoding n+1 , the execution subject is based on A n+1 The word attention weights of each word at the current moment are calculated with the encoded text feature vector B

[0052] Step 2032, based on the hidden state vector of the decoder at the current moment, the encoded sentence feature vector and the element label, determine the sentence attention weight of each sentence in the original text at the current moment.

[0053] In this embodiment, the sentence attention weight represents the correlation between the content features of the sentence (such as grammatical structure, semantics, etc.) and the element label. The higher the correlation, the higher the value of the sentence attention weight.

[0054] In a specific example, the execution subject can determine the value of the element label based on a preset list of correspondences between element labels and values, and convert it into a vector form to obtain an element label vector. Afterwards, the execution subject can combine the hidden state vector and the element label vector into a combined vector. Then, the execution subject calculates the correlation between the encoded sentence feature vector of each sentence and the combined vector, respectively, to obtain the sentence attention weight of each sentence.

[0055] Step 2033, based on the sentence attention weight and the word attention weight, determine the normalized word attention weight of each word in the original text at the current moment.

[0056] In this embodiment, the sentence attention weight represents the correlation between the content features of the sentence and the element label. In this way, the normalized word attention weight can also represent the correlation between the word and the element label.

[0057] As an example, the execution entity may use the product of the sentence attention weight and the word attention weight as the normalized word attention weight, so that the normalized word attention weight is positively correlated with the degree of correlation between the word and the element label.

[0058] Step 2034, based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment, estimate the target word at the current moment.

[0059] In this embodiment, the decoder loaded on the execution body determines the confidence of each word in the original text based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment, and then determines the word with the highest confidence as the target word at the current moment. After that, the decoder performs the next decoding step and outputs the target word at the next moment until the decoder completely decodes the encoded text feature vector and obtains the target word sequence.

[0060] In some optional implementations of this embodiment, a context feature vector at the current moment is generated based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment; and the target word at the current moment is estimated based on the context feature vector and the encoded text feature vector.

[0061] In this implementation, the context feature vector can characterize the influence of the context in the original text on the confidence of the word. In this way, the influence of the context on the word can be introduced in the process of the encoder determining the target word at the current moment, which can further improve the degree of fit between the target word and the content of the original text.

[0062] Step 204, generating a target text based on the target words output by the decoder at each time.

[0063] In this embodiment, the execution subject can concatenate the target words output by the decoder in time sequence to obtain the target text. Since each target word comes from the original text and is related to the element label, the target text can represent the text extracted from the original text by the execution subject according to the element label.

[0064] Continue to see Figure 3 , Figure 3 FIG2 is a schematic diagram of an application scenario of an embodiment of the method shown in FIG2. Figure 3In the scene 300 shown in , the original text is product information 303, which may include, for example, the product name and detailed information of the product, and the target text is a product information summary 305. The terminal device 301 may send the product information 303 and the element label 304 to the server 302 through the network, and the server 302 is loaded with a pre-trained text generation model, which may be an LSTM encoder and an LSTM decoder including an attention module. The server 302 inputs the received product information and element label into the text generation model, and performs the following text generation steps: the LSTM encoder encodes the product information into a coded text feature vector of a third preset dimension, and encodes each sentence in the product information into a coded sentence feature vector of a fourth preset dimension. Afterwards, the LSTM encoder gradually encodes the text feature vector and outputs the hidden state vector at the current moment. Then, the attention module determines the word attention weight of each word in the product information at the current moment based on the encoded text feature vector and the hidden state vector, and determines the sentence attention weight of each sentence in the product information based on the encoded sentence feature vector, the element label and the hidden state vector, and takes the product of the word attention weight of the word and the sentence attention weight of all sentences including the word as the normalized word attention weight to obtain the normalized word attention weight of each word in the original text at the current moment. Afterwards, the encoder estimates the target word at the current moment based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment, and concatenates the target words at each moment into the target text in time sequence, so as to obtain the summary of the product information, and the content in the summary is highly correlated with the element label.

[0065] The method and device for generating text provided by the embodiments of the present disclosure compress the original text through an encoder, determine the word attention weight of each word through a decoder, and determine the sentence attention weight of each sentence based on the correlation between each sentence and element label in the original text, and then determine the normalized word attention weight based on the word attention weight and the sentence attention weight, and estimate the target word based on the normalized word attention weight, and finally generate the target text based on the target word, so that the sentences and words in the target text have a high correlation with the element labels, thereby improving the accuracy and controllability of text generation.

[0066] Further references Figure 4 , which shows a process 400 of another embodiment of a method for generating text. The process 400 of the method for generating text includes the following steps:

[0067] Step 401, receiving original text.

[0068] Step 402, using preset punctuation marks as segmentation boundaries, segmenting the original text to obtain a set of short sentences.

[0069] In this embodiment, the execution subject can use a period, an exclamation mark or a question mark as a segmentation boundary to segment the original text into a number of short sentences to obtain a short sentence set, so that the semantic integrity of each short sentence can be guaranteed.

[0070] Step 403, the short sentences in the short sentence set that do not meet the preset conditions are deleted to obtain an updated short sentence set.

[0071] As an example, the following preset condition may be used: the short sentence includes at least one comma and the length of the short sentence is within a preset length range. The execution subject screens the short sentences obtained in step 402 based on the preset condition and deletes the short sentences that do not meet the condition, thereby reducing the interference of noise data.

[0072] Step 404 , based on the updated short sentence set, a pre-trained feature extraction model is used to extract word features of each word in each short sentence, and based on the word features, sentence features of each short sentence are determined.

[0073] In this embodiment, the sentence features of a short sentence represent the content features of the short sentence, and may include, for example, the grammatical structure features and semantic features of the short sentence.

[0074] The feature extraction model may adopt existing or future models such as TF-IDF or word2vec, and this application does not limit this.

[0075] As an example, the execution subject can use the pre-trained BERT (Bidirectional Encoder Representation from Transformers) model to extract word features from each short sentence, and take the weighted average of each word feature as the sentence feature of the short sentence.

[0076] Step 405 , based on the sentence features of each short sentence in the updated short sentence set, a pre-trained clustering model is used to determine a clustering label for each short sentence in the updated short sentence set to obtain a clustering label set.

[0077] As an example, the execution entity may use a K-means (K-means clustering algorithm) model to cluster the sentence features of short sentences, obtain a cluster label for each short sentence, and then obtain a cluster label set corresponding to the entire updated short sentence set.

[0078] The clustering model may also be an existing or future clustering model such as a variational autoencoder or DBSCAN (Density-Based Spatial Clustering of Applications with Noise, a density-based clustering algorithm).

[0079] Step 406: determine a target cluster label from the cluster label set.

[0080] Step 407: determine the target cluster label as the feature label.

[0081] In this embodiment, the cluster label can represent the content characteristics of the original text, and the two-element label can represent the user's content expectations for the target text. The element label determined from the cluster label set can take into account both the content characteristics of the original text and the user's content expectations.

[0082] Step 408: Based on the original text, a pre-trained encoder is used to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text. This step corresponds to the aforementioned step 202 and will not be described in detail here.

[0083] Step 409, using the pre-trained decoder, performs the following decoding steps. This step corresponds to the aforementioned step 203, and will not be repeated here.

[0084] Step 410: Generate target text based on the target words output by the decoder at each time. This step corresponds to the aforementioned step 204 and will not be described in detail here.

[0085] from Figure 4 It can be seen that process 400 of the method for generating text in this embodiment embodies the steps of extracting clustering labels of short sentences in the original text based on the clustering algorithm, and determining element labels from the clustering labels, which can improve the correlation between the element labels and the original text, thereby making the sentences and words in the generated target text more correlated with the element labels. In this way, the accuracy and controllability of text generation can be further improved.

[0086] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device 500 for generating text. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0087] like Figure 5As shown, the apparatus 500 for generating text of the present embodiment includes: a receiving unit 501, configured to receive an original text and an element label; an encoding unit 502, configured to generate an encoded text feature vector and an encoded sentence feature vector of each sentence in the original text based on the original text by using a pre-trained encoder; a decoding unit 503, configured to use a pre-trained decoder to perform the following decoding steps: determining the word attention weight of each word in the original text at the current moment based on the hidden state vector and the encoded text feature vector of the decoder at the current moment; determining the sentence attention weight of each sentence in the original text at the current moment based on the hidden state vector, the encoded sentence feature vector and the element label of the decoder at the current moment; determining the normalized word attention weight of each word in the original text at the current moment based on the sentence attention weight and the word attention weight; estimating the target word at the current moment based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment; a generating unit 504, configured to generate the target text based on the target word output by the decoder at each moment.

[0088] In this embodiment, the encoding unit 502 further includes: a feature matrix generation module, which is configured to generate a text feature matrix based on the original text; a first encoding module, which is configured to input the text feature matrix into a pre-trained first encoder to obtain an encoded text feature vector; a second encoding module, which is configured to input the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector of each word in the original text; a sentence feature vector generation module, which is configured to determine the sentence feature vector of each sentence in the original text based on the encoded word feature vector of each word in each sentence in the original text; and a third encoding module, which is configured to input the sentence feature vector of each sentence in the original text into a pre-trained third encoder to obtain an encoded sentence feature vector of each sentence in the original text.

[0089] In this embodiment, the decoding unit 503 further includes: a context feature vector generation module, which is configured to generate a context feature vector at the current moment based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment; and a target word output module, which is configured to estimate the target word at the current moment based on the context feature vector and the encoded text feature vector.

[0090] In this embodiment, the device 500 further includes an element label generation unit configured to: cluster the original text to obtain a cluster label set; determine a target cluster label from the cluster label set; and determine the target cluster label as the element label.

[0091] In this embodiment, the element label generation unit further includes: a segmentation module, which is configured to use preset punctuation marks as segmentation boundaries to segment the original text and obtain a short sentence set; a screening module, which is configured to delete short sentences in the short sentence set that do not meet preset conditions to obtain an updated short sentence set; a feature extraction module, which is configured to extract word features of each word in each short sentence based on the updated short sentence set using a pre-trained feature extraction model, and determine the sentence features of each short sentence based on the word features; a label generation module, which is configured to determine the clustering label of each short sentence in the updated short sentence set based on the sentence features of each short sentence in the updated short sentence set using a pre-trained clustering model to obtain a clustering label set.

[0092] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0093] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0094] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0095] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus, or a device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0096] The computer-readable medium may be included in the electronic device; or it may exist separately without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives the original text and the element label; based on the original text, uses a pre-trained encoder to generate an encoded text feature vector and an encoded sentence feature vector of each sentence in the original text; uses a pre-trained decoder to perform the following decoding steps: based on the hidden state vector and the encoded text feature vector of the decoder at the current moment, determine the word attention weight of each word in the original text at the current moment; based on the hidden state vector, the encoded sentence feature vector and the element label of the decoder at the current moment, determine the sentence attention weight of each sentence in the original text at the current moment; based on the sentence attention weight and the word attention weight, determine the normalized word attention weight of each word in the original text at the current moment; based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment, estimate the target word at the current moment; the method also includes: based on the target word output by the decoder at each moment, generate the target text.

[0097] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0099] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The units described may also be arranged in a processor, for example, it may be described as: a processor includes a receiving unit, an encoding unit, a decoding unit, and a generating unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the receiving unit may also be described as a "unit for receiving original text and element labels".

[0100] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A method for generating text, wherein: include: Receiving an original text and an element tag, wherein the element tag represents a user's expectation of the target text content; Based on the original text, using a pre-trained encoder to generate an encoded text feature vector and an encoded sentence feature vector for each sentence in the original text; Using the pre-trained decoder, perform the following decoding steps: Based on the hidden state vector of the decoder at the current moment and the encoded text feature vector, determine the word attention weight of each word in the original text at the current moment; based on the hidden state vector of the decoder at the current moment, the encoded sentence feature vector and the element label, determine the sentence attention weight of each sentence in the original text at the current moment; based on the sentence attention weight and the word attention weight, determine the normalized word attention weight of each word in the original text at the current moment; based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment, estimate the target word at the current moment, wherein the sentence attention weight represents the degree of correlation between the content feature of the sentence and the element label; The method further includes: generating a target text based on the target words output by the decoder at each time.

2. The method according to claim 1, wherein: Based on the original text, a pre-trained encoder is used to generate an encoded text feature vector and an encoded sentence feature vector of each sentence in the original text, including: Based on the original text, generate a text feature matrix; Inputting the text feature matrix into a pre-trained first encoder to obtain the encoded text feature vector; Inputting the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector of each word in the original text; Determine a sentence feature vector of each sentence in the original text based on the encoded word feature vector of each word in each sentence in the original text; The sentence feature vector of each sentence in the original text is input into a pre-trained third encoder to obtain an encoded sentence feature vector of each sentence in the original text.

3. The method according to claim 1, wherein: Based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment, estimating the target word at the current moment includes: Generate a context feature vector at the current moment based on the encoded text feature vector and the normalized word attention weights of each word in the original text at the current moment; Based on the context feature vector and the encoded text feature vector, the target word at the current moment is estimated.

4. The method according to any one of claims 1 to 3, wherein: The element label is generated through the following steps: Clustering the original text to obtain a clustering label set; Determining a target cluster label from the cluster label set; The target cluster label is determined as the element label.

5. The method according to claim 4, wherein: The original text is clustered to obtain a cluster label set, including: Using preset punctuation marks as segmentation boundaries, segmenting the original text to obtain a set of short sentences; Deleting short sentences that do not meet the preset conditions in the short sentence set to obtain an updated short sentence set; Based on the updated short sentence set, using a pre-trained feature extraction model, extracting word features of each word in each short sentence, and determining sentence features of each short sentence based on the word features; Based on the sentence features of each short sentence in the updated short sentence set, a pre-trained clustering model is used to determine the cluster label of each short sentence in the updated short sentence set to obtain the cluster label set.

6. A device for generating text, wherein: include: A receiving unit, configured to receive an original text and an element tag, wherein the element tag represents a user's expectation of the target text content; An encoding unit, configured to generate an encoded text feature vector and an encoded sentence feature vector of each sentence in the original text using a pre-trained encoder based on the original text; The decoding unit is configured to use a pre-trained decoder to perform the following decoding steps: based on the hidden state vector of the decoder at the current moment and the encoded text feature vector, determine the word attention weight of each word in the original text at the current moment; based on the hidden state vector of the decoder at the current moment, the encoded sentence feature vector and the element label, determine the sentence attention weight of each sentence in the original text at the current moment; based on the sentence attention weight and the word attention weight, determine the normalized word attention weight of each word in the original text at the current moment; based on the encoded text feature vector and the normalized word attention weight of each word in the original text at the current moment, estimate the target word at the current moment, wherein the sentence attention weight represents the degree of correlation between the content feature of the sentence and the element label; The generating unit is configured to generate a target text based on the target words output by the decoder at each time.

7. The device according to claim 6, wherein: The encoding unit further comprises: A feature matrix generating module is configured to generate a text feature matrix based on the original text; A first encoding module is configured to input the text feature matrix into a pre-trained first encoder to obtain the encoded text feature vector; A second encoding module is configured to input the text feature matrix into a pre-trained second encoder to obtain an encoded word feature vector of each word in the original text; a sentence feature vector generating module, configured to determine a sentence feature vector of each sentence in the original text based on the encoded word feature vector of each word in each sentence in the original text; The third encoding module is configured to input the sentence feature vector of each sentence in the original text into a pre-trained third encoder to obtain the encoded sentence feature vector of each sentence in the original text.

8. The device according to claim 6, wherein: The decoding unit further comprises: A context feature vector generation module, configured to generate a context feature vector at a current moment based on the encoded text feature vector and the normalized word attention weights of each word in the original text at a current moment; The target word output module is configured to estimate the target word at the current moment based on the context feature vector and the encoded text feature vector.

9. The device according to any one of claims 6 to 8, wherein: The device also includes an element tag generating unit configured to: Clustering the original text to obtain a clustering label set; Determining a target cluster label from the cluster label set; The target cluster label is determined as the element label.

10. The device according to claim 9, wherein: The element label generation unit further comprises: A segmentation module is configured to segment the original text using preset punctuation marks as segmentation boundaries to obtain a set of short sentences; A screening module is configured to delete short sentences in the short sentence set that do not meet the preset conditions to obtain an updated short sentence set; A feature extraction module is configured to extract word features of each word in each short sentence based on the updated short sentence set and use a pre-trained feature extraction model, and determine sentence features of each short sentence based on the word features; The label generation module is configured to determine the cluster label of each short sentence in the updated short sentence set based on the sentence features of each short sentence in the updated short sentence set and adopt a pre-trained clustering model to obtain the cluster label set.

11. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

12. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images