Long text generation method, device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence, and discloses a long text generation method, device, equipment and storage medium. The method comprises: obtaining multiple keywords to be processed and inputting each keyword into a preset vector encoder for vector conversion to obtain a word vector; concatenating the word vector and the preset variable and inputting the obtained initial vector into a preset short sentence generator for short sentence generation to obtain multiple first short sentences; performing average pooling operation on each first short sentence respectively and inputting the obtained multiple feature short sentence vectors into a preset short sentence encoder for code conversion to obtain a short sentence code; inputting the short sentence code, the initial vector and the preset random variable into a preset text associator for feature extraction to obtain a context association vector; inputting each first short sentence, the random variable and the context association vector into a preset long text generator for text generation to obtain a long text. The present invention can associate the context of the generated long text, thereby improving the coherence of the long text.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a long text generation method, device, equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence, artificial intelligence technology has been widely used in various industries. At the same time, it has also driven the development of the field of text generation. More and more texts are generated directly using data. The traditional text generation method can be divided into three steps: 1. Content planning; 2. Sentence planning; 3. Sentence realization. These three steps realize what content should be generated and how to generate content.

[0003] However, in traditional text generation methods, the generated content and how to generate the corresponding content are obtained separately according to different steps, resulting in semantic incoherence between the context and the generated long text, and a lack of relevance between sentences. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem that the existing text generation effect is poor.

[0005] A first aspect of the present invention provides a long text generation method, the long text generation method comprising:

[0006] Acquire multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector;

[0007] splicing the word vector and the preset variable to obtain an initial vector, and inputting the initial vector into a preset short sentence generator to generate a short sentence, thereby obtaining a plurality of first short sentences;

[0008] Performing an average pooling operation on each of the first short sentences respectively to obtain a plurality of characteristic short sentence vectors, and inputting each of the characteristic short sentence vectors into a preset short sentence encoder for encoding conversion to obtain a short sentence code;

[0009] splicing the short sentence code and the initial vector to obtain a first fused vector, and inputting the first fused vector and a preset random variable into a preset text associator for feature extraction to obtain a context association vector;

[0010] The first short sentences, the random variables and the context association vector are input into a preset long text generator to generate text, thereby obtaining a long text.

[0011] Optionally, in a first implementation of the first aspect of the present invention, the vector encoder includes multiple bidirectional GRU neural networks, and the acquiring multiple keywords to be processed and inputting each keyword into a preset vector encoder for vector conversion to obtain a word vector includes:

[0012] Get multiple keywords to be processed;

[0013] Inputting the keyword into the bidirectional GRU neural network for vector conversion to obtain a forward hidden state vector and a backward hidden state vector;

[0014] The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

[0015] Optionally, in a second implementation of the first aspect of the present invention, the short sentence generator includes a plurality of first unidirectional GRU neural networks, and the inputting the initial vector into the preset short sentence generator to generate short sentences to obtain the plurality of first short sentences includes:

[0016] Inputting the initial vector into the first unidirectional GRU neural network to generate short sentences, thereby obtaining a plurality of first short sentences;

[0017] Calling a preset rest function to calculate the fully connected layer in the first unidirectional GRU neural network to obtain the output value of the hidden layer neurons;

[0018] Determine whether the output value of the hidden layer neuron exceeds a preset threshold;

[0019] If yes, stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences;

[0020] If not, continue to generate short sentences through the first unidirectional GRU neural network until the output value of the hidden layer neuron exceeds the preset threshold, then stop generating short sentences, and use the short sentences obtained before stopping generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences.

[0021] Optionally, in a third implementation manner of the first aspect of the present invention, performing an average pooling operation on each of the first short sentences to obtain a plurality of feature short sentence vectors includes:

[0022] Calling a preset average pooling function to calculate the average value of each of the first short sentences respectively, to obtain a plurality of feature values ​​corresponding to each short sentence;

[0023] A plurality of characteristic short sentence vectors are generated according to the plurality of characteristic values ​​corresponding to the short sentences.

[0024] Optionally, in a fourth implementation of the first aspect of the present invention, the short sentence encoder includes a plurality of second unidirectional GRU neural networks, and the inputting each feature short sentence vector into a preset short sentence encoder for encoding conversion to obtain the short sentence encoding includes:

[0025] Inputting each of the feature short sentence vectors into the second unidirectional GRU neural network;

[0026] The second unidirectional GRU neural network is used to perform encoding conversion on each of the characteristic short sentence vectors to obtain a short sentence code.

[0027] Optionally, in a fifth implementation of the first aspect of the present invention, the sentence associator includes a plurality of third unidirectional GRU neural networks, the concatenation of the short sentence code and the initial vector to obtain a first fused vector, and the first fused vector and a preset random variable are input into a preset text associator for feature extraction, and the context association vector obtained includes:

[0028] Concatenating the short sentence code and the initial vector to obtain a first fusion vector;

[0029] The first fusion vector and the preset random variable are input into the third unidirectional GRU neural network to extract context-related features to obtain a context-related vector.

[0030] Optionally, in a sixth implementation of the first aspect of the present invention, the long text generator includes a plurality of fourth unidirectional GRU neural networks, and the inputting of the first short sentences, the random variables, and the context association vector into the preset long text generator for text generation to obtain the long text includes:

[0031] Concatenating the first short sentences, the random variable and the context association vector to obtain a second fusion vector;

[0032] Inputting the second fused vector into the fourth unidirectional GRU neural network for decoding to obtain a plurality of second short sentences;

[0033] The second short sentences are sequentially concatenated to obtain a long text.

[0034] A second aspect of the present invention provides a long text generation device, the long text generation device comprising:

[0035] An acquisition module is used to acquire multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector;

[0036] A processing module, used for concatenating the word vector and the preset variable to obtain an initial vector, and inputting the initial vector into a preset short sentence generator to generate a short sentence, thereby obtaining a plurality of first short sentences;

[0037] A conversion module, used to perform an average pooling operation on each of the first short sentences to obtain a plurality of characteristic short sentence vectors, and input each of the characteristic short sentence vectors into a preset short sentence encoder for encoding conversion to obtain a short sentence code;

[0038] An extraction module, used for concatenating the short sentence code and the initial vector to obtain a first fusion vector, and inputting the first fusion vector and a preset random variable into a preset text associator for feature extraction to obtain a context association vector;

[0039] The generating module is used for inputting the first short sentences, the random variables and the context association vector into a preset long text generator to generate text and obtain a long text.

[0040] Optionally, in a first implementation manner of the second aspect of the present invention, the vector encoder includes a plurality of bidirectional GRU neural networks, and the acquisition module is specifically used for:

[0041] Get multiple keywords to be processed;

[0042] Inputting the keyword into the bidirectional GRU neural network for vector conversion to obtain a forward hidden state vector and a backward hidden state vector;

[0043] The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

[0044] Optionally, in a second implementation of the second aspect of the present invention, the short sentence generator includes a plurality of first unidirectional GRU neural networks, and the processing module is specifically used for:

[0045] Inputting the initial vector into the first unidirectional GRU neural network to generate short sentences, thereby obtaining a plurality of first short sentences;

[0046] Calling a preset rest function to calculate the fully connected layer in the first unidirectional GRU neural network to obtain the output value of the hidden layer neurons;

[0047] Determine whether the output value of the hidden layer neuron exceeds a preset threshold;

[0048] If yes, stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences;

[0049] If not, continue to generate short sentences through the first unidirectional GRU neural network until the output value of the hidden layer neuron exceeds the preset threshold, then stop generating short sentences, and use the short sentences obtained before stopping generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences.

[0050] Optionally, in a third implementation manner of the second aspect of the present invention, the conversion module further includes a computing unit, and the computing unit is specifically configured to:

[0051] Calling a preset average pooling function, respectively calculating an average value for each of the first short sentences, to obtain a plurality of eigenvalues ​​corresponding to each short sentence; and generating a plurality of feature short sentence vectors according to the plurality of eigenvalues ​​corresponding to each of the short sentences.

[0052] Optionally, in a fourth implementation manner of the second aspect of the present invention, the short sentence encoder includes a plurality of second unidirectional GRU neural networks, and the conversion module further includes an encoding unit, and the encoding unit is specifically used to:

[0053] Input each of the characteristic short sentence vectors into the second unidirectional GRU neural network; and perform encoding conversion on each of the characteristic short sentence vectors through the second unidirectional GRU neural network to obtain a short sentence code.

[0054] Optionally, in a fifth implementation of the second aspect of the present invention, the sentence associator includes a plurality of third unidirectional GRU neural networks, and the extraction module is specifically used for:

[0055] The short sentence code and the initial vector are concatenated to obtain a first fusion vector; the first fusion vector and a preset random variable are input into the third unidirectional GRU neural network to extract context-related features to obtain a context-related vector.

[0056] Optionally, in a sixth implementation of the second aspect of the present invention, the long text generator includes a plurality of fourth unidirectional GRU neural networks, and the generation module is specifically used for:

[0057] Concatenating the first short sentences, the random variable and the context association vector to obtain a second fusion vector;

[0058] Inputting the second fused vector into the fourth unidirectional GRU neural network for decoding to obtain a plurality of second short sentences;

[0059] The second short sentences are sequentially concatenated to obtain a long text.

[0060] A third aspect of the present invention provides a long text generation device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the long text generation device executes the above-mentioned long text generation method.

[0061] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the long text generation method described above.

[0062] In the technical solution provided by the present invention, in order to generate a semantically coherent long text, the long text can be regarded as the splicing of multiple short sentences, and then the short sentences can be regarded as the splicing of multiple words, specifically: multiple keywords are obtained and the keywords are vectorized to obtain word vectors, and random variables are added when the keywords are selected, so that the short sentences generated are greatly different each time, thereby greatly increasing the diversity of sentence patterns. Then the corresponding first short sentence is generated through the word vector, and then the first short sentence is encoded to obtain the short sentence code, and the short sentence code, the first short sentence and the random variable are input into the long text generator to generate multiple second short sentences, and the multiple second short sentences are sequentially spliced ​​to obtain the long text. The present invention uses multiple keywords as the basis for generating long text, improves the coherence of the long text and has better richness of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a schematic diagram of a first embodiment of a method for generating a long text in an embodiment of the present invention;

[0064] Figure 2 Schematic diagram of a second embodiment of the long text generation method in an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of an embodiment of a long text generation device in an embodiment of the present invention;

[0066] Figure 4 The figure is a schematic diagram of an embodiment of a long text generation device in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] Embodiments of the present invention provide a long text generation method, apparatus, device and storage medium. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0068] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , the first embodiment of the long text generation method in the embodiment of the present invention includes:

[0069] 101. Acquire multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector;

[0070] It is understandable that the execution subject of the present invention may be a long text generation device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0071] In this embodiment, the vector encoder includes multiple bidirectional GRU neural networks, which are divided into two cells, one is a forward cell and the other is a reverse cell. The forward cell is actually a normal GRU, which feeds the data to the forward cell one by one, and the reverse cell feeds the data to the reverse cell one by one. Finally, the corresponding hidden layers are put together to form a bidirectional GRU. The bidirectional GRU is a doubled variable in the hidden layer, half for forward calculation and half for reverse calculation.

[0072] In this embodiment, the keywords are vectorized through a bidirectional GRU neural network, and the obtained word vector represents the keywords as vectors, and each word is represented as a long vector using the GRU neural network.

[0073] For example: apple[0, 0, 0, 1, 0, 0, 0, 0, 0, …].

[0074] Optionally, in one embodiment, the vector encoder includes a plurality of bidirectional GRU neural networks, and the acquiring of a plurality of keywords to be processed and inputting each keyword into a preset vector encoder for vector conversion to obtain a word vector includes:

[0075] Get multiple keywords to be processed;

[0076] Inputting the keyword into the bidirectional GRU neural network for vector conversion to obtain a forward hidden state vector and a backward hidden state vector;

[0077] The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

[0078] In this embodiment, the forward output of the bidirectional GRU neural network is a forward hidden state vector, and the backward output is a backward hidden state vector. The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

[0079] For example: the forward hidden state vector is [0, 1, 2], the backward hidden state vector is [1, 0, 0], and the word vector obtained after splicing is [0, 1, 2, 1, 0, 0].

[0080] 102. Concatenate the word vector and the preset variable to obtain an initial vector, and input the initial vector into a preset short sentence generator to generate a short sentence, thereby obtaining a plurality of first short sentences;

[0081] In this embodiment, the short sentence generator includes multiple unidirectional GRU neural networks, each of which has only one cell. By receiving the data processed by the previous GRU step at each step, the hidden layer in the cell is updated, and each GRU outputs a corresponding short sentence. The short sentence generator determines which keywords need to be said in each sentence in the long text. Each GRU output represents a sentence, and the output contains information about which keywords are used in the sentence.

[0082] For example, the short sentence generated by the first GRU is: I love Beijing. The short sentence generated by the second GRU is: I love my motherland. The keywords are: Beijing, motherland.

[0083] 103. Perform an average pooling operation on each of the first short sentences to obtain a plurality of characteristic short sentence vectors, and input each of the characteristic short sentence vectors into a preset short sentence encoder for encoding conversion to obtain a short sentence code;

[0084] In this embodiment, an average value is calculated for all the words in the short sentence output by each GRU, and after average pooling, a feature vector with a dimension equal to the number of categories is obtained, which is then directly input into the softmax layer for softmax operation to obtain multiple feature short sentence vectors. The average pooling operation can better match keywords with short sentences, reduce the number of parameters, and the average pooling layer has no parameters, which can prevent overfitting in this layer and integrate global spatial information. In this embodiment, the short sentence encoder includes multiple GRU neural networks, which encode multiple feature short sentence vectors input, and the short sentence encoder finally outputs an overall short sentence encoding.

[0085] Optionally, in one embodiment, performing an average pooling operation on each of the first short sentences to obtain a plurality of feature short sentence vectors includes:

[0086] Calling a preset average pooling function to calculate the average value of each of the first short sentences respectively, to obtain a plurality of feature values ​​corresponding to each short sentence;

[0087] A plurality of characteristic short sentence vectors are generated according to the plurality of characteristic values ​​corresponding to the short sentences.

[0088] In this embodiment, average pooling is to average the keywords in each short sentence output by the GRU, and then obtain a new 1*1 feature vector.

[0089] For example, the first short sentence is [I love Beijing], and after global average pooling, the feature vector obtained is [2, 3, 1, 1].

[0090] Optionally, in one embodiment, the short sentence encoder includes a plurality of second unidirectional GRU neural networks, and the inputting each characteristic short sentence vector into a preset short sentence encoder for encoding conversion to obtain the short sentence encoding includes:

[0091] Inputting each of the feature short sentence vectors into the second unidirectional GRU neural network;

[0092] The second unidirectional GRU neural network is used to perform encoding conversion on each of the characteristic short sentence vectors to obtain a short sentence code.

[0093] In this embodiment, the second unidirectional GRU neural network encodes the feature vector output by average pooling to obtain a short sentence code.

[0094] For example, when the feature vectors are [2,3,1,1] and [5,4,1,1], the resulting short sentence code is [2,3,5,4,0,0,0,0], and the key information included in the short sentence code is: Beijing, China.

[0095] 104. Concatenate the short sentence code and the initial vector to obtain a first fused vector, and input the first fused vector and a preset random variable into a preset text associator for feature extraction to obtain a context association vector;

[0096] In this embodiment, the text associator includes multiple unidirectional GRU neural networks, each of which outputs a context association vector. The text associator performs feature extraction by resampling keywords to make the generated short sentences richer, because the short sentences generated each time are quite different and there will not be a large number of repeated fragments. The latent variable is a newly created variable used to control which words should be included in the generated sentence.

[0097] Optionally, in one embodiment, the sentence associator includes a plurality of third unidirectional GRU neural networks, the concatenation of the short sentence code and the initial vector to obtain a first fusion vector, and the first fusion vector and a preset random variable are input into a preset text associator for feature extraction to obtain a context association vector including:

[0098] Concatenating the short sentence code and the initial vector to obtain a first fusion vector;

[0099] The first fusion vector and the preset random variable are input into the third unidirectional GRU neural network to extract context-related features to obtain a context-related vector.

[0100] In this embodiment, the short sentence code and the initial vector are concatenated to obtain a first fusion vector.

[0101] For example, when the short sentence is encoded as [0,2,1,0] and the initial vector is [1,0,1,2], the first fused vector is [0,2,1,0,1,0,1,2].

[0102] 105. Input the first short sentences, the random variables and the context association vector into a preset long text generator to generate text, thereby obtaining a long text.

[0103] In this embodiment, the long text generator includes multiple unidirectional GRU neural networks, wherein each GRU neural network outputs a short sentence, and the first short sentence, the second short sentence, ... the Nth short sentence generated each time are connected in sequence to obtain a long text.

[0104] Optionally, in one embodiment, the long text generator includes a plurality of fourth unidirectional GRU neural networks, and the inputting of the first short sentences, the random variables and the context association vector into the preset long text generator for text generation to obtain the long text includes:

[0105] Concatenating the first short sentences, the random variable and the context association vector to obtain a second fusion vector;

[0106] Inputting the second fused vector into the fourth unidirectional GRU neural network for decoding to obtain a plurality of second short sentences;

[0107] The second short sentences are sequentially concatenated to obtain a long text.

[0108] In this embodiment, the first short sentences, the random variables and the context association vector are concatenated to obtain a second fused vector. The second fused vector is input into a fourth unidirectional GRU neural network, each GRU neural network in the fourth unidirectional GRU neural network outputs a short sentence, and finally the short sentences are concatenated in sequence to obtain a long text.

[0109] In an embodiment of the present invention, in order to generate a semantically coherent long text, the long text can be regarded as the splicing of multiple short sentences, and then the short sentences can be regarded as the splicing of multiple words, specifically: multiple keywords are obtained and the keywords are vectorized to obtain word vectors, and random variables are added when the keywords are selected, so that the short sentences generated are greatly different each time, thereby greatly increasing the diversity of sentence patterns. Then, the corresponding first short sentence is generated through the word vector, and then the first short sentence is encoded to obtain the short sentence code, and the short sentence code, the first short sentence and the random variable are input into the long text generator to generate multiple second short sentences, and the multiple second short sentences are sequentially spliced ​​to obtain the long text. The present invention uses multiple keywords as the basis for generating long text, improves the coherence of the long text and has better richness of the text.

[0110] See also Figure 2, the second embodiment of the long text generation method in the embodiment of the present invention includes:

[0111] 201. Acquire multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector;

[0112] 202. Concatenate the word vector and the preset variable to obtain an initial vector;

[0113] 203. Input the initial vector into the first unidirectional GRU neural network to generate short sentences, thereby obtaining a plurality of first short sentences;

[0114] 204. Calling a preset rest function to calculate the fully connected layer in the first unidirectional GRU neural network to obtain the output value of the hidden layer neuron;

[0115] 205. Determine whether the output value of the hidden layer neuron exceeds a preset threshold;

[0116] 206. If yes, stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences;

[0117] 207. If not, continue to generate short sentences through the first unidirectional GRU neural network until the output value of the hidden layer neuron exceeds the preset threshold, then stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences;

[0118] In this embodiment, the rest function is:

[0119]

[0120] in, is the output value of the hidden layer neuron, σ is the sigmoid function, which is used to map the data to the (0,1) interval, and w c and b c Represent the weights and biases of the fully connected layer of the short sentence generator, Represents a GRU neural network.

[0121] In this embodiment, Used to control the length of the generated short sentences, different The size setting can make the generated sentences of different lengths. After reaching the preset threshold, the generation of short sentences stops. The short sentence generator determines that each sentence in the long text needs to contain keywords. Each GRU neural network outputs a sentence once, and the output short sentence contains information about which keywords are used in the sentence. Multiple outputs of multiple GRU neural networks represent that the generated long text will contain multiple sentences.

[0122] 208. Perform an average pooling operation on each of the first short sentences to obtain a plurality of characteristic short sentence vectors, and input each of the characteristic short sentence vectors into a preset short sentence encoder for encoding conversion to obtain a short sentence code;

[0123] 209. Concatenate the short sentence code and the initial vector to obtain a first fused vector, and input the first fused vector and a preset random variable into a preset text associator for feature extraction to obtain a context association vector;

[0124] 210. Input the first short sentences, the random variables, and the context association vector into a preset long text generator to generate text, thereby obtaining a long text.

[0125] In the embodiment of the present invention, the long text generation is divided into several short texts, and then the short texts are generated by using keywords, while ensuring the semantic coherence of the previous and subsequent short texts. The short sentence generator generates multiple short sentences containing keywords, which can not only ensure the semantic coherence of each phrase, but also improve the coherence of the entire long text. A pause instruction is added in the process of keyword generation of short sentences. The setting can effectively control the text length. When the value of exceeds the preset threshold, the generation of short sentences is stopped to keep the length of the generated text within a controllable range.

[0126] The long text generation method in the embodiment of the present invention is described above. The long text generation device in the embodiment of the present invention is described below. Figure 3 , an embodiment of the long text generation device in the embodiment of the present invention includes:

[0127] An acquisition module 301 is used to acquire a plurality of keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector;

[0128] The processing module 302 is used to concatenate the word vector and the preset variable to obtain an initial vector, and input the initial vector into a preset short sentence generator to generate a short sentence to obtain a plurality of first short sentences;

[0129] The conversion module 303 is used to perform an average pooling operation on each of the first short sentences to obtain a plurality of characteristic short sentence vectors, and input each of the characteristic short sentence vectors into a preset short sentence encoder for encoding conversion to obtain a short sentence code;

[0130] An extraction module 304 is used to concatenate the short sentence code and the initial vector to obtain a first fusion vector, and input the first fusion vector and a preset random variable into a preset text associator for feature extraction to obtain a context association vector;

[0131] The generating module 305 is used to input the first short sentences, the random variables and the context association vector into a preset long text generator to generate text and obtain a long text.

[0132] Optionally, in a first implementation of the second aspect of the present invention, the vector encoder includes a plurality of bidirectional GRU neural networks, and the acquisition module 301 is specifically used for:

[0133] Get multiple keywords to be processed;

[0134] Inputting the keyword into the bidirectional GRU neural network for vector conversion to obtain a forward hidden state vector and a backward hidden state vector;

[0135] The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

[0136] Optionally, in a second implementation of the second aspect of the present invention, the short sentence generator includes a plurality of first unidirectional GRU neural networks, and the processing module 302 is specifically used for:

[0137] Inputting the initial vector into the first unidirectional GRU neural network to generate short sentences, thereby obtaining a plurality of first short sentences;

[0138] Calling a preset rest function to calculate the fully connected layer in the first unidirectional GRU neural network to obtain the output value of the hidden layer neurons;

[0139] Determine whether the output value of the hidden layer neuron exceeds a preset threshold;

[0140] If yes, stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences;

[0141] If not, continue to generate short sentences through the first unidirectional GRU neural network until the output value of the hidden layer neuron exceeds the preset threshold, then stop generating short sentences, and use the short sentences obtained before stopping generation as the output of the first unidirectional GRU neural network to obtain multiple first short sentences.

[0142] Optionally, in a third implementation of the second aspect of the present invention, the conversion module 303 further includes a computing unit 3031, and the computing unit 3031 is specifically configured to:

[0143] Calling a preset average pooling function, respectively calculating an average value for each of the first short sentences, to obtain a plurality of eigenvalues ​​corresponding to each short sentence; and generating a plurality of feature short sentence vectors according to the plurality of eigenvalues ​​corresponding to each of the short sentences.

[0144] Optionally, in a fourth implementation of the second aspect of the present invention, the short sentence encoder includes multiple second unidirectional GRU neural networks, and the conversion module 303 also includes an encoding unit 3032, and the encoding unit 3032 is specifically used to:

[0145] Input each of the characteristic short sentence vectors into the second unidirectional GRU neural network; and perform encoding conversion on each of the characteristic short sentence vectors through the second unidirectional GRU neural network to obtain a short sentence code.

[0146] Optionally, in a fifth implementation of the second aspect of the present invention, the sentence associator includes a plurality of third unidirectional GRU neural networks, and the extraction module 304 is specifically used for:

[0147] The short sentence code and the initial vector are concatenated to obtain a first fusion vector; the first fusion vector and a preset random variable are input into the third unidirectional GRU neural network to extract context-related features to obtain a context-related vector.

[0148] Optionally, in a sixth implementation of the second aspect of the present invention, the long text generator includes a plurality of fourth unidirectional GRU neural networks, and the generating module 305 is specifically used for:

[0149] Concatenating the first short sentences, the random variable and the context association vector to obtain a second fusion vector;

[0150] Inputting the second fused vector into the fourth unidirectional GRU neural network for decoding to obtain a plurality of second short sentences;

[0151] The second short sentences are sequentially concatenated to obtain a long text.

[0152] In an embodiment of the present invention, in order to generate a semantically coherent long text, the long text can be regarded as the splicing of multiple short sentences, and then the short sentences can be regarded as the splicing of multiple words, specifically: multiple keywords are obtained and the keywords are vectorized to obtain word vectors, and random variables are added when the keywords are selected, so that the short sentences generated are greatly different each time, thereby greatly increasing the diversity of sentence patterns. Then, the corresponding first short sentence is generated through the word vector, and then the first short sentence is encoded to obtain the short sentence code, and the short sentence code, the first short sentence and the random variable are input into the long text generator to generate multiple second short sentences, and the multiple second short sentences are sequentially spliced ​​to obtain the long text. The present invention uses multiple keywords as the basis for generating long text, improves the coherence of the long text and has better richness of the text.

[0153] above Figure 3The long text generation device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The long text generation device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0154] Figure 4 4 is a schematic diagram of the structure of a long text generation device provided by an embodiment of the present invention. The long text generation device 400 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 410 (for example, one or more processors) and a memory 420, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 433 or data 432. Among them, the memory 420 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the long text generation device 400. Furthermore, the processor 410 may be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the long text generation device 400.

[0155] The long text generation device 400 may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input and output interfaces 460, and / or one or more operating systems 431, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 4 The structure of the long text generating device shown does not constitute a limitation on the long text generating device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0156] The present invention also provides a long text generation device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the long text generation method in the above embodiments.

[0157] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the long text generation method.

[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0160] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A long text generation method, characterized in that: The long text generation method comprises: Obtain multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector; splicing the word vector and the preset variable to obtain an initial vector, and inputting the initial vector into a preset short sentence generator to generate a short sentence, thereby obtaining a plurality of first short sentences; Performing an average pooling operation on each first short sentence respectively to obtain a plurality of characteristic short sentence vectors, and inputting each characteristic short sentence vector into a preset short sentence encoder for encoding conversion to obtain a short sentence code; splicing the short sentence code and the initial vector to obtain a first fused vector, and inputting the first fused vector and a random variable into a preset text associator for feature extraction to obtain a context association vector; Inputting each first short sentence, the random variable and the context association vector into a preset long text generator to generate text, thereby obtaining a long text; The short sentence generator includes a plurality of first unidirectional GRU neural networks, and the inputting the initial vector into the preset short sentence generator to generate short sentences to obtain a plurality of first short sentences includes: inputting the initial vector into the first unidirectional GRU neural network to generate short sentences to obtain a plurality of first short sentences; calling a preset rest function to calculate the fully connected layer in the first unidirectional GRU neural network to obtain a hidden layer neuron output value; judging whether the hidden layer neuron output value exceeds a preset threshold; if so, stopping generating short sentences, and using the short sentences obtained before stopping generating as the output of the first unidirectional GRU neural network to obtain a plurality of first short sentences; if not, continuing to generate short sentences through the first unidirectional GRU neural network until the hidden layer neuron output value exceeds the preset threshold, then stopping generating short sentences, and using the short sentences obtained before stopping generating as the output of the first unidirectional GRU neural network to obtain a plurality of first short sentences; The preset text associator includes a plurality of third unidirectional GRU neural networks, the concatenating the short sentence code and the initial vector to obtain a first fused vector, and inputting the first fused vector and a random variable into the preset text associator for feature extraction to obtain a context association vector includes: concatenating the short sentence code and the initial vector to obtain a first fused vector; inputting the first fused vector and the random variable into the third unidirectional GRU neural network for context association feature extraction to obtain a context association vector; The long text generator includes multiple fourth unidirectional GRU neural networks. The first short sentence, the random variable and the context association vector are input into the preset long text generator for text generation to obtain the long text, which includes: splicing the first short sentences, the random variable and the context association vector to obtain a second fusion vector; inputting the second fusion vector into the fourth unidirectional GRU neural network for decoding to obtain multiple second short sentences; and splicing the second short sentences in turn to obtain the long text.

2. The long text generation method according to claim 1, characterized in that: The vector encoder includes a plurality of bidirectional GRU neural networks, and the plurality of keywords to be processed are obtained, and each keyword is input into a preset vector encoder for vector conversion, and the word vector obtained includes: Get multiple keywords to be processed; Inputting the keyword into the bidirectional GRU neural network for vector conversion to obtain a forward hidden state vector and a backward hidden state vector; The forward hidden state vector and the backward hidden state vector are concatenated to obtain a word vector.

3. The long text generation method according to claim 1, characterized in that: The average pooling operation is performed on each first short sentence to obtain multiple feature short sentence vectors, including: Call the preset average pooling function to calculate the average value of each first short sentence to obtain multiple feature values ​​corresponding to each short sentence; According to the multiple feature values ​​corresponding to each short sentence, multiple feature short sentence vectors are generated.

4. The long text generation method according to claim 1 or 3, characterized in that: The short sentence encoder includes a plurality of second unidirectional GRU neural networks, and the inputting each characteristic short sentence vector into a preset short sentence encoder for encoding conversion to obtain the short sentence encoding includes: Input each feature short sentence vector into the second unidirectional GRU neural network; Through the second unidirectional GRU neural network, each characteristic short sentence vector is encoded and converted to obtain a short sentence code.

5. A long text generation device, characterized in that: The long text generating device comprises: An acquisition module is used to acquire multiple keywords to be processed, and input each keyword into a preset vector encoder for vector conversion to obtain a word vector; A processing module, used for concatenating the word vector and the preset variable to obtain an initial vector, and inputting the initial vector into a preset short sentence generator to generate a short sentence, thereby obtaining a plurality of first short sentences; A conversion module, used to perform an average pooling operation on each first short sentence to obtain a plurality of characteristic short sentence vectors, and input each characteristic short sentence vector into a preset short sentence encoder for encoding conversion to obtain a short sentence code; An extraction module, used for concatenating the short sentence code and the initial vector to obtain a first fusion vector, and inputting the first fusion vector and a random variable into a preset text associator for feature extraction to obtain a context association vector; A generating module, used for inputting each first short sentence, the random variable and the context association vector into a preset long text generator to generate text and obtain a long text; The short sentence generator includes a plurality of first unidirectional GRU neural networks, and the processing module is specifically used to: input the initial vector into the first unidirectional GRU neural network to generate short sentences, and obtain a plurality of first short sentences; call a preset rest function, calculate the fully connected layer in the first unidirectional GRU neural network, and obtain a hidden layer neuron output value; determine whether the hidden layer neuron output value exceeds a preset threshold; if so, stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network, and obtain a plurality of first short sentences; if not, continue to generate short sentences through the first unidirectional GRU neural network until the hidden layer neuron output value exceeds the preset threshold, then stop generating short sentences, and use the short sentences obtained before stopping the generation as the output of the first unidirectional GRU neural network, and obtain a plurality of first short sentences; The preset text associator includes a plurality of third unidirectional GRU neural networks, and the extraction module is specifically used to: concatenate the short sentence code and the initial vector to obtain a first fusion vector; input the first fusion vector and the random variable into the third unidirectional GRU neural network to extract context association features to obtain a context association vector; The long text generator includes multiple fourth unidirectional GRU neural networks, and the generation module is specifically used to: splice each first short sentence, the random variable and the context association vector to obtain a second fusion vector; input the second fusion vector into the fourth unidirectional GRU neural network for decoding to obtain multiple second short sentences; splice each second short sentence in turn to obtain a long text.

6. A long text generation device, characterized in that: The long text generation device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the long text generation device to execute the long text generation method according to any one of claims 1 to 4.

7. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the long text generation method according to any one of claims 1 to 4 is implemented.

Images