Article generation method, electronic device and storage medium

Abstract

The present application discloses an article generation method, electronic device, and storage medium. The article generation method includes: obtaining an article title, encoding the article title using a title encoder to obtain title encoding features, performing article planning prediction on the title encoding features using a planning decoder to obtain target planning features; and then, using a writing decoder to perform article prediction based on the target planning features and the title encoding features to obtain a target predicted article corresponding to the article title. In this manner, the article generation method disclosed in the present application can use the planning decoder and the writing decoder separately, separating article planning from article writing, to achieve article generation based on the article title.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, in particular to an article generation method, an electronic device and a storage medium. Background Art

[0002] At present, the texts generated by common text generation technologies have requirements on the length of the text and generally only support the generation of shorter sentences. In addition, the text generation model used by this text generation technology can use the same decoder for planning and writing. In other words, planning and writing are carried out simultaneously. The text generated based on one decoder is lacking in logic and structural layout, and is prone to problems such as incoherent sentences. Summary of the Invention

[0003] The present application provides an article generation method, electronic device and storage medium. By using a planning decoder and a writing decoder separately, the purpose of obtaining a predicted article based on the article title is achieved by planning first and then writing, which can improve the logic and readability of the predicted article.

[0004] In order to solve the above technical problems, a technical solution adopted in this application is: to provide an article generation method, which includes: obtaining the title of the article; encoding the title of the article using a title encoder to obtain title encoding features; performing article planning prediction on the title encoding features using a planning decoder to obtain target planning features; and performing article prediction based on the target planning features and the title encoding features using a writing decoder to obtain a target predicted article corresponding to the title of the article.

[0005] Among them, using the planning decoder to predict the article planning based on the title coding features to obtain the target planning features, including: obtaining the historical planning features of the planning decoder; using the planning decoder to combine the title coding features and the historical planning features to predict the article planning to obtain the target planning features.

[0006] Among them, the planning decoder includes a planning attention layer, and the planning decoder is used to combine the topic encoding features and historical planning features to perform article planning prediction to obtain the target planning features, including: using the planning attention layer to combine the topic encoding features and historical planning features to perform article planning prediction to obtain a planning attention matrix, and using the planning attention matrix as the target planning feature.

[0007] Among them, the planning decoder is used in combination with the title coding features and the historical planning features to predict the article planning and obtain the target planning features, including: using the planning decoder in combination with the title coding features to extract keyword features or entity relationship features from the historical planning features; and the article planning is predicted based on the extracted keyword features or entity relationship features to obtain the target planning features.

[0008] Among them, the writing decoder is used to predict articles according to the target planning features and the title encoding features to obtain the target predicted article corresponding to the title of the article, including: obtaining the historical predicted articles of the writing decoder; using the writing decoder to combine the target planning features, the title encoding features and the historical predicted articles to predict the article to obtain the target predicted article corresponding to the title of the article.

[0009] Among them, the target planning features include the planning attention matrix; using the writing decoder in combination with the target planning features, the title encoding features and the historical prediction articles to perform article prediction, and obtain the target prediction article corresponding to the article title, including: using the writing decoder in combination with the planning attention matrix, the title encoding features and the historical prediction articles to perform article prediction, and obtain the target prediction article corresponding to the article title.

[0010] Among them, the writing decoder includes a writing attention layer, and the writing decoder is used to combine the planning attention matrix, the topic encoding features and the historical prediction articles to perform article prediction, and obtain the target predicted article corresponding to the article title, including: using the writing attention layer in combination with the planning attention matrix to obtain the writing attention matrix corresponding to the writing attention layer; using the writing attention matrix to perform article prediction on the topic encoding features and the historical prediction articles, and obtain the target predicted article corresponding to the article title.

[0011] Among them, after using the writing decoder to predict the article according to the target planning features and the topic encoding features to obtain the target predicted article corresponding to the article title, it includes: obtaining the real plan and the real article corresponding to the article title; determining the first loss value using the real plan and the target planning features; determining the second loss value using the real article and the target predicted article; and adjusting the parameters of the topic encoder, the planning decoder and the writing decoder according to the first loss value and the second loss value.

[0012] In order to solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, which includes a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to execute the computer programs to implement the above article generation method.

[0013] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the above article generation method when executed by a processor.

[0014] The beneficial effect of the present application is: different from the existing technology, the article generation method provided by the present application first uses the title encoder to encode the article title to obtain the title encoding feature, and then uses the planning decoder to predict the article planning on the title encoding feature to obtain the target planning feature, and then uses the writing decoder to predict the article according to the target planning feature and the title encoding feature to obtain the target prediction article corresponding to the article title. The planning and writing are decoded using different decoders in the manner of planning first and then writing, which not only reduces the difficulty of article prediction, but also improves the logic and readability of the predicted article, thereby improving the quality and diversity of the generated articles. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0016] Figure 1 This is a flowchart of the first embodiment of the article generation method provided by this application;

[0017] Figure 2 This is a flowchart of an embodiment of step 13 provided in this application;

[0018] Figure 3 This is a flowchart of an embodiment of step 14 provided in this application;

[0019] Figure 4 This is a flow chart of the second embodiment of the article generation method provided by this application;

[0020] Figure 5 It is a schematic diagram of an embodiment of the method for generating an argumentative essay provided by the present application;

[0021] Figure 6 This is a structural diagram of an embodiment of an electronic device provided by the present application;

[0022] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] See Figure 1 , Figure 1 : is a flowchart of the first embodiment of the article generation method provided by this application, the method comprising:

[0025] Step 11: Get the article title.

[0026] In some embodiments, in response to a user's input instruction, the article title is input into a trained article generation model, so that the article generation model is used to perform corresponding operations on the article title to generate an article corresponding to the input article title.

[0027] Among them, the article generation model can be a model based on the Transformer structure, such as the BERT (Bidirectional Encoder Representations from Transformer) model and the T5 (Text-to-Text Transfer Transformer) model, which are not limited here. The BERT model is a language model based on the Transformer structure that can combine bidirectional and autoregressive methods. The T5 model is a model based on the Transformer structure that can convert all NLP (Natural Language Processing) tasks into the same text-to-text format. The input and output of the task are both text strings.

[0028] Step 12: Use the title encoder to encode the article title to obtain the title encoding feature.

[0029] The title encoder is an encoder embedded in the article generation model.

[0030] In some embodiments, the article generation model is a BERT model, and the obtained article title is encoded using the encoder of the BERT model to obtain the corresponding title encoding feature, i.e., H e =Encoder(X), where X represents the title of the article, H e Indicates the coding features of the question.

[0031] Specifically, the article title is represented by X = {x1, x2, ..., xn}, and the encoder of the article generator uses the embedding mapping function to map X = {x1, x2, ..., xn} to Emb(X) = {e1, e2, ..., en}, and then uses the context mapping function / Enc function to map it to H e=Enc(Emb(X))={h1, h2, ..., hn}.

[0032] Step 13: Use the planning decoder to predict the article planning based on the title encoding features to obtain the target planning features.

[0033] The planning decoder is a Transformer-based decoder used to generate article plans based on article titles. It generates article plans using a regression approach, taking historical plans as input and combining them with the title encoding features to predict them word by word to obtain a predicted plan.

[0034] In some embodiments, see Figure 2 , step 13 may include the following process:

[0035] Step 21: Obtain historical planning features of the planning decoder.

[0036] It is understandable that the historical planning feature is a plan obtained by historical prediction. In some embodiments, the historical planning feature is represented by Y p <t To express.

[0037] Step 22: Use the planning decoder to combine the title encoding features and historical planning features to predict the article planning and obtain the target planning features.

[0038] In some embodiments, the planning decoder is a decoder of the BERT model, which can perform article planning prediction based on the title encoding features and historical planning features to obtain target planning features.

[0039] First, the planning decoder is used to process the topic encoding features and historical planning features to obtain H p t =PD(H e , Y p <t ), where PD represents Plan Decoder. p t represents the hidden state representation of the t-th character.

[0040] In some embodiments, obtaining H p t Afterwards, we can use the learnable network parameters W p 、b p , using the normalization function to H p t Processing to obtain target planning features (W p *H p t +bp ),in, represents the t-th character in the target plan features predicted by the plan decoder.

[0041] In some embodiments, the planning decoder includes a planning attention layer, which can be used to combine the topic encoding features and historical planning features to perform article planning prediction, obtain a planning attention matrix, and use the planning attention matrix as the target planning feature.

[0042] Each Transformer layer of the planning decoder contains three sublayers, namely the self-attention layer, the interactive attention layer, and the feedforward network layer. The self-attention layer is the planning attention layer of the planning decoder. It can be used to combine the self-attention layer with the title encoding features and the historical planning features to predict the article planning and obtain the planning attention matrix. The planning attention matrix includes the key matrix and the value matrix. The key matrix and the value matrix can be used as the target planning features. The key matrix is ​​denoted as K p , the value matrix is ​​recorded as V p .

[0043] In some embodiments, a planning decoder is used in combination with title coding features to extract keyword features or entity relationship features from historical planning features, so as to perform article planning prediction based on the extracted keyword features or entity relationship features to obtain target planning features.

[0044] Specifically, keyword feature extraction from historical planning features using the planning decoder combined with the title encoding features can be achieved using TF-IDF (Term Frequency-Inverse Document Frequency). TF-IDF is a commonly used weighting technique used in information retrieval and data mining, where TF stands for term frequency and IDF stands for inverse document frequency. TF-IDF is also a statistical method that can be used to assess the importance of a word to a document in a document collection / corpus.

[0045] The expression for extracting keyword features using TF-IDF is Y p =k1#1|k2#2|…|k i #i, where k i Refers to the i-th keyword. In addition, the purpose of adding “#i” is to enable the article generation model to perceive the number of predicted keyword features. p As a historical planning feature.

[0046] Specifically, the entity relationship feature extraction of historical planning features using the planning decoder combined with the topic coding features can be achieved by using the OpenIE tool. The OpenIE tool is an open domain information extraction tool, which is an ontology-free information extraction paradigm. The extraction form of the OpenIE tool is (subject: relationship: object), (subject: relationship: object) is also called a triple, where "relationship" is an entity relationship. Entity relationships are divided into two types, one is property (property) and the other is relationship (relation). If the entity relationship is a property, the two entities contained in the triple are a topic (topic) and a string. For example, if the property is a type, the topic is a person, and the string is Xiao Ming, then the triple is (Xiao Ming, type, person); if the entity relationship is a relation, the two entities contained in the triple are usually two topics (topic). For example, if the relationship is a birthplace, the topic is a person and a place name, then the triple is (Xiao Ming, birthplace, London). In the embodiment of the present application, the entity relationship in the triple used is a relation.

[0047] The expression for extracting entity relationship features using OpenIE tool is Y p =s1#r1#o1#1|s2#r2#o2#2|…|s i #r i #o i #i, the corresponding triple is (s i , r i , o i ), where s i represents the i-th main entity, r i represents the i-th relationship, o i Indicates the i-th target entity, and the purpose of adding “#i” is to enable the article generation model to perceive the number of predicted entity relationship features. p As a historical planning feature.

[0048] In other embodiments, a planning decoder is used in combination with topic coding features to extract semantic role information features from historical planning features, so as to perform article planning prediction based on the extracted voice role information features and obtain target planning features.

[0049] In other embodiments, the article planning may be manually generated to obtain target planning features.

[0050] Step 14: Use the writing decoder to predict the article based on the target planning features and the title encoding features to obtain the target predicted article corresponding to the article title.

[0051] In some embodiments, see Figure 3 , step 14 may include the following process:

[0052] Step 31: Get the historical prediction articles for the writing decoder.

[0053] Understandably, historical prediction articles correspond to historical planning features and historical article titles. e <t express.

[0054] Step 32: Use the writing decoder to combine the target planning features, the title encoding features and the historical predicted articles to predict the article and obtain the target predicted article corresponding to the article title.

[0055] In some embodiments, the target planning features include a planning attention matrix, and a writing decoder can be used to combine the planning attention matrix, the topic encoding features, and historical predicted articles to perform article prediction to obtain a target predicted article corresponding to the article title.

[0056] The writing decoder can predict articles based on historical planning features, topic encoding features and historical predicted articles, namely H w t =WD(H e , K p , V p , Y e <t ), where WD is the Word Decoder, K p and V p They are the key matrix and value matrix of the planning decoder respectively.

[0057] Get H w t Afterwards, we can use the learnable network parameters W in the neural network to w 、b w , using the normalization function to H w t Processing, that is in, represents the t-th character in the target prediction article predicted by the writing decoder.

[0058] In some embodiments, the writing decoder includes a writing attention layer, which can be used in combination with the planning attention matrix to obtain a writing attention matrix corresponding to the writing attention layer, and then use the writing attention matrix to perform article prediction on the title encoding features and historical predicted articles to obtain a target predicted article corresponding to the article title.

[0059] The attention layer of each Transformer layer in the decoder consists of a key matrix, a value matrix, and a query matrix. The key matrix, the value matrix, and the query matrix are represented by K w 、V w , Q w express.

[0060] Among them, Q w =Q' w ; Q' w , K' w and V' w They represent the original query matrix, key matrix, and value matrix in the decoder respectively.

[0061] Indicates that the matrix is ​​concatenated in the first dimension, such as "matrix Matrix 2", where Matrix 1 is [[[1],[2]], [[5],[6]], [[9],[8]]], and Matrix 2 is [[[3],[4]], [[7],[8]], [[3],[2]]], then "Matrix Matrix 2" represents [[[1],[2],[3],[4]],[[5],[6],[7],[8]],[[9],[8],[3],[2]]].

[0062] By using two decoders to implement text planning and article writing respectively, and the planning decoder guiding the writing decoder's writing through planning attention, this approach can avoid interference between the planning and writing processes, thereby improving the prediction quality of text planning.

[0063] Different from the existing technology, the article generation method provided by this application encodes the article title through a title encoder to obtain the title encoding features, and then uses a planning decoder to perform article planning prediction on the title encoding features to obtain the target planning features, and finally uses a writing decoder to perform article prediction based on the target planning features and the article title to obtain the target predicted article. The above method separates the planning decoder and the writing decoder by planning first and then writing, which can improve the logic and readability of the predicted article to obtain articles that better meet the requirements.

[0064] In some embodiments, the above-mentioned topic encoder, planning decoder and writing decoder can be trained, and the trained topic encoder, planning decoder and writing decoder can be used to achieve article prediction.

[0065] See Figure 4 , Figure 4: is a flow chart of a second embodiment of the article generation method provided by this application, the method comprising:

[0066] Step 41: Get the article title.

[0067] Step 42: Encode the article title using the title encoder to obtain the title encoding feature.

[0068] Step 43: Use the planning decoder to perform article planning prediction on the title encoding features to obtain the target planning features.

[0069] Step 44: Use the writing decoder to predict the article according to the target planning features and the title encoding features to obtain the target predicted article corresponding to the article title.

[0070] Steps 41 to 44 may have the same or similar technical features as any of the above embodiments, and will not be repeated here.

[0071] Step 45: Obtain the real plan and real article corresponding to the article title.

[0072] In some embodiments, the real plan may belong to the historical plan feature, with Y p <t Indicates that the real article can belong to the historical prediction article, with Y e <t express.

[0073] Step 46: Determine a first loss value using the true plan and target plan features.

[0074] In some embodiments, the real plan is Y p <t Indicates that the target planning feature Y p By several Y p t Composition (where t has different values), since the summation can be used here to meet ∑, the target planning feature is Y p t Indicates that the calculation of the first loss value can use the negative log-likelihood as the loss function, expressed as L w =-∑ i t＝1 logP(Y p t |Y p <t , X), where t ranges from [1, i], i represents Y p t , where X represents the title of the article.

[0075] Step 47: Determine a second loss value using the true article and the target predicted article.

[0076] In some embodiments, the real article may belong to the historical prediction article, and Y e <t Indicates that the target prediction article Y e By several Y e t Composition (where t has different values), since the summation can be used here, the target prediction article uses Y e t Indicates that the calculation of the second loss value can use the negative log likelihood as the loss function, expressed as L p =-∑ n t＝1 logP(Y e t |Y e <t , Y p , X), where t ranges from [1, n], and n represents the number of Y in the predicted article. e t The number of

[0077] Step 48: Adjust the parameters of the topic encoder, planning decoder, and writing decoder according to the first loss value and the second loss value.

[0078] Based on the first loss value and the second loss value, the total loss value can be obtained as L=L w +L p .

[0079] In addition, the calculation of the first loss value involves the topic encoder and the planning decoder, so the parameters of the topic encoder and the planning decoder can be changed based on the first loss value to minimize the first loss value; the calculation of the second loss value involves the topic encoder, the planning decoder, and the writing decoder, so the parameters of the topic encoder, the planning decoder, and the writing decoder can be changed based on the second loss value to minimize the second loss value. In general, the purpose of adjusting the parameters of the topic encoder, the planning decoder, and the writing decoder is to minimize the total loss function L. During the adjustment, the parameters of the topic encoder, the planning decoder, and the writing decoder can be dynamically adjusted based on the first loss value and the second loss value.

[0080] Different from the existing technology, the article generation method provided in this application can separate the planning decoder and the writing decoder by planning first and then writing to obtain articles that better meet the requirements. After obtaining the article, the first loss value and the second loss value can be obtained to dynamically adjust the parameters of the title encoder, planning decoder and writing decoder based on the first loss value and the second loss value to continuously train the article generation model, so that the target predicted article generated by the article generation model is more accurate and the logic and readability of the predicted article are improved.

[0081] The article in this application can be a logical article, such as an argumentative essay, see Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the argumentative essay generation method provided in the present application. The title of the argumentative essay can be input into the title encoder to obtain the title encoding features, and then the planning decoder is used to perform article planning prediction based on the title encoding features and historical planning features to obtain the target planning features and the planning output probability distribution. Then, the target planning features are processed using the planning attention mechanism in the planning decoder to obtain the planning attention matrix. Then, the writing decoder can be used to combine the planning attention matrix, the title encoding features and the historical predicted articles to perform article prediction to obtain the target predicted article corresponding to the article title and the argumentative essay output probability distribution.

[0082] The planning output probability distribution may be the probability distribution of keywords included in the article planning; and the argumentative paper output probability distribution may be the probability distribution of the article planning included in the argumentative paper.

[0083] Through the above method, planning and writing can be decoded using different decoders in the manner of planning first and then writing. This not only reduces the difficulty of article prediction, but also improves the logic and readability of the predicted articles, thereby improving the quality and diversity of the generated articles.

[0084] See Figure 6 , Figure 6 This is a structural diagram of an embodiment of an electronic device provided in the present application. The electronic device 60 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs, and the processor 602 is used to execute computer programs to implement the article generation method of any of the above embodiments. No further details will be given here.

[0085] See Figure 7 , Figure 7 This is a structural diagram of an embodiment of a computer-readable storage medium provided in the present application. The computer-readable storage medium 70 stores a computer program 701. When the computer program 701 is executed by a processor, it is used to implement the article generation method of any of the above embodiments, which will not be described in detail here.

[0086] In summary, the article generation method provided by the present application can decode planning and writing using different decoders in a planning-first-then-writing manner, which not only reduces the difficulty of article prediction, but also improves the logic and readability of the predicted article, thereby improving the quality and diversity of the generated article. After obtaining the article, the first loss value and the second loss value can be obtained to dynamically adjust the parameters of the topic encoder, planning decoder, and writing decoder based on the first loss value and the second loss value, so as to continuously train the article generation model, so that the target predicted article generated by the article generation model is more accurate, and the logic and readability of the predicted article can be improved.

[0087] The present application can also design an article generation tool for terminal devices (such as laptops and smart phones) based on the method described in any of the above embodiments. The display interface of the article generation tool on the terminal device may include area A and area B. Area A supports users to customize the input of article titles and display the input article titles. Area B is used to display predicted articles related to the article titles.

[0088] The processor involved in this application can be called a CPU (Central Processing Unit), which may be an integrated circuit chip, or a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0089] The storage media used in this application include various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), or an optical disk.

[0090] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating an article, characterized in that: The method comprises: Get the article title; Encode the article title using a title encoder to obtain a title encoding feature; Using a planning decoder to perform article planning prediction on the title encoding features to obtain target planning features; wherein the target planning features include a planning attention matrix; Obtaining historical prediction articles from a writing decoder; the writing decoder including a writing attention layer; Utilizing the writing attention layer in combination with the planning attention matrix, a writing attention matrix corresponding to the writing attention layer is obtained; The writing attention matrix is ​​used to perform article prediction on the title encoding features and the historical predicted articles to obtain a target predicted article corresponding to the article title.

2. The method according to claim 1, characterized in that The method of using a planning decoder to perform article planning prediction on the title encoding feature to obtain a target planning feature includes: Obtaining historical planning features of the planning decoder; The planning decoder is used to combine the title coding feature and the historical planning feature to perform article planning prediction to obtain the target planning feature.

3. The method according to claim 2, characterized in that The planning decoder includes a planning attention layer. The planning decoder is used to combine the topic encoding features and the historical planning features to perform article planning prediction to obtain the target planning features, including: The planning attention layer is used in combination with the topic coding feature and the historical planning feature to perform article planning prediction, thereby obtaining a planning attention matrix, which is used as the target planning feature.

4. The method according to claim 2, characterized in that The method of using a planning decoder to combine the topic coding feature and the historical planning feature to perform article planning prediction to obtain the target planning feature includes: Using the planning decoder in combination with the title coding feature, keyword feature extraction or entity relationship feature extraction is performed on the historical planning feature; Article planning prediction is performed based on the extracted keyword features or entity relationship features to obtain the target planning features.

5. The method according to claim 1, wherein After obtaining the target predicted article corresponding to the article title, the method includes: Obtaining the real plan and real article corresponding to the article title; Determining a first loss value using the actual plan and the target plan characteristics; Determine a second loss value using the real article and the target predicted article; Adjust parameters of the topic encoder, the planning decoder, and the writing decoder according to the first loss value and the second loss value.

6. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 5.

Images