Text summarization generation method and device, electronic equipment and storage medium

Abstract

The application discloses a text abstract generation method and device, electronic equipment and storage medium, and belongs to the technical field of natural language processing. The method comprises the following steps: obtaining semantic features corresponding to a text sequence of a source text; calculating a first word attention weight and an encoding end context vector by using a time attention network of an encoding end according to the semantic features; obtaining a hidden state of a current time step of a decoder corresponding to a summary text sequence of the source text; calculating a decoding end context vector by using a decoding end attention network according to the hidden state of the current time step of the decoder and hidden states of historical time steps of the decoder; and generating a text abstract of the source text according to the first word attention weight, the encoding end context vector and the decoding end context vector. The application solves the technical problem of repeated generation of abstracts in the related art.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for generating text summaries. Background Technology

[0002] Sequence-to-Sequence (Seq2Seq) generative text summarization models are mainly applied to news headline generation and key information retrieval. They solve the problem of unequal input and output sequence lengths while enhancing the model's ability to learn the semantics and syntax of text content. The Seq2Seq model mainly consists of an encoder and a decoder. The summarization process is as follows: First, the encoder calculates the current hidden state based on the current input text sequence and the hidden state of the previous time step. After obtaining the hidden states at each time step, a fixed-length context semantic vector C is calculated using a nonlinear function. This vector can be considered as the semantic representation of the input text; this process is called encoding. Second, the semantic vector C is passed to the decoder. The decoder calculates the hidden state of the current time step based on the hidden state of the previous time step, the output, and C; this process is called decoding. Finally, the probability of each predicted word is calculated using the Softmax function, and the word with the highest probability is output.

[0003] However, in the Encoder-Decoder structure, during the decoding stage, the source text information received by the Decoder is represented only by a compressed and invariant-length semantic vector C. This means that any word in the input sequence X has the same influence on the result of the summary generation, lacking attention to key keywords, which leads to the problem of duplicate generated summary results.

[0004] There are currently no effective solutions to the aforementioned problems in the relevant technologies. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for generating text summaries to solve the technical problem of repetitive summaries generated in related technologies.

[0006] According to one aspect of the embodiments of this application, a method for generating text summaries is provided, comprising: obtaining semantic features corresponding to a sequence of body text of a source text; calculating a first word attention weight and an encoding-side context vector using a temporal attention network at the encoding end based on the semantic features; obtaining the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text; calculating a decoding-side context vector using a decoding-side attention network based on the hidden state of the decoder at the current time step and the hidden state of the decoder at historical time steps; and generating a text summary of the source text based on the encoding-side context vector, the decoding-side context vector, and the first word attention weight.

[0007] According to another aspect of the embodiments of this application, a text summarization generation apparatus is also provided, comprising: a first acquisition module, configured to acquire semantic features corresponding to a sequence of body text of a source text; a first calculation module, configured to calculate a first word attention weight and an encoding-end context vector using a temporal attention network at the encoding end based on the semantic features; a second acquisition module, configured to acquire the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text; a second calculation module, configured to calculate a decoding-end context vector using a decoding-end attention network based on the hidden state of the decoder at the current time step and the hidden state of the decoder at historical time steps; and a generation module, configured to generate a text summary of the source text based on the encoding-end context vector, the decoding-end context vector, and the first word attention weight.

[0008] Furthermore, the first acquisition module includes a first acquisition unit, used to extract features from the main text sequence to obtain main text word vectors; input the main text word vectors into an encoder, the encoder being a single-layer bidirectional gated recurrent neural network Bi-GRU; and merge the main text word vectors and the historical output data of the encoder at the previous time step through the Bi-GRU to obtain the semantic features of the encoder at the current time step.

[0009] Further, the first calculation module includes a first calculation unit, used to obtain the hidden state and unit state of the current time step output by the decoder, and merge the hidden state and the unit state to obtain a merged hidden state; concatenate the merged hidden state and the semantic features into the temporal attention network of the encoding end, and normalize the concatenated features using a temporal attention function to obtain the first word attention weight, wherein the temporal attention function is... t is the t-th time step, and EXP(·) is an exponential function with the natural constant e as the base; the first word attention weight and the semantic features are weighted and summed to obtain the encoding context vector.

[0010] Furthermore, the second acquisition module includes a second acquisition unit, used to extract the built-in summary content of the source text and obtain a summary text sequence of the built-in summary content; perform feature extraction on the summary text sequence to obtain summary word vectors; concatenate the summary word vectors with the encoding end context vector to obtain the decoding end input vector; and input the decoding end input vector and the historical output data of the decoder at the previous time step into the decoder to obtain the hidden state of the decoder at the current time step.

[0011] Furthermore, the second calculation module includes a second calculation unit, which is used to input the hidden state of the current time step of the decoder and the hidden state of the historical time step of the decoder into the decoding end attention network for concatenation, obtain the attention scale through a nonlinear activation function, input the attention scale into the Softmax layer to obtain the second word attention weight, and perform a weighted summation of the second word attention weight and the hidden state of the historical time step of the decoder to obtain the decoding end context vector.

[0012] Furthermore, the generation module includes a first generation unit, used to concatenate the summary word vector corresponding to the summary text sequence with the encoding end context vector to obtain the decoding end input vector; merge the hidden state and unit state of the current time step output by the decoder to obtain the merged hidden state; merge the decoding end context vector, the merged hidden state, and the decoding end input vector to obtain the weight coefficient Pgen; and generate the text summary of the source text based on Pgen and the first word attention weight.

[0013] Further, the generation module includes a second generation unit, used to merge the encoding end context vector, the hidden state of the decoder at the current time step, and the decoding end input vector to obtain a real number vector; using a single linear connection layer to convert the real number vector into the score corresponding to each output word in the training vocabulary, and multiplying the score by Pgen to obtain the first distribution probability of the generated word on the training vocabulary; multiplying the attention weight of the first word by the weight a to obtain the second distribution probability of the generated word in the input text, where a + Pgen = 1; adding the first distribution probability and the second distribution probability to obtain the third distribution probability of the generated word on the expanded vocabulary, and outputting the sentence with the largest third distribution probability as the text summary of the source text.

[0014] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above steps when the program is run.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the programs stored in the memory.

[0016] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above-described method.

[0017] This application adds a temporal attention network to the text encoding stage, records the hidden state at each step of the encoding and decoding, calculates word attention weights through the hidden states, and calculates the current word attention weight using the historical attention weights before each input step. By making full use of different parts of the input, it enhances the focus on key keywords and effectively solves the problem of duplicate generated summaries in the Pointer Generator model. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of a text digest generation method according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the pointer-time attention model in the embodiments of this application;

[0022] Figure 4 This is a comparative diagram of the loss functions in the embodiments of this application;

[0023] Figure 5 This is a structural block diagram of a text summary generation apparatus according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile phone, computer, tablet, or similar computing device. Taking running on a computer as an example, Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application. Figure 1 As shown, a computer may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the computer may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer described above. For example, the computer may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a text digest generation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a computer's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0030] This embodiment provides a method for generating text summaries. Figure 2 This is a flowchart of a text digest generation method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0031] Step S10: Obtain the semantic features corresponding to the main text sequence of the source text;

[0032] Step S20: Based on the semantic features, a temporal attention network is used to calculate the attention weight of the first word and the encoding context vector;

[0033] The network architecture used in this embodiment consists of a temporal attention network and an encoder-decoder combination. The temporal attention network is set at the encoder end. The source text includes its own main content and summary content. The main content is input into the encoder to obtain the semantic features corresponding to the output text sequence of the encoder. In this embodiment, the semantic features are the encoder output, denoted as enc_out. Specifically, the encoder output includes the semantic features enc_out at the current time step and the hidden state enc_h of the encoder at the current time step. The attention weight of the first word can be calculated using the temporal attention network at the encoder end. and encoding context vector

[0034] Step S30: Obtain the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text;

[0035] Step S40: Based on the hidden state of the current time step of the decoder and the hidden state of the historical time steps of the decoder, the decoder attention network is used to calculate the decoder context vector.

[0036] The source text's built-in summary is input into the decoder to obtain the hidden state at the current time step output by the decoder. In this embodiment, the hidden state at the current time step of the decoder is denoted as dec_h. The decoder-side attention network is used to calculate the decoder-side context vector.

[0037] Step S50: Generate a text summary of the source text based on the encoding end context vector, the decoding end context vector, and the first word attention weight.

[0038] Specifically, the decoding context vector The hidden state of the current time step output by the decoder and the hidden state obtained by concatenating the unit states. The weight coefficient Pgen is obtained by merging the decoder input vector y with the decoder input vector y, where the decoder input vector y is the summation word vector and the encoder context vector. The generated words are obtained by concatenation. Finally, the vocabulary distribution at the decoding end is multiplied by Pgen to obtain the probability distribution of the generated words on the training vocabulary. The attention weights of the words at the encoding end are then applied. Multiply by the weight 'a', where a + pgen = 1, to obtain the probability distribution of the generated word in the input text. Add the two probability distributions to obtain the probability distribution of the final generated word on the expanded vocabulary. Output the sentence with the highest probability as the text summary.

[0039] By adding a temporal attention network to the text encoding stage through the above steps, the hidden state of each step at the encoding and decoding ends is recorded. The word attention weight is calculated through the hidden state, and the current word attention weight is calculated by using the historical attention weights before each input step. This fully utilizes different parts of the input, enhances the focus on key keywords, and effectively solves the problem of duplicate generated summaries in the Pointer Generator Network model.

[0040] In the first embodiment of this example, obtaining the semantic features corresponding to the main text sequence of the source text includes:

[0041] Step 11: Extract features from the main text sequence to obtain main text word vectors;

[0042] Step 12: Input the text word vectors into the encoder, which is a single-layer bidirectional gated recurrent neural network Bi-GRU;

[0043] Step 13: The Bi-GRU is used to merge the word vectors in the text and the historical output data of the encoder at the previous time step to obtain the semantic features of the encoder output at the current time step.

[0044] The source text is segmented into sentences. The segmented sentences are optionally processed using the Jieba word segmentation tool to obtain multiple corresponding word vectors. These word vectors are then input into an encoder. In this embodiment, the encoder is a Bi-GRU (Bidirectional Gated Recurrent Neural Network). The encoder's historical output data from the previous time step includes the hidden state and semantic features output from the previous time step. In this embodiment, Bi-GRU is simply referred to as Bidirectional GRU. For details, please refer to [reference needed]. Figure 3 The bidirectional GRU consists of a forward-propagating GRU and a backward-propagating GRU. The GRU uses an activation function to merge the current input word vectors, the hidden state from the previous time step, and the semantic features output from the previous time step to obtain the hidden state for the current time step. The GRU includes an update gate and a reset gate. The update gate determines whether to use the hidden state of the current time step to replace the hidden state of the previous time step, while the reset gate updates the hidden state. The two GRUs with opposite propagation directions concatenate their respective hidden states and outputs to obtain the bidirectional GRU's hidden state `enc_h` and the output semantic features `enc_out`, thus associating the current hidden state and the output semantic features with the previous and subsequent states, better capturing global information.

[0045] The encoder in this embodiment is composed of a Bi-GRU, which calculates the correlation between different words within the sequence, globally searches for connections within the sequence, and thus strengthens related words and weakens irrelevant words. This effectively solves the long-term dependency problem of the Seq2Seq model and prevents gradient explosion or gradient vanishing. Moreover, the bi-directional GRU consists of only update gates and reset gates, which greatly reduces the number of parameters and makes it easy to converge, significantly shortening the training time. At the same time, it reduces the requirements for computing resources and reduces resource costs, solving the problem of large number of parameters and high computing resource requirements of the Pointer Generator model.

[0046] In the second embodiment of this example, calculating the first word attention weight and the encoding context vector using a temporal attention network at the encoding end, based on the semantic features, includes:

[0047] Step 21: Obtain the hidden state dec_h and cell state dec_c of the current time step output by the decoder, and merge the hidden state dec_h and the cell state dec_c to obtain the merged hidden state.

[0048] Step 22: The merged hidden state and the semantic features are input into the temporal attention network at the encoding end and concatenated. The temporal attention function is used to normalize the concatenated features to obtain the attention weight of the first word.

[0049] Step 23: The attention weight of the first word is weighted and summed with the semantic features to obtain the encoding context vector.

[0050] In this embodiment, the decoder is an LSTM (Bidirectional Long Short Term Memory) network. The hidden state and cell state at the current time step output by the decoder are merged to obtain the merged hidden state. Will The semantic features enc_out output by the encoder are input together into the temporal attention network at the encoder end.

[0051] Specifically, refer to Figure 3 In Temporal Attention networks, After `enc_out` and `enc_out` are each passed through a linear layer, the output of the linear layer is... Concat the features with enc_out, and then apply the tanh activation function to the concatenated features to obtain the attention score. Using the time attention function Normalization yields the first word; pay attention to the weight. Will The encoding context vector is obtained by weighted summation of the semantic features enc_out output by the encoder. (Encoder Context Vector). This embodiment uses a time attention function: Where t is the t-th time step, and EXP(·) is an exponential function with the natural constant e as the base. It reduces the enc_out of the decoder in the historical time steps that have high attention weights, prevents the model from focusing on the same input word vectors, and further reduces the phenomenon of duplicate generated summaries.

[0052] In the third embodiment of this example, obtaining the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text includes:

[0053] Step 31: Extract the built-in summary content of the source text and obtain the summary text sequence of the built-in summary content;

[0054] Step 32: Extract features from the summary text sequence to obtain summary word vectors;

[0055] Step 33: Concatenate the summary word vector with the encoding end context vector to obtain the decoding end input vector;

[0056] Step 34: Input the input vector of the decoding end and the historical output data of the decoder at the previous time step into the decoder to obtain the hidden state of the decoder at the current time step.

[0057] Extract the built-in abstract content from the source text to obtain the abstract text sequence. Use the Jieba word segmentation tool to divide the abstract text sequence into abstract word vectors. Combine the abstract word vectors with the encoding context vector. The input vector y is obtained by concatenating the two data. The input vector y and the historical output data of the decoder from the previous time step are then input into the decoder. The historical output data from the previous time step includes the hidden state and cell state of the LSTM from the previous time step. The decoder is a single-layer LSTM. The LSTM uses the current input vector y and the hidden state and cell state of the LSTM from the previous time step as inputs to obtain the current hidden state dec_h. The LSTM includes a forget gate, an update gate, and an output gate. The forget gate is used to determine how much of the cell state from the previous time step is kept in the current cell state. The update gate is used to determine how much of the current input vector is kept in the current cell state. The output gate is used to control how much of the current cell state is kept in the current hidden state.

[0058] In the fourth embodiment of this example, the decoding context vector is calculated using a decoding-side attention network based on the hidden state of the decoder at the current time step and the hidden state of the decoder at historical time steps, including:

[0059] Step 41: The hidden state of the current time step and the hidden state of the decoder's historical time steps are respectively input into the decoding end attention network and spliced ​​together. The attention scale is obtained through a non-linear activation function, and the attention scale is input into the Softmax layer to obtain the second word attention weight.

[0060] Step 42: The second word attention weight is weighted and summed with the hidden state of the decoder's historical time step to obtain the decoding context vector.

[0061] In this embodiment, the decoder-side attention network is the built-in attention mechanism of the decoder. The hidden state dec_h of the current time step of the decoder and the hidden states of the decoder at previous time steps are respectively input into the decoder-side attention network. In the decoder-side attention network, dec_h and the hidden states of the decoder at previous time steps are concatenated after passing through a linear layer, and then the attention scale is obtained by passing through the tanh activation function. Will Input to the softmax layer, obtain the second word attention weights from the softmax layer output. Will We perform a weighted summation with the hidden states of the decoder's historical time steps to obtain the decoder's context vector.

[0062] In the fifth embodiment of this example, generating the text summary of the source text based on the encoder context vector, the decoder context vector, and the first word attention weight includes:

[0063] Step 51: Concatenate the summary word vectors corresponding to the summary text sequence with the encoding end context vector to obtain the decoding end input vector;

[0064] Step 52: Merge the hidden state and cell state of the current time step output by the decoder to obtain the merged hidden state;

[0065] The above implementation method can be used to combine the summary word vectors with the encoding context vectors. The input vector y at the decoder is obtained by concatenation. The hidden state dec_h and the cell state dec_c at the current time step output by the decoder are merged to obtain the merged hidden state.

[0066] Step 53: Merge the decoding context vector, the merged hidden state, and the decoding input vector to obtain the weight coefficient Pgen;

[0067] Step 54: Generate a text summary of the source text based on the Pgen and the first word attention weight.

[0068] Decode the context vector Hidden state after merging The input vector y at the decoding end is merged to obtain the weight coefficient Pgen. The weight coefficient Pgen is used to select either a word copied from the source text at the current time step as the prediction word, or a word generated from the vocabulary as the prediction word.

[0069] Specifically, generating a text summary of the source text based on the Pgen and the first word attention weight includes:

[0070] Step 541: Merge the encoding end context vector, the hidden state of the decoder at the current time step, and the decoding end input vector to obtain a real number vector;

[0071] Step 542: The real vector is converted into a score corresponding to each output word in the training vocabulary using a single linear connection layer, and the score is multiplied by Pgen to obtain the first distribution probability of the generated word in the training vocabulary.

[0072] Encoder context vector The hidden state dec_h at the current time step of the decoder and the input vector y from the decoder are concatenated to form a real-valued vector. A single linear connection layer is used, where a single linear connection layer converts the real-valued vector into a score corresponding to each output word in the training vocabulary. This score is then multiplied by the weight coefficient Pgen to obtain the first distribution probability of the generated word in the training vocabulary. In this embodiment, the encoder context vector is added to the decoder input, and the decoder input replaces the decoder context vector when inputting to the vocabulary distribution. This makes the model more focused on the original topic, ensuring that the generated summary does not deviate from the main theme.

[0073] In related technologies, two linear connection layers are used to reduce the number of parameters, but this also causes information loss. In this embodiment, the number of parameters is reduced by using a bidirectional GRU. The GRU consists of only update gates and reset gates, which greatly reduces the number of parameters and makes it easy to converge, thus greatly shortening the training time. At the same time, it reduces the requirements for computing resources and reduces resource costs. Therefore, only one linear layer is used before the distribution probability of generated words on the training vocabulary is obtained, which reduces information loss and improves the accuracy of generated summaries.

[0074] Step 543: Multiply the first word attention weight by the weight a to obtain the second distribution probability of the generated word in the input text, where a + Pgen = 1;

[0075] Step 544: Add the first distribution probability to the second distribution probability to obtain the third distribution probability of the generated word on the extended vocabulary, and output the sentence with the largest third distribution probability as the text summary of the source text.

[0076] Temporal attention networks assign attention weights to the first word at the encoding end. Multiplying by a weight 'a' yields the probability distribution of the generated word in the input text, where a = 1 - Pgen. Adding the two probability distributions gives the final probability distribution of the generated word on the expanded vocabulary. The sentence with the highest probability is output as the text summary. The encoder context vector is added to the decoder input, and the decoder input replaces the decoder context vector when inputting to the vocabulary distribution, making the model more focused on the original text's topic. Simultaneously, to better preserve vocabulary distribution information, one linear layer replaces two linear layers, reducing information loss and improving the accuracy of the generated summary.

[0077] In related technologies, Pointer Generator Networks (PGNs) are used to generate text summaries. The proposed PGN effectively solves the Out-of-Vocabulary (OOV) problem, and the Coverage repetition mechanism alleviates the redundancy problem in generative summaries. In its network architecture, the Encoder is a single-layer Bidirectional Long Short-Term Memory (Bi-LSTM) network, and the Decoder is a single-layer Unidirectional LSTM. In the decoding step, the hidden states of the Encoder and the output of the Decoder are weighted and summed to obtain the attention distribution. Finally, the PGN is used to determine the probability of copying words from the source text and generating words from the vocabulary. However, LSTMs consist of forget gates, update gates, and output gates, resulting in a large number of parameters and increased computational resource requirements. Furthermore, the attention mechanism only focuses on the current states of the Encoder and Decoder, causing the model to focus on the same parts of the input in different decoding steps, leading to the problem of repetitive generated summaries.

[0078] In this embodiment, the network architecture consists of a temporal attention network and an Encoder-Decoder joint. The temporal attention network records the hidden states of the Encoder and Decoder at each step, and calculates the attention weight of the current word using the attention weights from the previous input step, ensuring that different parts of the input are used, effectively solving the problem of duplicate summaries generated in the Pointer Generator. The Encoder is a single-layer Bidirectional Gated Recurrent Neural Network (Bi-GRU), consisting only of update and reset gates, significantly reducing the number of parameters and facilitating convergence, effectively saving computational resources and shortening training time. To better focus on the original text in the generated summary, an Encoder context vector is added to the Decoder input, and the Decoder input replaces the Decoder context vector when inputting to the vocabulary distribution. Simultaneously, to better preserve vocabulary distribution information, one linear layer replaces two linear layers, reducing information loss and improving the accuracy of the generated summary.

[0079] Model comparison experiments were conducted on the large-scale public text summarization microblog dataset LCSTS:

[0080] Model Encoder Decoder Rouge-1 Rouge-2 Rouge-3 Pointer Generator Bi-LSTM LSTM 25.62 10.85 23.97 Temporal attention model Bi-GRU LSTM 26.22 11.93 24.53

[0081] Rouge-1 calculates the overlap of each word between the machine summary obtained by the model and the manually defined standard reference summary; Rouge-2 calculates the overlap of bigrams between the machine summary and the reference summary; Rouge-3 calculates the overlap of the longest common subsequence between the machine summary and the reference summary. Rouge (Recall-Oriented Understudy for Gisting Evaluation) is a statistical metric for phrase co-occurrence. On the LCSTS dataset, a comparative experiment was conducted between the Pointer Generator model and the temporal attention model in this embodiment. The experimental data shows that the temporal attention model in this embodiment outperforms the Pointer Generator model in Rouge-1, Rouge-2, and Rouge-3 metrics. During the training process of 10,000 iterations, the convergence speed of the temporal attention model is significantly faster than that of the Pointer Generator model. Specifically, as shown below... Figure 4 As shown.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0083] Example 2

[0084] This embodiment also provides a text summarization generation apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0085] Figure 5 This is a structural block diagram of a text digest generation apparatus according to an embodiment of this application, such as... Figure 5 As shown, the device includes: a first acquisition module 61, a first calculation module 62, a second acquisition module 63, a second calculation module 64, and a generation module 65, wherein...

[0086] The first acquisition module 61 is used to acquire the semantic features corresponding to the main text sequence of the source text;

[0087] The first calculation module 62 is used to calculate the first word attention weight and the encoding context vector using the encoding end temporal attention network based on the semantic features.

[0088] The second acquisition module 63 is used to acquire the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text;

[0089] The second calculation module 64 is used to calculate the decoding context vector using a decoding attention network based on the hidden state of the current time step of the decoder and the hidden state of the historical time steps of the decoder.

[0090] The generation module 65 is used to generate a text summary of the source text based on the encoding end context vector, the decoding end context vector, and the first word attention weight.

[0091] Optionally, the first acquisition module includes a first acquisition unit, used to extract features from the main text sequence to obtain main text word vectors; input the main text word vectors into an encoder, the encoder being a single-layer bidirectional gated recurrent neural network Bi-GRU; and merge the main text word vectors and the historical output data of the encoder at the previous time step through the Bi-GRU to obtain the semantic features output by the encoder at the current time step.

[0092] Optionally, the first calculation module includes a first calculation unit, used to obtain the hidden state and unit state of the current time step output by the decoder, and merge the hidden state and the unit state to obtain a merged hidden state; concatenate the merged hidden state and the semantic features into the temporal attention network of the encoding end, and normalize the concatenated features using a temporal attention function to obtain the first word attention weight, wherein the temporal attention function is... t is the t-th time step, and EXP(·) is an exponential function with the natural constant e as the base; the first word attention weight and the semantic features are weighted and summed to obtain the encoding context vector.

[0093] Optionally, the second acquisition module includes a second acquisition unit, used to extract the built-in summary content of the source text and obtain a summary text sequence of the built-in summary content; perform feature extraction on the summary text sequence to obtain summary word vectors; concatenate the summary word vectors with the encoding end context vector to obtain the decoding end input vector; and input the decoding end input vector and the historical output data of the decoder at the previous time step into the decoder to obtain the hidden state of the decoder at the current time step.

[0094] Optionally, the second calculation module includes a second calculation unit, used to input the hidden state of the current time step of the decoder and the hidden state of the historical time step of the decoder into the decoding end attention network for concatenation, obtain the attention scale through a nonlinear activation function, input the attention scale into a Softmax layer to obtain the second word attention weight, and perform a weighted summation of the second word attention weight and the hidden state of the historical time step of the decoder to obtain the decoding end context vector.

[0095] Optionally, the generation module includes a first generation unit, configured to concatenate the summary word vector corresponding to the summary text sequence with the encoding end context vector to obtain the decoding end input vector; merge the hidden state and unit state of the current time step output by the decoder to obtain the merged hidden state; merge the decoding end context vector, the merged hidden state, and the decoding end input vector to obtain the weight coefficient Pgen; and generate the text summary of the source text based on Pgen and the first word attention weight.

[0096] Optionally, the generation module includes a second generation unit, used to merge the encoding end context vector, the hidden state of the decoder at the current time step, and the decoding end input vector to obtain a real number vector; use a single linear connection layer to convert the real number vector into the score corresponding to each output word in the training vocabulary, and multiply the score by Pgen to obtain the first distribution probability of the generated word on the training vocabulary; multiply the first word attention weight by weight a to obtain the second distribution probability of the generated word in the input text, where a + Pgen = 1; add the first distribution probability and the second distribution probability to obtain the third distribution probability of the generated word on the expanded vocabulary, and output the sentence with the largest third distribution probability as the text summary of the source text.

[0097] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0098] Example 3

[0099] Embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0100] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0101] S1, Obtain the semantic features corresponding to the main text sequence of the source text;

[0102] S2, based on the semantic features, the first word attention weight and the encoding context vector are calculated using the temporal attention network at the encoding end;

[0103] S3, obtain the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text;

[0104] S4. Based on the hidden state of the current time step of the decoder and the hidden state of the historical time steps of the decoder, the decoder attention network is used to calculate the decoder context vector.

[0105] S5, generate a text summary of the source text based on the encoding end context vector, the decoding end context vector, and the first word attention weight.

[0106] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0107] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0108] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0109] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0110] S1, Obtain the semantic features corresponding to the main text sequence of the source text;

[0111] S2, based on the semantic features, the first word attention weight and the encoding context vector are calculated using the temporal attention network at the encoding end;

[0112] S3, obtain the hidden state of the decoder at the current time step corresponding to the summary text sequence of the source text;

[0113] S4. Based on the hidden state of the current time step of the decoder and the hidden state of the historical time steps of the decoder, the decoder attention network is used to calculate the decoder context vector.

[0114] S5, generate a text summary of the source text based on the encoding end context vector, the decoding end context vector, and the first word attention weight.

[0115] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0116] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0122] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating a text summary, characterized by, The method comprises: obtaining semantic features corresponding to a text sequence of a source text; according to the semantic features, calculating first word attention weights and an encoder context vector by using a time attention network at the encoding end; obtaining a hidden state of a current time step of a decoder corresponding to a summary text sequence of the source text; according to the hidden state of the current time step of the decoder and a hidden state of a historical time step of the decoder, calculating a decoder context vector by using a decoding end attention network; generating a text summary of the source text according to the encoder context vector, the decoder context vector and the first word attention weights; generating a text summary of the source text according to the encoder context vector, the decoder context vector and the first word attention weights comprises: concatenating a summary word vector corresponding to the summary text sequence and the encoder context vector to obtain a decoding end input vector; merging a hidden state and a cell state of a current time step of a decoder output to obtain a merged hidden state; merging the decoder context vector, the merged hidden state and the decoding end input vector to obtain a weight coefficient Pgen; generating a text summary of the source text according to the Pgen and the first word attention weights.

2. The method of claim 1, wherein, obtaining semantic features corresponding to a text sequence of a source text comprises: performing feature extraction on the text sequence to obtain text word vectors; inputting the text word vectors into an encoder, the encoder being a single-layer bidirectional gated recurrent neural network Bi-GRU; merging the text word vectors and historical output data of a previous time step of the encoder by using the Bi-GRU to obtain semantic features output by the encoder at a current time step.

3. The method of claim 1, wherein, according to the semantic features, calculating first word attention weights and an encoder context vector by using a time attention network at the encoding end comprises: obtaining a hidden state and a cell state of a current time step of a decoder output, and merging the hidden state and the cell state to obtain a merged hidden state; The merged hidden state and the semantic feature are input into a time attention network at an encoding end to be spliced, and a time attention function is used to normalize the spliced feature to obtain a first word attention weight, wherein the time attention function is , t is a t-th time step, and EXP(·) is an exponential function with a natural constant e as a base. performing weighted summation on the first word attention weights and the semantic features to obtain an encoder context vector.

4. The method of claim 1, wherein, obtaining a hidden state of a current time step of a decoder corresponding to a summary text sequence of the source text comprises: extracting self-contained summary content of the source text to obtain a summary text sequence of the self-contained summary content; performing feature extraction on the summary text sequence to obtain summary word vectors; concatenating the summary word vectors and the encoder context vector to obtain a decoding end input vector; inputting the decoding end input vector and historical output data of a previous time step of a decoder into the decoder to obtain a hidden state of a current time step of the decoder.

5. The method of claim 1, wherein, according to the hidden state of the current time step of the decoder and a hidden state of a historical time step of the decoder, calculating a decoder context vector by using a decoding end attention network comprises: The hidden state of the current time step of the decoder and the hidden state of the historical time step of the decoder are input into a decoding attention network respectively, spliced, and then an attention scale is obtained through a nonlinear activation function, and the attention scale is input into a Softmax layer to obtain a second word attention weight; The second word attention weight is weighted and summed with the hidden state of the historical time step of the decoder to obtain a decoding context vector.

6. The method of claim 1, wherein, Generating the text summary of the source text according to the Pgen and the first word attention weight comprises: The encoding context vector, the hidden state of the current time step of the decoder, and the decoding input vector are merged to obtain a real vector; The real vector is converted into a score corresponding to each output word in a training vocabulary by using a single linear connection layer, and the score is multiplied by the Pgen to obtain a first distribution probability of the generated word on the training vocabulary; The first word attention weight is multiplied by a weight a to obtain a second distribution probability of the generated word in the input text, wherein a+Pgen=1; The first distribution probability and the second distribution probability are added to obtain a third distribution probability of the generated word on an extended vocabulary, and the sentence with the maximum third distribution probability is output as the text summary of the source text.

7. An apparatus for generating a text summary, characterized by Comprise: The first acquisition module is used for acquiring semantic features corresponding to a body text sequence of a source text; The first calculation module is used for calculating a first word attention weight and an encoding context vector by using a time attention network of an encoding end according to the semantic features; The second acquisition module is used for acquiring a hidden state of a current time step of a decoder corresponding to an abstract text sequence of the source text; The second calculation module is used for calculating a decoding context vector by using a decoding attention network according to the hidden state of the current time step of the decoder and a hidden state of a historical time step of the decoder; The generation module is used for generating a text summary of the source text according to the encoding context vector, the decoding context vector, and the first word attention weight; Generating the text summary of the source text according to the Pgen and the first word attention weight comprises: The abstract word vector corresponding to the abstract text sequence is spliced with the encoding context vector to obtain a decoding input vector; The hidden state and the cell state of the current time step output by the decoder are merged to obtain a merged hidden state; The decoding context vector, the merged hidden state, and the decoding input vector are merged to obtain a weight coefficient Pgen; Generating the text summary of the source text according to the Pgen and the first word attention weight.

8. An electronic device, comprising: Comprise a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; wherein: The memory is used for storing a computer program; The processor is used for executing the method steps of any one of claims 1 to 6 by running the program stored on the memory.

9. A storage medium, characterized by The storage medium comprises a stored program, wherein the program, when executed, performs the method steps of any one of claims 1 to 6.

Images