A text abstract generation method, device, system and storage medium

Abstract

The application provides a text abstract generation method, device and system and a storage medium, and belongs to the field of language processing.The method comprises the following steps: preprocessing an original text dataset to obtain title word data and body word data; performing keyword analysis on the title word data and all the body word data by using a trained model to obtain keyword vectors, body word vectors and hidden state data; and performing analysis on the keyword vectors, body word vectors and hidden state data to obtain a text abstract generation result.The application can pay more attention to important information in the original text, can ignore other redundant information, can make the generated abstract cover the key content of the original text to the greatest extent, can reduce the loss of useful information, can fully play the strong semantic information of the document theme and the context information of the document, and thus a text abstract with better quality can be generated.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of language processing, and particularly relates to a text abstract generation method, device, system and storage medium. BACKGROUND

[0002] The existing Chinese text abstract generation technology still has many deficiencies to be improved and solved. At present, generating a Chinese abstract will face the problems of out-of-vocabulary (OOV) and low quality of generated abstract, and most generative models will ignore the important clues provided by the theme word information and keyword information in the process of generating an abstract, thereby causing the loss of keyword information. SUMMARY

[0003] The present application mainly relates to the technical field of language processing, and particularly relates to a text abstract generation method, device, system and storage medium.

[0004] The technical scheme for solving the above technical problems is as follows: a text abstract generation method, comprising the following steps:

[0005] Importing an original text data set, preprocessing the original text data set to obtain a plurality of title word data and a plurality of body word data;

[0006] Constructing a training model, performing keyword analysis on all the title word data and all the body word data through the training model to obtain a keyword vector, a body word vector corresponding to each body word data, and hidden state data corresponding to each body word data;

[0007] Performing analysis on the keyword vector, all the body word vectors and all the hidden state data to obtain a text abstract generation result.

[0008] Another technical scheme for solving the above technical problems is as follows: a text abstract generation device, comprising:

[0009] An importing module for importing an original text data set;

[0010] A preprocessing module for preprocessing the original text data set to obtain a plurality of title word data and a plurality of body word data;

[0011] An analysis module for constructing a training model, performing keyword analysis on all the title word data and all the body word data through the training model to obtain a keyword vector, a body word vector corresponding to each body word data, and hidden state data corresponding to each body word data;

[0012] The abstract generation result obtaining module is configured to analyze the keyword vector, all the text word vectors and all the hidden state data to obtain a text abstract generation result.

[0013] Based on the above-mentioned text abstract generation method, the application further provides a text abstract generation system.

[0014] Another technical solution of the application to solve the above technical problems is as follows: a text abstract generation system, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the text abstract generation method is realized.

[0015] Based on the above-mentioned text abstract generation method, the application further provides a computer readable storage medium.

[0016] Another technical solution of the application to solve the above technical problems is as follows: a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the text abstract generation method is realized.

[0017] The application has the following beneficial effects: the title word data and the text word data are obtained by preprocessing the original text data set, the keyword vector, the text word vector and the hidden state data are obtained by training the model to analyze the keywords of the title word data and the text word data, the text abstract generation result is obtained by analyzing the prediction word probability of the keyword vector, the text word vector and the hidden state data, the important information in the original text can be paid more attention to, other redundant information can be ignored, the generated abstract can cover the key content of the original text to the greatest extent, the loss of useful information is reduced, the strong semantic information of the document theme and the context information of the document are fully utilized, and thus a text abstract with better quality is generated. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a text abstract generation method provided by the embodiment of the application is shown.

[0019] Figure 2 A module block diagram of a text abstract generation device provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0020] The principles and characteristics of the application are described below in combination with the drawings, and the examples are only used to explain the application and not to limit the scope of the application.

[0021] Figure 1 A flowchart of a text abstract generation method provided by the embodiment of the application is shown.

[0022] As shown in Figure 1 A text abstract generation method, comprising the following steps:

[0023] Importing an original text data set, preprocessing the original text data set to obtain a plurality of title word data and a plurality of body word data;

[0024] Building a training model, performing keyword analysis on all the title word data and all the body word data through the training model to obtain keyword vectors, body word vectors corresponding to each body word data, and hidden state data corresponding to each body word data;

[0025] Performing analysis of the prediction word probability on the keyword vectors, all the body word vectors, and all the hidden state data to obtain a text abstract generation result.

[0026] It should be understood that the original text data set includes title data and body data.

[0027] In the above embodiment, the title word data and the body word data are obtained by preprocessing the original text data set, the keyword vectors, the body word vectors, and the hidden state data are obtained by keyword analysis of the title word data and the body word data through the training model, and the text abstract generation result is obtained by prediction word probability analysis of the keyword vectors, the body word vectors, and the hidden state data. The important information in the original text can be paid more attention to, and other redundant information can be ignored, so that the generated abstract can cover the key content of the original text to the greatest extent, the loss of useful information is reduced, the strong semantic information of the document theme and the context information of the document are fully utilized, and thus a better text abstract is generated.

[0028] Optionally, as an embodiment of the present application, the preprocessing of the original text data set to obtain a plurality of title word data and a plurality of body word data comprises:

[0029] The original text data set is segmented by using the jieba Chinese segmentation library to obtain a plurality of title word data and a plurality of body word data.

[0030] It should be understood that the jieba Chinese word segmentation library is a popular Chinese word segmentation library for segmenting Chinese text into individual words. It is an open source project with easy-to-use and high-performance features, widely used in Chinese natural language processing tasks. In short, jieba is a powerful Chinese word segmentation library that provides simple and easy-to-use interfaces and multiple segmentation modes, and can play an important role in Chinese text processing. Whether in information retrieval, text classification, sentiment analysis and other tasks, jieba can provide reliable solutions for Chinese text segmentation needs.

[0031] It should be understood that the original data set is first preprocessed, the data set (i.e., the original text data set) is segmented by the python function package jieba (i.e., the jieba Chinese word segmentation library), the words in the sentence are separated by spaces, and a dictionary file is established.

[0032] In the above embodiment, the jieba Chinese word segmentation library is used to segment the original text data set, obtaining a plurality of title word data and a plurality of body word data, providing a reliable solution for Chinese text segmentation needs, and playing an important role in Chinese text processing.

[0033] Optionally, as an embodiment of the present application, the training model includes a topic extraction neural network, an encoder, and a decoder,

[0034] The process of performing keyword analysis on all the title word data and all the body word data by the training model to obtain keyword vectors, body word vectors corresponding to each of the body word data, and hidden state data corresponding to each of the body word data includes:

[0035] Performing feature extraction on each of the title word data and each of the body word data by a preset BERT model to obtain title word vectors corresponding to each of the title word data and body word vectors corresponding to each of the body word data;

[0036] Performing topic analysis on each of the body word vectors by the topic extraction neural network to obtain topic mixed vectors corresponding to each of the body word data and hidden state data corresponding to each of the body word data;

[0037] Performing encoding analysis on each of the title word vectors and all the body word vectors by the encoder to obtain target context vectors corresponding to each of the body word data;

[0038] decoding each of the topic mixture vectors and the target context vectors corresponding to each of the text word data by the decoder to obtain keyword generation probabilities corresponding to each of the text word data;

[0039] filtering maximum values of all the keyword generation probabilities to obtain a maximum keyword generation probability after filtering, and taking a text word vector corresponding to the maximum keyword generation probability as a keyword vector.

[0040] It should be understood that the model initial parameters are set, and the syntax filter, the forward and gradient back propagation functions are constructed.

[0041] It should be understood that the preset BERT model, i.e., BERT (Bidirectional Encoder Representations from Transformers), is a pre-training language model based on the Transformer architecture, which was proposed by Google in 2018. The goal of BERT is to learn a general language representation by unsupervised pre-training on a large-scale text corpus, thereby providing rich semantic understanding capabilities. The BERT model adopts a bidirectional training method, which is different from the traditional left-to-right or right-to-left unidirectional model. This means that during the pre-training process, BERT can simultaneously utilize the left and right context information of a word for modeling, thereby better capturing the semantic and contextual relationships of words.

[0042] Specifically, the pre-training model BERT (i.e., the preset BERT model) is imported for training, and a keyword generation module based on topic perception and title-oriented fusion is used to extract keywords, which includes a topic extraction neural module (i.e., the topic extraction neural network), a title-oriented hierarchical encoder (i.e., the encoder), and a keyword decoder (i.e., the decoder).

[0043] In the above embodiment, the title word data and the text word data are subjected to keyword analysis by the trained model to obtain keyword vectors, text word vectors, and hidden state data, which can better capture the semantic and contextual relationships of words, so that the generated summary maximally covers the key content of the original text, reduces the loss of useful information, fully utilizes the strong semantic information of the document topic and the contextual information of the document, and thus generates a better text summary.

[0044] Optionally, as an embodiment of the present application, the topic extraction neural network includes a neural topic network and a BoW decoder,

[0045] The process of subject analysis on each of the text word vectors by the subject extraction neural network respectively, to obtain the subject mixture vector corresponding to each of the text word data and the hidden state data corresponding to each of the text word data, comprises:

[0046] The process of feature extraction on each of the text word vectors by the neural subject network respectively, to obtain the first prior variable corresponding to each of the text word data and the second prior variable corresponding to each of the text word data;

[0047] The process of decoding on each of the first prior variable and the second prior variable corresponding to each of the text word data by the BoW decoder respectively, to obtain the Gaussian vector corresponding to each of the text word data, the encoder hidden state corresponding to each of the text word data and the decoder hidden state corresponding to each of the text word data, wherein the hidden state data comprises the encoder hidden state and the decoder hidden state;

[0048] The process of normalization on each of the Gaussian vectors by the first formula, to obtain the subject mixture vector corresponding to each of the text word data, wherein the first formula is:

[0049]

[0050] wherein θ i is the subject mixture vector corresponding to the i-th text word data, W θ is a learnable parameter, Z i is the Gaussian vector corresponding to the i-th text word data, and softmax() is a softmax function.

[0051] It should be understood that the BoW decoder, i.e. the BoW (Bag-of-Words) encoder and decoder, is a text feature representation method for converting text into numerical vectors and performing encoding and decoding processes. The encoding process converts text into vector representation, while the decoding process restores vector representation to the original text.

[0052] Specifically, each document (i.e. the text word data) is processed into a bag-of-words vector X b (the text word vector), and X b (the text word vector) is input into the neural subject network, and the prior variables μ and σ are estimated by the BoW encoder, and the formula is as follows:

[0053] μ = ∫ μ (∫ e (X b )), and logσ = ∫ σ (∫ e (Xb )),

[0054] where * (.) is a neural perceptron with a RuLU activation function, followed by a BoW decoder, and the prior variables mu and sigma are used for the topic representation Z ~ N(mu, sigma 2 ), and the softmax function is used to construct the topic mixture theta, and the theta vector is used as the document topic representation to guide the keyword generation as follows:

[0055] theta = softmax(w θ Z),

[0056] w θ is a learnable parameter.

[0057] In the above embodiment, the topic extraction neural network is used to perform topic analysis on the body word vectors, to obtain the topic mixture vector and the hidden state data, which can better capture the semantic and contextual relationship of the words, so that the generated summary can cover the key content of the original text to the greatest extent, and the loss of useful information is reduced.

[0058] Optionally, as an embodiment of the present application, the encoder comprises a sequence encoding layer, a matching layer and a merging layer,

[0059] The process of encoding and analyzing each of the title word vectors and all the body word vectors through the encoder to obtain the target context vector corresponding to each of the body word data comprises:

[0060] Each of the title word vectors and each of the body word vectors is mapped through a preset vector table to obtain a title dense vector corresponding to each of the title word data and a body dense vector corresponding to each of the body word data.

[0061] Each of the title dense vectors is encoded through the sequence encoding layer to obtain an initial title context vector corresponding to each of the title word data, specifically:

[0062] Each of the title dense vectors is encoded through a second formula to obtain an initial title context vector corresponding to each of the title word data, and the second formula is:

[0063]

[0064] wherein,

[0065] wherein, v j is the initial title context vector corresponding to the jth title word data, a title first hidden vector corresponding to the jth title word data, a title second hidden vector corresponding to the jth title word data, j ∈ [1, L], L being a total number of title word data, GRU() is a bidirectional gate recurrent unit, t j a title dense vector corresponding to the jth title word data, a title first hidden vector corresponding to the j-1th title word data, a title second hidden vector corresponding to the j+1th title word data;

[0066] each of the text dense vectors is encoded by the sequence encoding layer to obtain an initial text context vector corresponding to each of the text word data, specifically:

[0067] each of the text dense vectors is encoded by a third formula to obtain an initial text context vector corresponding to each of the text word data, the third formula being:

[0068]

[0069] wherein,

[0070] wherein, u i an initial text context vector corresponding to the ith text word data, a text first hidden vector corresponding to the ith text word data, a text second hidden vector corresponding to the ith text word data, i ∈ [1, N], N being a total number of text word data, GRU() is a bidirectional gate recurrent unit, x i a text dense vector corresponding to the ith text word data, a text first hidden vector corresponding to the i-1th text word data, a text second hidden vector corresponding to the i+1th text word data;

[0071] each of the initial text context vectors and all the initial title context vectors is matched by the matching layer to obtain an aggregated information vector corresponding to each of the text word data, specifically:

[0072] each of the initial text context vectors and all the initial title context vectors is matched by a fourth formula to obtain an aggregated information vector corresponding to each of the text word data, the fourth formula being:

[0073]

[0074] wherein,

[0075] wherein, c i is the aggregated information vector corresponding to the i-th body word data, a i,j is the normalized attention between the initial body context vector corresponding to the i-th body word data and the initial title context vector corresponding to the j-th title word data, v j is the initial title context vector corresponding to the j-th title word data, L is the total number of title word data, u i is the initial body context vector corresponding to the i-th body word data, N is the total number of body word data, and W is a learnable parameter;

[0076] The merging layer merges each of the initial body context vectors and the aggregated information vectors corresponding to each of the body word data to obtain target context vectors corresponding to each of the body word data, specifically by:

[0077] The merging layer merges each of the initial body context vectors and the aggregated information vectors corresponding to each of the body word data to obtain target context vectors corresponding to each of the body word data, specifically by:

[0078]

[0079] wherein,

[0080] wherein, is the target context vector corresponding to the i-th body word data, a is a hyperparameter, is the target context first hidden vector corresponding to the i-th body word data, is the target context second hidden vector corresponding to the i-th body word data, and GRU() is a bidirectional gate recurrent unit, i is the aggregated information vector corresponding to the i-th body word data, u i is the initial body context vector corresponding to the i-th body word data, is the target context first hidden vector corresponding to the i-1-th body word data, is the target context second hidden vector corresponding to the i+1-th body word data.

[0081] It should be understood that the title-oriented hierarchical encoder (i.e., the encoder) is composed of the sequence encoding layer, the matching layer, and the merging layer.

[0082] In particular, the title input (i.e., the title word vectors) and the body content input (i.e., the body word vectors) are read and their contextual representations are learned separately, first a vector table maps each word in the title and body into a d e dimensional dense vector (i.e., the title dense vector), then two bidirectional gated recurrent units (GRUs) are used to encode the context and title respectively and combine the contextual information into the representation of each word, formulated as follows:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] x i is the i-th word vector in the corpus document (i.e., the body dense vector), t j is the j-th word vector in the title (i.e., the title dense vector), u i and v j are the contextual vectors for the i-th document word and j-th title word (i.e., the initial body contextual vector and the initial title contextual vector), are hidden vectors (i.e., the body first hidden vector, the body second hidden vector, the title first hidden vector, and the title second hidden vector), → denotes the right encoding direction, and ← denotes the left encoding direction.

[0090] In particular, the attention-based matching layer is used to aggregate relevant information in each word in the context title, the matching layer is also composed of two parts. One part is the self-matching of the title to the title, and the other part is the matching from the body to the title, formulated as follows:

[0091]

[0092]

[0093] c i is the aggregated information vector for the i-th word of the document x, a i,j is the normalized attention between u i and v j , and w1 is a learnable parameter.

[0094] Specifically, the document topic representation θ (i.e., the topic mixture vector) and the title-oriented context representation set M (i.e., the target context vector) are input into the decoder to obtain the generation probability of the keyword Y (i.e., the keyword generation probability), and the formula is as follows: i i The title-oriented context representation set M (i.e., the target context vector) is obtained by inputting the initial text context vector u (i.e., the initial text context vector) and the aggregation information vector c

[0095]

[0096]

[0097]

[0098]

[0099] α is a corresponding hyperparameter.

[0100] In the above embodiment, the target context vector is obtained by encoding and analyzing each title word vector and all text word vectors by the encoder, other redundant information can be ignored, the generated summary can cover the key content of the original text to the greatest extent, the loss of useful information is reduced, the strong semantic information of the document topic and the context information of the document are fully utilized, and thus a better text summary is generated.

[0101] Optionally, as one embodiment of the present application, the process of obtaining the keyword generation probability corresponding to each text word data by decoding each topic mixture vector and the target context vector corresponding to each text word data by the decoder includes:

[0102] The keyword generation probability corresponding to each text word data is obtained by decoding each topic mixture vector and the target context vector corresponding to each text word data by the sixth formula, and the sixth formula is as follows:

[0103]

[0104] Wherein, P(Y i ) is the keyword generation probability corresponding to the i-th text word data, is the target context vector corresponding to the i-th text word data, θ i is the topic mixture vector corresponding to the i-th text word data.

[0105] Specifically, the document topic representation θ (i.e., the topic mixture vector) and the title-oriented context representation set M (i.e., the target context vector) are input into the decoder to obtain the generation probability of the keyword Y (i.e., the keyword generation probability), and the formula is as follows: i ​​

[0106]

[0107] In the above embodiments, the sixth method is used to decode the mixed vector of each topic and the target context vector to obtain the keyword generation probability, which reduces the loss of useful information and gives full play to the strong semantic information of the document topic and the context information of the document, thereby generating a better quality text summary.

[0108] Optionally, as an embodiment of the present invention, the hidden state data includes the encoder hidden state and the decoder hidden state.

[0109] The process of analyzing the keyword vectors, all the text word vectors, and all the hidden state data to predict word probabilities and obtain the text summary generation result includes:

[0110] The attention score for each of the text words is obtained by calculating the attention score of the keyword vector and the encoder hidden state corresponding to each of the text words using the seventh formula. The seventh formula is:

[0111] a i =Softmax[V T tanh(W h h i +W s Y+b attn )],

[0112] Among them, a i V and W represent the keyword attention scores corresponding to the i-th word in the main text. h W s and b attn All are learnable parameters, h i Y represents the encoder hidden state corresponding to the i-th word in the main text, and Y is the keyword vector.

[0113] The keyword extraction probability is obtained by calculating the probability of attention scores for all the aforementioned keywords using the eighth formula:

[0114] P c (w)=∑a i ,

[0115] Among them, P c (w) represents the keyword extraction probability, a i The keyword attention score corresponding to the i-th word in the main text;

[0116] The ninth formula is used to calculate the probability of the keyword vector, the encoder hidden state corresponding to each text word data, and the keyword attention score corresponding to each text word data, to obtain the gating selection probability corresponding to each text word data, and the ninth formula is:

[0117]

[0118] wherein,

[0119] wherein, is the gating selection probability corresponding to the ith text word data, is the semantic vector corresponding to the ith text word data, σ, w Y , w X , and b ptr are all learnable parameters, Y is the keyword vector, X bi is the text word vector corresponding to the ith text word data, a i is the keyword attention score corresponding to the ith text word data, h i is the encoder hidden state corresponding to the ith text word data;

[0120] The tenth formula is used to calculate the probability of the encoder hidden state corresponding to each text word data, the keyword attention score corresponding to each text word data, and the decoder hidden state corresponding to each text word data, to obtain the first word extraction probability corresponding to each text word data, and the tenth formula is:

[0121]

[0122] wherein,

[0123] wherein, is the first word extraction probability corresponding to the ith text word data, V', V, b, and b' are all learnable parameters, s i is the decoder hidden state corresponding to the ith text word data, is the semantic vector corresponding to the ith text word data, a i is the keyword attention score corresponding to the ith text word data, h i is the encoder hidden state corresponding to the ith text word data;

[0124] The eleventh formula is used to calculate the probability of all the encoder hidden states and all the text word vectors, to obtain the second word extraction probability, and the eleventh formula is:

[0125] Ps (w) =∑b i ,

[0126] wherein, b i = Softmax[V T tanh(W h h i + W s X bi + b attn )],

[0127] wherein, P s (w) is the second word extraction probability, b i is the text attention score corresponding to the i-th text word data, W h , V, W s and b attn are all learnable parameters, X bi is the text word vector corresponding to the i-th text word data, h i is the encoder hidden state corresponding to the i-th text word data;

[0128] The target word prediction probability corresponding to each text word data is obtained by calculating the probabilities of the second word extraction probability, the keyword extraction probability, the gate selection probability corresponding to each text word data, and the first word extraction probability corresponding to each text word data through the twelfth formula, and all the target word prediction probabilities are taken as the text summary generation result, and the twelfth formula is:

[0129]

[0130] wherein, P i (w) is the target word prediction probability corresponding to the i-th text word data, P s (w) is the second word extraction probability, is the first word extraction probability corresponding to the i-th text word data, is the gate selection probability corresponding to the i-th text word data, and P c (w) is the keyword extraction probability.

[0131] It should be understood that the double-pointer generation network is introduced, the pointer network points the unrecognizable named entity to the corresponding position in the original text, solves the OOV problem, and a pointer for copying keyword information is newly added on the basis of the original pointer.

[0132] Specifically, the information of the current keyword (i.e., the keyword vector) and the hidden layer state of the encoder (i.e., the encoder hidden state) output by the keyword generation module are used to calculate the attention score a iThe calculation of the attention score of the current keyword on the original word (i.e., the keyword attention score) is calculated using the following formula:

[0133] a i = Softmax [V T tanh (W h h i + W s Y i + b attn )

[0134] V, W h , W S , b attn are learnable parameters, Y i is the current keyword information output by the keyword generation module (i.e., the keyword vector), h i is the hidden layer state of the encoder in the pre-trained model (i.e., the encoder hidden state).

[0135] It should be understood that the extraction probability P c (ω) on the keyword (i.e., the keyword extraction probability) is calculated by the attention score on the keyword (i.e., the keyword attention score), as follows:

[0136] P c (w) = ∑a i .

[0137] Specifically, the formula for the final predicted word distribution probability (i.e., the target word prediction probability) is:

[0138]

[0139] where p c (w) is the probability of extracting a word from the keyword (i.e., the keyword extraction probability),

[0140] P vocab (w) represents the probability of extracting a word from the table (i.e., the first word extraction probability), which is calculated as follows:

[0141]

[0142] V, V', b, b' are learnable parameters, s i is the decoder hidden state, is a semantic vector generated by the attention score:

[0143] p s (w) is the probability of extracting a word from the original text (i.e., the second word extraction probability), and the calculation method is the same as p c(w) is as follows:

[0144] b i = Softmax [V T tanh (W h h i + W s X b + b attn )],

[0145] P s (w) = ∑b i .

[0146] By p gen (that is, the gating selection probability) to determine whether to generate a word on the vocabulary or copy a word from the original text and the keywords, the purpose of copying a word from the keywords and the original text is achieved, the pointer efficiency is improved, and the formula is as follows:

[0147]

[0148] Where σ、 w Y , w X , b ptr are parameters that can be learned by the model.

[0149] In the above embodiment, the keyword vector, all text word vectors and all hidden state data are analyzed to obtain the word probability, the text summary generation result is obtained, the purpose of copying a word from the keywords and the original text is achieved, the pointer efficiency is improved, the important information in the original text can be paid more attention to, other redundant information can be ignored, the generated summary can cover the key content of the original text to the greatest extent, the loss of useful information is reduced, the strong semantic information of the document theme and the context information of the document are fully utilized, and therefore a better text summary is generated.

[0150] Optionally, as another embodiment of the application, the method of the application comprises:

[0151] First, the original data set is preprocessed, the keyword generation module is constructed to extract keywords, so that the generated keywords can contain more original text information and title information, the initial parameters of the model are set, the double-pointer network is introduced to solve the OOV problem and better mine the keyword information, the forward and gradient back propagation functions are constructed, the text summary generation model is trained, and the output summary is evaluated using evaluation indicators.

[0152] Optionally, as another embodiment of the application, the application mainly constructs a keyword generation module, enriches semantic features by mining potential topics, and also uses highly summarized and valuable information in the title to jointly guide the generation of keywords, so that the strong semantic information of the document topic and the context information of the document are fully utilized, a better quality text summary is generated, the pointer network can solve the OOV problem, but the single pointer cannot fully utilize the keyword information, and in order to realize copying of words from keyword information, a double-pointer network is used.

[0153] Optionally, as another embodiment of the application, the application constructs a keyword generation module based on topic perception and title-oriented fusion, including a neural topic model, a title-oriented hierarchical encoder and a double-pointer topic perception sequence decoder. In the encoding stage, the semantic guidance of the title words is integrated into the matching layer, and the attention mechanism is used to aggregate the title information of each word in the document content. In addition, in the decoding process, the results of the trained neural topic model are used to improve the decoder with topic perception capability to generate keywords with topic sensitivity.

[0154] Optionally, as another embodiment of the application, the pointer network of the application can solve the OOV problem, but the single pointer cannot fully utilize the keyword information, in order to realize copying of words from keyword information, a keyword information pointer is copied on the basis of the single-pointer network to form a double-pointer network, and the keyword information is better mined.

[0155] Optionally, as another embodiment of the application, the technical problems to be solved by the application are:

[0156] Currently, generating a Chinese abstract will face the problems of out-of-vocabulary (OOV) and low quality of generated abstracts.

[0157] The generative model ignores the important clues provided by the keyword information in the process of generating the abstract, resulting in the loss of keyword information.

[0158] The technical method for solving the above problems is as follows:

[0159] For the problem of out-of-vocabulary (OOV), most named entities are not included in the dictionary when constructing the dictionary, and the named entities are endless, so it is impossible to include all of them. Therefore, the pointer generation network is used to solve the problem by pointing to the corresponding position of the word in the original text when encountering a word that cannot be distinguished by the dictionary.

[0160] The keyword generation module based on the theme perception and title-oriented fusion is constructed, potential themes are mined to enrich semantic features, and the highly summarized and valuable information in the title is used to jointly guide the generation of keywords, so that the strong semantic information of the document theme and the context information of the document are fully utilized, and a better quality text summary is generated.

[0161] A keyword information pointer is copied on the basis of the single-pointer network to form a double-pointer network, so that the keyword information is better mined.

[0162] Optionally, as another embodiment of the present application, the output summary is evaluated by the ROUGE evaluation index.

[0163] Figure 2 A module block diagram of a text summary generation device provided for an embodiment of the present application.

[0164] Optionally, as another embodiment of the present application, as shown in Figure 2 A text summary generation device includes:

[0165] An import module is configured to import an original text data set.

[0166] A preprocessing module is configured to preprocess the original text data set to obtain a plurality of title word data and a plurality of body word data.

[0167] An analysis module is configured to construct a training model, perform keyword analysis on all the title word data and all the body word data through the training model, and obtain a keyword vector, a body word vector corresponding to each body word data, and hidden state data corresponding to each body word data.

[0168] A summary generation result obtaining module is configured to analyze the keyword vector, all the body word vectors, and all the hidden state data to obtain a text summary generation result.

[0169] Optionally, another embodiment of the present application provides a text summary generation system, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the text summary generation method as described above is implemented. The system can be a computer system.

[0170] Optionally, another embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the text summary generation method as described above is implemented.

[0171] It should be noted that, in the present document, the terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a list of elements does not only include those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or apparatus.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0173] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0174] The unit described as a separate component can or can not be physically separated, and the component displayed as a unit can or can not be a physical unit, that is, it can be located in one place, or it can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0175] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0176] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0177] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A text summarization method, characterized in that, Includes the following steps: Import the original text dataset, preprocess the original text dataset to obtain multiple title word data and multiple body text word data; A training model is constructed, and keyword analysis is performed on all the title word data and all the body text word data through the training model to obtain keyword vectors, body text word vectors corresponding to each of the body text word data, and hidden state data corresponding to each of the body text word data. The keyword vectors, all the text word vectors, and all the hidden state data are analyzed to predict word probabilities, and the text summary generation results are obtained. The training model includes a topic extraction neural network, an encoder, and a decoder. The process of performing keyword analysis on all title word data and all body word data using the training model to obtain keyword vectors, body word vectors corresponding to each body word data, and hidden state data corresponding to each body word data includes: By using a preset BERT model, features are extracted from each of the title word data and each of the body text word data to obtain title word vectors corresponding to each of the title word data and body text word vectors corresponding to each of the body text word data. The topic extraction neural network is used to perform topic analysis on each of the main text word vectors to obtain topic mixing vectors corresponding to each of the main text word data and hidden state data corresponding to each of the main text word data; The encoder is used to encode and analyze each of the title word vectors and all the body text word vectors to obtain the target context vector corresponding to each of the body text word data. The decoder decodes each of the topic mixing vectors and the target context vectors corresponding to each of the main text word data to obtain the keyword generation probability corresponding to each of the main text word data. The maximum value of the generation probability of all the keywords is selected, and the maximum keyword generation probability is obtained. The word vector of the main text corresponding to the maximum keyword generation probability is used as the keyword vector.

2. The text summarization method according to claim 1, characterized in that, The process of preprocessing the original text dataset to obtain multiple title word data and multiple body text word data includes: The original text dataset was segmented using the jieba Chinese word segmentation library to obtain multiple title word data and multiple body text word data.

3. The text summarization method according to claim 1, characterized in that, The topic extraction neural network includes a neural topic network and a BoW decoder. The process of performing topic analysis on each of the text word vectors using the topic extraction neural network to obtain a topic mixture vector corresponding to each of the text word data and hidden state data corresponding to each of the text word data includes: The neural topic network is used to extract features from each of the main text word vectors to obtain a first prior variable and a second prior variable corresponding to each of the main text word data. The BoW decoder decodes each of the first prior variables and the second prior variables corresponding to each of the main text word data to obtain the Gaussian vector corresponding to each of the main text word data, the encoder hidden state corresponding to each of the main text word data, and the decoder hidden state corresponding to each of the main text word data. The hidden state data includes the encoder hidden state and the decoder hidden state. By normalizing each of the Gaussian vectors using the first formula, a topic mixing vector corresponding to each of the main text word data is obtained. The first formula is: Where, θ i W is the topic mixing vector corresponding to the i-th word in the main text. θ Z is a learnable parameter. i Let be the Gaussian vector corresponding to the i-th word in the text, and softmax() be the softmax function.

4. The text summarization method according to claim 1, characterized in that, The encoder includes a sequence coding layer, a matching layer, and a merging layer. The process of encoding and analyzing each title word vector and all body text word vectors using the encoder to obtain the target context vector corresponding to each body text word data includes: By mapping each title word vector and each body word vector using a preset vector table, a title dense vector and a body dense vector corresponding to each title word data are obtained. The sequence coding layer encodes each of the title dense vectors to obtain an initial title context vector corresponding to each of the title word data, specifically: The first formula is used to encode each of the title dense vectors to obtain the initial title context vector corresponding to each of the title word data. The second formula is: in, Among them, v j This is the initial title context vector corresponding to the j-th title word data. Let j be the first hidden vector of the title corresponding to the j-th title word data. Let be the second hidden vector of the title corresponding to the j-th title word, where j∈[1,L] and L is the total number of title words. GRU() is a bidirectional gate recursive unit, t j This is the dense vector of the title corresponding to the j-th title word. This is the first hidden vector of the title corresponding to the (j-1)th title word. This is the second hidden vector of the title corresponding to the (j+1)th title word data; Each of the dense text vectors is encoded using the sequence coding layer to obtain an initial text context vector corresponding to each of the text word data, specifically: The third equation is used to encode each of the dense text vectors to obtain the initial text context vector corresponding to each of the text word data. The third equation is: in, Among them, u i Let be the initial text context vector corresponding to the i-th word in the text. Let i be the first hidden vector of the text corresponding to the i-th word in the text. Let be the second hidden vector of the text corresponding to the i-th word, where i ∈ [1, N] and N is the total number of words in the text. GRU() is a bidirectional gate recursive unit, x i Let i be the dense vector of the text corresponding to the i-th word in the text. Let i be the first hidden vector of the text corresponding to the (i-1)th word in the text. This is the second hidden vector of the main text corresponding to the (i+1)th word in the main text. The matching layer matches each of the initial text context vectors and all of the initial title context vectors to obtain an aggregated information vector corresponding to each of the text word data, specifically: By matching each of the initial text context vectors and all the initial title context vectors using the fourth formula, an aggregated information vector corresponding to each of the text word data is obtained. The fourth formula is: in, Among them, c i Let α be the aggregated information vector corresponding to the i-th word in the main text. i,j For the standardized attention between the initial text context vector corresponding to the i-th text word and the initial title context vector corresponding to the j-th title word, v j Let u be the initial title context vector corresponding to the j-th title word data, L be the total number of title word data, and u be the initial title context vector. i Let W1 be the initial text context vector corresponding to the i-th text word data, N be the total number of text words data, and W1 be the learnable parameters. The merging layer merges each initial text context vector and the aggregated information vector corresponding to each text word data to obtain the target context vector corresponding to each text word data. Specifically: The fifth equation is used to merge the initial text context vectors and the aggregated information vectors corresponding to each text word data to obtain the target context vector corresponding to each text word data. in, in, Let α be the target context vector corresponding to the i-th word in the main text, and let α be a hyperparameter. Let i be the first hidden vector of the target context corresponding to the i-th word in the text. Let c be the second hidden vector of the target context corresponding to the i-th word in the text data, and GRU() be a bidirectional gate recursive unit. i Let u be the aggregated information vector corresponding to the i-th word in the main text. i Let be the initial text context vector corresponding to the i-th word in the text. Let i be the first hidden vector of the target context corresponding to the (i-1)th word in the text. This is the second hidden vector of the target context corresponding to the (i+1)th word in the text.

5. The text summarization method according to claim 1, characterized in that, The process of decoding each topic mixing vector and the target context vector corresponding to each text word data using the decoder to obtain the keyword generation probability corresponding to each text word data includes: By decoding each of the aforementioned topic mixing vectors and the target context vectors corresponding to each of the aforementioned text word data using the sixth formula, the keyword generation probability corresponding to each of the aforementioned text word data is obtained. The sixth formula is: Wherein, P(Y i ) represents the keyword generation probability corresponding to the i-th word in the main text. Let θ be the target context vector corresponding to the i-th word in the main text. i Let be the topic mixing vector corresponding to the i-th word in the main text.

6. The text summarization method according to claim 1, characterized in that, The hidden state data includes the encoder hidden state and the decoder hidden state. The process of analyzing the keyword vectors, all the text word vectors, and all the hidden state data to predict word probabilities and obtain the text summary generation result includes: The attention score for each of the text words is obtained by calculating the attention score of the keyword vector and the encoder hidden state corresponding to each of the text words using the seventh formula. The seventh formula is: ai=Softmax[VTtanh(Whhi+WsY+battn)], Among them, a i V and W represent the keyword attention scores corresponding to the i-th word in the main text. h W s and b attn All are learnable parameters, h i Y represents the encoder hidden state corresponding to the i-th word in the main text, and Y is the keyword vector. The keyword extraction probability is obtained by calculating the probability of attention scores for all the aforementioned keywords using the eighth formula: P c (w)=∑a i , Among them, P c (w) represents the keyword extraction probability, a i The keyword attention score corresponding to the i-th word in the main text; The ninth equation calculates the probabilities of the keyword vector, the encoder hidden state corresponding to each of the main text words, and the keyword attention score corresponding to each of the main text words to obtain the gating selection probability corresponding to each of the main text words. The ninth equation is as follows: in, in, The gating selection probability corresponding to the i-th word in the main text. Let σ be the semantic vector corresponding to the i-th word in the main text. w Y w X and b ptr All are learnable parameters, Y is the keyword vector, and X is... bi Let a be the text word vector corresponding to the i-th text word data. i h is the keyword attention score corresponding to the i-th word in the main text. i This represents the encoder's hidden state corresponding to the i-th word in the main text. The tenth formula is used to calculate the probability of the encoder hidden state, the keyword attention score, and the decoder hidden state corresponding to each of the main text word data. This yields the first word extraction probability corresponding to each of the main text word data. in, in, Let V', V', b, and b' be the probability of extracting the first word corresponding to the i-th word in the main text, where V', V', b', and b' are all learnable parameters. i This represents the decoder hidden state corresponding to the i-th word in the text. Let a be the semantic vector corresponding to the i-th word in the main text. i h is the keyword attention score corresponding to the i-th word in the main text. i This represents the encoder's hidden state corresponding to the i-th word in the main text. The probability of extracting the second word is obtained by calculating the probability of all the encoder hidden states and all the text word vectors using the eleventh formula, which is: Ps(w) = ∑bi, Among them, b i =Softmax[V T tanh(W h h i +W s X bi +b attn )], Among them, P s (w) represents the probability of extracting the second word, b i W represents the text attention score corresponding to the i-th word in the text. h V, W s and b attn All are learnable parameters, X bi h is the text word vector corresponding to the i-th text word data. i This represents the encoder's hidden state corresponding to the i-th word in the main text. The twelfth formula is used to calculate the probability of the second word extraction, the keyword extraction probability, the gating selection probability corresponding to each of the main text word data, and the first word extraction probability corresponding to each of the main text word data. This yields the target word prediction probability corresponding to each of the main text word data, and all the target word prediction probabilities are used as the text summarization result. The twelfth formula is: Among them, P i (w) represents the predicted probability of the target word corresponding to the i-th text word, P s (w) represents the probability of extracting the second word. Let be the probability of extracting the first word corresponding to the i-th word in the main text. Let P be the gating selection probability corresponding to the i-th word in the text. c (w) represents the keyword extraction probability.

7. A text summarization generation device, characterized in that, include: The import module is used to import raw text datasets; The preprocessing module is used to preprocess the original text dataset to obtain multiple title word data and multiple body text word data; The analysis module is used to build a training model, and to perform keyword analysis on all the title word data and all the body text word data through the training model to obtain keyword vectors, body text word vectors corresponding to each body text word data, and hidden state data corresponding to each body text word data. The summary generation result acquisition module is used to analyze the keyword vector, all the main text word vectors and all the hidden state data to predict the word probability and obtain the text summary generation result; The training model includes a topic extraction neural network, an encoder, and a decoder. The analysis module is specifically used for: By using a preset BERT model, features are extracted from each of the title word data and each of the body text word data to obtain title word vectors corresponding to each of the title word data and body text word vectors corresponding to each of the body text word data. The topic extraction neural network is used to perform topic analysis on each of the main text word vectors to obtain topic mixing vectors corresponding to each of the main text word data and hidden state data corresponding to each of the main text word data; The encoder is used to encode and analyze each of the title word vectors and all the body text word vectors to obtain the target context vector corresponding to each of the body text word data. The decoder decodes each of the topic mixing vectors and the target context vectors corresponding to each of the main text word data to obtain the keyword generation probability corresponding to each of the main text word data. The maximum value of the generation probability of all the keywords is selected, and the maximum keyword generation probability is obtained. The word vector of the main text corresponding to the maximum keyword generation probability is used as the keyword vector.

8. A text summarization generation system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the text summarization generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the text summarization method as described in any one of claims 1 to 6.

Images