A text summarization extraction method and device

Abstract

The application discloses a text abstract extraction method and device, and relates to the technical field of natural language processing. A specific embodiment of the method comprises the following steps: converting each sentence in a text into a semantic vector, inputting the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence; converting each sentence in the text into a syntax structure vector, inputting the syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence; inputting the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector; and inputting the text content vector into a third recurrent neural network to output a target sentence as a text abstract. The embodiment can solve the technical problem that an extractive text abstract does not conform to the central meaning of a text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a text summary extraction method and device. BACKGROUND

[0002] The influx of large amounts of information has caused many blockages in information understanding and information analysis, and therefore extracting core information from large amounts of information has become a major problem in the Internet.

[0003] Currently, information is extracted from massive data mainly by using a text summary extraction method. The text summary extraction method is mainly divided into an extraction type and a generation type. The extraction type of text summary is mainly to directly extract core sentences from a text to represent the summary of the text, and therefore the extracted summary is more in line with human language rules, is more readable, and is more fluent in expression.

[0004] For the extraction type of text summary, there are many algorithms used at present, such as a kmeans clustering algorithm, a textrank algorithm, and a neural network method. The extracted sentences by using these algorithms have the problem of not being in line with the core meaning of the text, especially in Chinese information extraction, because Chinese has complex syntax logic and text structure, which makes it more difficult to extract a Chinese summary. SUMMARY

[0005] Therefore, the embodiments of the present application provide a text summary extraction method and device to solve the technical problem of the extraction type of text summary not being in line with the core meaning of the text.

[0006] To achieve the above object, according to one aspect of the embodiments of the present application, a text summary extraction method is provided, comprising:

[0007] converting each sentence in a text into a semantic vector, inputting the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence;

[0008] converting each sentence in the text into a syntax structure vector, inputting the syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence;

[0009] inputting the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector;

[0010] inputting the text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0011] Optionally, converting each sentence in the text into a syntax structure vector comprises:

[0012] For each sentence in the text, the number of entities in the sentence is calculated, so as to generate a syntax vector of the sentence according to the number of entities in the sentence;

[0013] For each sentence in the text, the distance of the sentence from a baseline sentence is calculated, so as to generate a structure vector of the sentence according to the distance of the sentence from the baseline sentence;

[0014] The syntax vector and the structure vector of the sentence are input into a syntax structure convolution layer to output a syntax structure vector of the sentence.

[0015] Optionally, the distance of the sentence from the baseline sentence is calculated, comprising:

[0016] Each sentence in the text is sequentially numbered in a positive order, and the number of sentences in the text is counted as N;

[0017] The sentence corresponding to the number N / 2 is taken as a baseline sentence, and the difference between the number of the sentence and the number of the baseline sentence is calculated.

[0018] Optionally, each sentence in the text is converted into a semantic vector, comprising:

[0019] Each sentence in the text is converted into a semantic vector using a BERT model.

[0020] Optionally, the syntax structure vector of each sentence is input into a second recurrent neural network to output a structure representation vector of each sentence, comprising:

[0021] The syntax structure vector of each sentence is input into a first proofreading layer to output a proofread syntax structure vector of each sentence; wherein the first proofreading layer uses a difference degree vector to proofread the syntax structure vector of each sentence;

[0022] The proofread syntax structure vector of each sentence is input into a second recurrent neural network to output a structure representation vector of each sentence.

[0023] Optionally, the syntax structure vector of each sentence is proofread using a difference degree vector, comprising:

[0024] The syntax structure vector of each sentence is multiplied by the difference degree vector bit by bit.

[0025] Optionally, the text content vector is input into a third recurrent neural network to output a target sentence as a text summary, comprising:

[0026] inputting the text content vector into a second correction layer to output a corrected text content vector; wherein the second correction layer corrects the text content vector by using a difference degree vector;

[0027] inputting the corrected text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0028] Optionally, the correcting the text content vector by using a difference degree vector comprises:

[0029] bitwise multiplying the text content vector and the difference degree vector.

[0030] Optionally, before converting each sentence in the text into a semantic vector, the method further comprises:

[0031] converting the summary sentence of the text into a semantic vector by using a BERT model;

[0032] calculating a difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence output by the third recurrent neural network.

[0033] Optionally, the first recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the second recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the third recurrent neural network is a unidirectional long short-term memory artificial neural network.

[0034] In addition, according to another aspect of an embodiment of the present application, a text summary extraction device is provided, comprising:

[0035] a first representation module configured to convert each sentence in a text into a semantic vector, and input the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence;

[0036] a second representation module configured to convert each sentence in the text into a syntax structure vector, and input the syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence;

[0037] a text content module configured to input the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector;

[0038] an extraction module configured to input the text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0039] Optionally, the second representation module is further configured to:

[0040] For each sentence in the text, calculate the number of entities in the sentence, thereby generating a syntax vector of the sentence according to the number of entities in the sentence;

[0041] For each sentence in the text, calculate the distance of the sentence from the baseline sentence, thereby generating a structure vector of the sentence according to the distance of the sentence from the baseline sentence;

[0042] Input the syntax vector and the structure vector of the sentence into a syntax structure convolution layer to output a syntax structure vector of the sentence.

[0043] Optionally, the second representation module is further used to:

[0044] Number each sentence in the text in a positive order, and count the number of sentences in the text as N;

[0045] Take the sentence corresponding to the number N / 2 as the baseline sentence, and calculate the difference between the number of the sentence and the number of the baseline sentence.

[0046] Optionally, the first representation module is further used to:

[0047] Convert each sentence in the text into a semantic vector using a BERT model.

[0048] Optionally, the second representation module is further used to:

[0049] Input the syntax structure vector of each sentence into a first proofreading layer to output a proofread syntax structure vector of each sentence; wherein the first proofreading layer uses a difference degree vector to proofread the syntax structure vector of each sentence;

[0050] Input the proofread syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence.

[0051] Optionally, the second representation module is further used to:

[0052] Bitwise multiply the syntax structure vector of each sentence with the difference degree vector.

[0053] Optionally, the extraction module is further used to:

[0054] Input the text content vector into a second proofreading layer to output a proofread text content vector; wherein the second proofreading layer uses a difference degree vector to proofread the text content vector;

[0055] Input the proofread text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0056] Optionally, the extracting module is further configured to:

[0057] multiply the text content vector and the difference degree vector bit by bit.

[0058] Optionally, the method further comprises a difference degree module configured to:

[0059] convert the summary sentence of the text into a semantic vector using a BERT model before converting each sentence in the text into a semantic vector.

[0060] calculate a difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence output by the third recurrent neural network.

[0061] Optionally, the first recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the second recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the third recurrent neural network is a unidirectional long short-term memory artificial neural network.

[0062] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising:

[0063] one or more processors;

[0064] a storage device configured to store one or more programs,

[0065] when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of the above embodiments.

[0066] According to another aspect of the embodiments of the present application, a computer readable medium having stored thereon a computer program is also provided, the program being executed by a processor to implement the method of any of the above embodiments.

[0067] An embodiment of the above application has the following advantages or beneficial effects: because the technical means of converting each sentence in the text into a semantic vector and a syntax structure vector, inputting the semantic vector and the syntax structure vector into a first recurrent neural network and a second recurrent neural network respectively, outputting a semantic representation vector and a structure representation vector of each sentence respectively, then outputting a text content vector through a semantic structure convolution layer, and finally outputting a text summary through a third recurrent neural network, the technical problem of an extractive text summary not conforming to the central meaning of the text in the prior art is overcome. The embodiment of the application considers the syntax and text structure of a Chinese sentence in the design of a text summary extraction model, converts each sentence into a semantic vector and a syntax structure vector, thereby accurately extracting target sentences that conform to the central meaning of the text. Moreover, the embodiment of the application calculates a difference degree vector between a summary sentence and a target sentence, and feeds it back to a proofreading layer as a check, trains parameters of the entire text summary extraction model, so that the entire model makes consistent adjustments to the same parameters; as can be seen from the entire text summary extraction model, because the proofreading layer is added, the extracted information will be as close as possible to the most similar sentence, the model will have a preference for selecting the most similar sentence, thereby reducing the sentence error to a minimum in the proofreading layer.

[0068] Further effects of the above non-conventional optional mode will be described below in conjunction with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0069] The accompanying drawings are used to better understand the application and do not constitute an improper limitation on the application. Among them:

[0070] Figure 1 is a schematic diagram of the main process of a text summary extraction method according to an embodiment of the application;

[0071] Figure 2 is a structural schematic diagram of a text summary extraction model in an extraction stage according to an embodiment of the application;

[0072] Figure 3 is a structural schematic diagram of a text summary extraction model in a training stage according to an embodiment of the application;

[0073] Figure 4 is a schematic diagram of the main process of a text summary extraction method according to an embodiment of the application;

[0074] Figure 5 is a schematic diagram of the main modules of a text summary extraction device according to an embodiment of the application;

[0075] Figure 6 is an exemplary system architecture diagram to which the embodiment of the application can be applied;

[0076] Figure 7 is a structural schematic diagram of a computer system of a terminal device or a server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0077] Exemplary embodiments of the present application are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the present application by individuals of ordinary skill in the art, and should not be construed as limiting the scope of the present application. Thus, those of ordinary skill in the art will recognize various changes and modifications of the embodiments described herein, which do not depart from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0078] According to the current domestic situation analysis, most text summary extraction algorithms are used for English data sets for research, thereby ignoring the particularity of the Chinese scene, and directly using such models on Chinese will have some shortcomings. Therefore, an embodiment of the present application designs an algorithm for Chinese. The embodiment of the present application mainly aims at the technical problem that the extracted text summary does not conform to the central meaning of the text, and proposes a text summary extraction method which considers the unique grammatical characteristics of Chinese and the structure of different texts, and compares the difference between the extracted target sentence and the summary sentence, and performs model verification based on the comparison result, so that the entire model performs well on Chinese information extraction.

[0079] Figure 1 is a schematic diagram of the main process of the text summary extraction method according to an embodiment of the present application. As shown in Figure 1 , the text summary extraction method can include:

[0080] Step 101, converting each sentence in the text into a semantic vector, inputting the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence.

[0081] First, a sentence splitting tool can be used to split the text, and then each sentence is converted into a semantic vector, and then the semantic vector of each sentence is input into a first recurrent neural network, and the first recurrent neural network learns the semantic vector of each sentence, thereby outputting a semantic representation vector of each sentence.

[0082] Optionally, as shown in Figure 2As shown, each sentence in the text is converted into a semantic vector, including: using the BERT model to convert each sentence in the text into a semantic vector. After the text is segmented, the BERT model can be used to convert the sentences into semantic vectors, and then input them into a first recurrent neural network. Optionally, the first recurrent neural network is a bidirectional recurrent neural network, and learning the semantic vectors of the sentences through the bidirectional recurrent neural network helps to improve the representation ability of the semantic vectors. Optionally, the first recurrent neural network is a bidirectional long short-term memory recurrent neural network (bidirectional LSTM), and bidirectional learning through the bidirectional long short-term memory recurrent neural network helps to accurately output the semantic representation vectors of each sentence.

[0083] Step 102: Convert each sentence in the text into a grammatical structure vector, and input the grammatical structure vector of each sentence into a second recurrent neural network to output a structural representation vector of each sentence.

[0084] Chinese text has unique and complex grammar and text structure, so the present invention takes these two factors into account when designing the text summarization model. To account for the impact of grammar on sentences, the pyltp model can be used to perform grammatical analysis on sentences. From a linguistic perspective, sentences containing nouns and verbs are significantly more likely to represent the central idea of ​​the text. From a text structure perspective, sentences at the beginning or end of an article are also more likely to represent the central idea of ​​the text.

[0085] Optionally, each sentence in the text is converted into a grammatical structure vector, including: for each sentence in the text, calculating the number of entities in the sentence, thereby generating a grammatical vector for the sentence based on the number of entities in the sentence; for each sentence in the text, calculating the difference between the sentence and a text baseline sentence, thereby generating a structural vector for the sentence based on the difference between the sentence and the text baseline sentence; inputting the grammatical vector and structural vector of the sentence into a grammatical structure convolution layer to output the grammatical structure vector of the sentence. For each sentence in the text, a grammatical analysis is performed on it, for example, a pyltp model can be used to identify entities such as verbs and names in each sentence, and the number of these entities is counted, and then the number of entities in the sentence is converted into a grammatical vector of the sentence based on vector matrix filling, and then the difference between the sentence and the text baseline sentence is calculated, thereby converting the difference between the sentence and the text baseline sentence into a structural vector of the sentence based on vector matrix filling, and finally inputting the grammatical vector and structural vector of the sentence into a grammatical structure convolution layer, and performing a convolution operation on it through the grammatical structure convolution layer to output the grammatical structure vector of the sentence.

[0086] Optionally, the difference between the sentence and the baseline sentence is calculated, including: numbering each sentence in the text in a positive order, and counting the number of sentences in the text as N; taking the sentence corresponding to the number N / 2 as the baseline sentence, and calculating the difference between the number of the sentence and the number of the baseline sentence. Since the end and the beginning are generally the focus of the text, more weight is given to the beginning and the end of the text. Specifically, the text is segmented, each sentence is numbered in a positive order, and the number of sentences in the text is counted as N. Then, the sentence corresponding to the number N / 2 is taken as the baseline sentence of the text. Finally, the difference between the number of each sentence and the number of the baseline sentence of the text is calculated. In the embodiment of the application, the importance of the sentence in the overall structure of the text can be determined according to the difference between the number of the sentence and the number of the baseline sentence.

[0087] Optionally, as shown in Figure 2 the syntax structure vector of each sentence is input into the second recurrent neural network to output the structure representation vector of each sentence, including: inputting the syntax structure vector of each sentence into the first proofreading layer to output the proofread syntax structure vector of each sentence; wherein the first proofreading layer corrects the syntax structure vector of each sentence using the difference degree vector; and inputting the proofread syntax structure vector of each sentence into the second recurrent neural network to output the structure representation vector of each sentence. After obtaining the syntax structure vector of each sentence, the first proofreading layer corrects the syntax structure vector of each sentence, so that the difference degree vector is fused with the structure representation vector of the sentence to obtain the proofread syntax structure vector of the sentence. After proofreading, the proofread syntax structure vector of each sentence is input into the second recurrent neural network, and the second recurrent neural network learns the proofread syntax structure vector of each sentence to output the structure representation vector of each sentence.

[0088] Optionally, the difference degree vector is used to correct the syntax structure vector of each sentence, including: multiplying the syntax structure vector of each sentence by the difference degree vector bit by bit. For example, the following formula can be used for correction:

[0089]

[0090] wherein verification is the difference degree vector, article_structrue is the syntax structure vector of the sentence, and article_structrue' is the proofread syntax structure vector.

[0091] Optionally, the second recurrent neural network is a bidirectional recurrent neural network, and learning the syntax structure vector of the sentence by the bidirectional recurrent neural network helps to improve the representation ability of the syntax structure vector. Optionally, the second recurrent neural network is a bidirectional long short-term memory recurrent neural network (bidirectional LSTM), and bidirectional learning by the bidirectional long short-term memory recurrent neural network helps to accurately output the structure representation vector of each sentence. The embodiment of the present application uses the bidirectional LSTM because the Chinese text structure needs to be determined by the context, and therefore the overall structure of the text needs to be considered.

[0092] In step 103, the semantic representation vector and the structure representation vector of each sentence are input into a semantic structure convolution layer to output a text content vector.

[0093] Through steps 101 and 102, the semantic representation vector and the structure representation vector of each sentence based on Chinese special analysis are obtained, and convolution operation is performed on them, so that the corresponding text content vector representation is obtained. Specifically, as shown in Figure 2 After obtaining the semantic representation vector and the structure representation vector of each sentence, the semantic representation vector and the structure representation vector of the sentence are input into the semantic structure convolution layer together, convolution operation is performed on them by the semantic structure convolution layer, and thus the text content vector, i.e. the content vector representation of the entire text, is output.

[0094] In step 104, the text content vector is input into a third recurrent neural network to output a target sentence as a text summary.

[0095] After the text content vector is output by the semantic structure convolution layer, it is continuously input into the third recurrent neural network, and the text summary vector is output by decoding. Optionally, the third recurrent neural network is a unidirectional long short-term memory artificial neural network, and in this embodiment, the text content vector is decoded by the unidirectional long short-term memory artificial neural network, so that the semantic vector of the target sentence, i.e. the semantic vector of the text summary, is output.

[0096] Optionally, as shown in Figure 2As shown, the text content vector is input into a third recurrent neural network to output a target sentence as a text summary, including: inputting the text content vector into a second proofreading layer to output a proofread text content vector; wherein the second proofreading layer corrects the text content vector using a difference degree vector; inputting the proofread text content vector into the third recurrent neural network to output a target sentence as a text summary. After obtaining the text content vector, the second proofreading layer corrects the text content vector, so the difference degree vector is fused with the text content vector to obtain the proofread text content vector. After proofreading, the proofread text content vector is continuously input into the third recurrent neural network, the third recurrent neural network decodes the proofread text content vector, thereby outputting a semantic vector of the target sentence, and the target sentence is taken as a text summary.

[0097] Optionally, correcting the text content vector using the difference degree vector includes: bitwise multiplying the text content vector and the difference degree vector. For example, the following formula can be used for correction:

[0098]

[0099] Wherein, verification is the difference degree vector, content is the text content vector, and content' is the proofread text content vector.

[0100] Most of the current neural networks used in extractive text summarization algorithms do not consider their generality, and the output sentences cannot be well fed back to the network parameters, resulting in poor extraction effect of the model. The difference degree vector is calculated by the difference between the marked summary sentence and the target sentence output by the third recurrent neural network. Optionally, before step 101, it also includes: converting the summary sentence of the text into a semantic vector using a BERT model; calculating the difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence output by the third recurrent neural network. First, the summary sentence of the text can be marked in an artificial manner, such as Figure 3The BERT model is used to convert the summary sentence into a semantic vector, and then the semantic vector of the summary sentence is input into a difference calculation layer, and the semantic vector of the target sentence output by the third recurrent neural network is also input into the difference calculation layer. The difference calculation layer calculates the difference vector between the semantic vector of the summary sentence and the semantic vector of the target sentence by using the Euclidean distance. The greater the difference is, the less the extracted information meets the requirements. The smaller the difference is, the higher the quality of the extracted information is. In the training stage of the model, the difference calculation layer is mainly used to calculate the difference between the summary sentence and the target summary, and feed it back to the first correction layer and the second correction layer, so as to adjust the parameters of the entire text extraction model, so that the extraction effect of the model is better.

[0101] Since the correction layer is used for correcting the semantics and the text structure of the sentence, in similar sentences, the sentence with the largest similarity is selected, so as to reduce the error, and the model can extract the optimal information. The embodiment of the present application not only considers the special grammatical characteristics of Chinese in the text summary extraction model, but also considers the structure of the entire text, and uses the summary sentence correction method to make the model parameters more comprehensive training, which helps to extract the target sentence with the central meaning of the text.

[0102] The method provided by the embodiment of the present application can be used in a large number of commodity summaries to extract a sentence representing the characteristics of the commodity as the introduction of the commodity, without manually writing the corresponding introduction, thereby saving the corresponding manpower.

[0103] According to the above various embodiments, it can be seen that the embodiment of the present application solves the technical problem that the extracted text summary does not meet the central meaning of the text by converting each sentence in the text into a semantic vector and a grammatical structure vector, inputting the semantic vector and the grammatical structure vector into the first recurrent neural network and the second recurrent neural network respectively, outputting the semantic representation vector and the structure representation vector of each sentence respectively, then outputting the text content vector through the semantic structure convolution layer, and finally outputting the text summary through the third recurrent neural network. The embodiment of the present application considers the grammar and text structure of the Chinese sentence in the design of the text summary extraction model, and converts each sentence into a semantic vector and a grammatical structure vector, so as to accurately extract the target sentence meeting the central meaning of the text. Moreover, the embodiment of the present application calculates the difference vector between the summary sentence and the target sentence, and feeds it back to the correction layer as a check, so as to train the parameters of the entire text summary extraction model, so that the entire model makes consistent adjustments to the same parameter. As can be seen from the entire text summary extraction model, since the correction layer is added, the extracted information is as close to the most similar sentence as possible, the model has a preference for sentences with different degrees of similarity, and the most similar sentence is selected, so that the sentence error is reduced to a minimum in the correction layer.

[0104] Figure 4 is a schematic diagram of the main flow of a text summary extraction method according to one embodiment of the present application. As another embodiment of the present application, as shown in Figure 4 the text summary extraction method can include:

[0105] Step 401, using a BERT model to convert each sentence in the text into a semantic vector.

[0106] First, the text can be segmented using a sentence segmentation tool, and then the BERT model is used to convert each sentence in the text into a semantic vector.

[0107] Step 402, input the semantic vector of each sentence into a first recurrent neural network to output the semantic representation vector of each sentence.

[0108] Then, input the semantic vector of each sentence into the first recurrent neural network, and the first recurrent neural network learns the semantic vector of each sentence to output the semantic representation vector of each sentence.

[0109] Step 403, for each sentence in the text, calculate the number of entities in the sentence, and generate the syntax vector of the sentence according to the number of entities in the sentence.

[0110] From the perspective of linguistics, the probability that a sentence with nouns and verbs represents the central meaning of the text will greatly increase, so the pyltp model can be used to perform syntax analysis on the sentence, calculate the number of entities in each sentence, and then convert the number of entities in each sentence into the syntax vector of the sentence based on vector matrix filling.

[0111] Step 404, for each sentence in the text, calculate the distance between the sentence and the baseline sentence, and generate the structure vector of the sentence according to the distance between the sentence and the baseline sentence.

[0112] From the analysis of the text structure, the probability that a sentence at the beginning or end of an article represents the central meaning of the text is also relatively large, so each sentence in the text can be sequentially numbered in ascending order, the number of sentences in the text is N, and the sentence numbered N / 2 is taken as the baseline sentence. The difference between the number of each sentence and the number of the baseline sentence is calculated, and then the distance between the sentence and the baseline sentence is converted into the structure vector of the sentence based on vector matrix filling.

[0113] Step 405, input the syntax vector and structure vector of the sentence into the syntax structure convolution layer to output the syntax structure vector of the sentence.

[0114] Subsequently, the syntax vector and the structure vector of the sentence are input into a syntax structure convolution layer, and the syntax structure vector of the sentence is output by convolution operation of the syntax structure convolution layer.

[0115] In step 406, the syntax structure vector of each sentence is input into a first proofreading layer to output a proofread syntax structure vector of each sentence; wherein the first proofreading layer corrects the syntax structure vector of each sentence by using the difference degree vector.

[0116] After obtaining the syntax structure vector of each sentence, the syntax structure vector of each sentence is corrected by the first proofreading layer, so that the difference degree vector is fused with the structure representation vector of the sentence to obtain the proofread syntax structure vector of the sentence.

[0117] In step 407, the proofread syntax structure vector of each sentence is input into a second recurrent neural network to output a structure representation vector of each sentence.

[0118] After proofreading, the proofread syntax structure vector of each sentence is continuously input into the second recurrent neural network, and the second recurrent neural network learns the proofread syntax structure vector of each sentence to output the structure representation vector of each sentence.

[0119] In step 408, the semantic representation vector and the structure representation vector of each sentence are input into a semantic structure convolution layer to output a text content vector.

[0120] After obtaining the semantic representation vector and the structure representation vector of each sentence based on Chinese special analysis, convolution operation is performed on the semantic representation vector and the structure representation vector to obtain a corresponding text content vector.

[0121] In step 409, the text content vector is input into a second proofreading layer to output a proofread text content vector; wherein the second proofreading layer corrects the text content vector by using the difference degree vector.

[0122] After obtaining the text content vector, the text content vector is corrected by the second proofreading layer, so that the difference degree vector is fused with the text content vector to obtain the proofread text content vector.

[0123] In step 410, the proofread text content vector is input into a third recurrent neural network to output a target sentence as a text summary.

[0124] After proofreading, the proofread text content vector is continuously input into the third recurrent neural network, and the third recurrent neural network decodes the proofread text content vector to output a semantic vector of a target sentence, and the target sentence is taken as a text summary.

[0125] It should be noted that, in the training phase of the text summary extraction model, as shown in Figure 3 The embodiment of the application calculates the difference degree vector through the difference between the marked summary sentence and the target sentence output by the third recurrent neural network. First, the summary sentence of the text can be marked manually, as shown in Figure 3 Then, the BERT model is used to convert the summary sentence into a semantic vector, and then the semantic vector of the summary sentence is input into the difference degree calculation layer, and the semantic vector of the target sentence output by the third recurrent neural network is also input into the difference degree calculation layer. The difference degree calculation layer calculates the difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence using the Euclidean distance. The greater the difference degree, the less the extracted information meets the requirements; the smaller the difference, the higher the quality of the extracted information.

[0126] In the training phase of the model, the difference degree calculation layer is mainly used to calculate the difference between the summary sentence and the target summary, and feed it back to the first correction layer and the second correction layer, so as to adjust the parameters of the entire text extraction model, so that the extraction effect of the model is better.

[0127] The embodiment of the application designs a text summary extraction model combining semantic and syntactic structure considering the characteristics of Chinese, and simultaneously trains and adjusts the first correction layer and the second correction layer uniformly, so that the entire model makes consistent adjustment to the same parameter, so that the parameters in the network are more comprehensively trained. Using the model can accurately extract the target sentence in the text as the representative of the central meaning, so as to help quickly obtain important information in the text.

[0128] In addition, in the specific implementation content of the text summary extraction method according to an embodiment of the application, which has been described in detail in the above text summary extraction method, the repeated content will not be described here.

[0129] Figure 5 is a schematic diagram of the main modules of the text summary extraction device according to the embodiment of the application, as shown in Figure 5As shown, the text summary extraction device 500 comprises a first representation module 501, a second representation module 502, a text content module 503, and an extraction module 504; the first representation module 501 is configured to convert each sentence in the text into a semantic vector, input the semantic vector of each sentence into a first recurrent neural network, and output a semantic representation vector of each sentence; the second representation module 502 is configured to convert each sentence in the text into a syntax structure vector, input the syntax structure vector of each sentence into a second recurrent neural network, and output a structure representation vector of each sentence; the text content module 503 is configured to input the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer, and output a text content vector; and the extraction module 504 is configured to input the text content vector into a third recurrent neural network, and output a target sentence as a text summary.

[0130] Optionally, the second representation module 502 is further configured to:

[0131] For each sentence in the text, calculate the number of entities in the sentence, thereby generating a syntax vector of the sentence according to the number of entities in the sentence;

[0132] For each sentence in the text, calculate the distance of the sentence from a baseline sentence, thereby generating a structure vector of the sentence according to the distance of the sentence from the baseline sentence;

[0133] Input the syntax vector and the structure vector of the sentence into a syntax structure convolution layer, and output a syntax structure vector of the sentence.

[0134] Optionally, the second representation module 502 is further configured to:

[0135] Number each sentence in the text in a positive order in sequence, and count the number of sentences in the text as N;

[0136] Take the sentence corresponding to the number N / 2 as a baseline sentence, and calculate the difference between the number of the sentence and the number of the baseline sentence.

[0137] Optionally, the first representation module 501 is further configured to:

[0138] Convert each sentence in the text into a semantic vector using a BERT model.

[0139] Optionally, the second representation module 502 is further configured to:

[0140] Input the syntax structure vector of each sentence into a first proofreading layer, and output a proofread syntax structure vector of each sentence; wherein the first proofreading layer proofreads the syntax structure vector of each sentence using a difference degree vector.

[0141] input the proofreading syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence.

[0142] Optionally, the second representation module 502 is further configured to:

[0143] bitwise multiply the syntax structure vector of each sentence with the difference degree vector.

[0144] Optionally, the extraction module 504 is further configured to:

[0145] input the text content vector into a second proofreading layer to output a proofreading text content vector; wherein the second proofreading layer proofreads the text content vector using the difference degree vector;

[0146] input the proofreading text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0147] Optionally, the extraction module 504 is further configured to:

[0148] bitwise multiply the text content vector with the difference degree vector.

[0149] Optionally, the method further comprises a difference degree module configured to:

[0150] convert the summary sentence of the text into a semantic vector using a BERT model before converting each sentence in the text into a semantic vector;

[0151] calculate a difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence output by the third recurrent neural network.

[0152] Optionally, the first recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the second recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the third recurrent neural network is a unidirectional long short-term memory artificial neural network.

[0153] It should be noted that the specific implementation of the text summary extraction device described in the present application has been described in detail in the text summary extraction method described above, and therefore the repeated content will not be described here.

[0154] Figure 6 An exemplary system architecture 600 is shown, which can apply the text summary extraction method or the text summary extraction device of the embodiments of the present application.

[0155] As Figure 6As shown, the system architecture 600 can include terminal devices 601, 602, 603, a network 604 and a server 605. The network 604 is a medium for providing communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0156] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0157] The terminal devices 601, 602, 603 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0158] The server 605 can be a server providing various services, such as a background management server supporting a shopping website browsed by the user using the terminal devices 601, 602, 603 (only as an example). The background management server can analyze and process received item information query requests and other data, and feed back the processing results to the terminal devices.

[0159] It should be noted that the text abstract extraction method provided by the embodiments of the present application is generally executed by the server 605, and accordingly, the text abstract extraction device is generally provided in the server 605. The text abstract extraction method provided by the embodiments of the present application can also be executed by the terminal devices 601, 602, 603, and accordingly, the text abstract extraction device can be provided in the terminal devices 601, 602, 603.

[0160] It should be understood that Figure 6 The number of terminal devices, networks and servers in the system architecture 600 is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0161] Reference is made below to Figure 7 which shows a structural schematic diagram of a computer system 700 of a terminal device suitable for implementing the embodiments of the present application. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0162] As Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701 which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 702 or loaded into a random access memory (RAM) 703 from a storage section 708. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0163] Connected to the I / O interface 705 are an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a display device such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as necessary. A removable recording medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 710 as necessary, so that a computer program read therefrom is installed into the storage section 708 as necessary.

[0164] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable recording medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-described functions defined in the system of the present disclosure are executed.

[0165] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or instrument, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or instrument. In the present application, the computer-readable signal medium can include a data signal propagating in a baseband or as a carrier wave part of a carrier wave, in which computer-readable program code is carried. Such a propagating data signal can take various forms, including but not limited to electromagnetic signals, optical signals or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or instrument. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0166] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer programs according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0167] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. The modules described can also be arranged in a processor, for example, a processor can be described as comprising a first representation module, a second representation module, a text content module and an extraction module, wherein the names of these modules do not constitute a limitation on the modules themselves in some cases.

[0168] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, when the one or more programs are executed by the device, the device implements the following method: converting each sentence in a text into a semantic vector, inputting the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence; converting each sentence in the text into a syntax structure vector, inputting the syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence; inputting the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector; inputting the text content vector into a third recurrent neural network to output a target sentence as a text summary.

[0169] According to the technical scheme of the embodiments of the present application, because the technical means of converting each sentence in a text into a semantic vector and a syntax structure vector, inputting the semantic vector and the syntax structure vector into a first recurrent neural network and a second recurrent neural network respectively, outputting a semantic representation vector and a structure representation vector of each sentence respectively, then outputting a text content vector through a semantic structure convolution layer, and finally outputting a text summary through a third recurrent neural network, the technical problem of an extractive text summary not conforming to the central meaning of the text in the prior art is overcome. The embodiments of the present application consider the syntax and text structure of Chinese sentences in the design of a text summary extraction model, and convert each sentence into a semantic vector and a syntax structure vector, thereby accurately extracting a target sentence conforming to the central meaning of the text. Moreover, the embodiments of the present application calculate a difference degree vector between a summary sentence and a target sentence, and feed it back to a proofreading layer as a check, and train parameters of the entire text summary extraction model, so that the entire model makes consistent adjustments for the same parameters; as can be seen from the entire text summary extraction model, since the proofreading layer is added, the extracted information will be as close as possible to the most similar sentence, the model will have a preference for sentences with different degrees of similarity, and will select the most similar sentence, thereby reducing the sentence error to a minimum in the proofreading layer.

[0170] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of specific terminology. However, embodiments thereof can be practiced with the exact description not being presented in detail. The term "device" should be understood to encompass devices operating in various modes, such as active mode, sleep mode, hibernate mode, and the like. The terms "coupled" and "connected," along with derivatives thereof, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, particular circuitry that can be said to be coupled to, or connected with, other circuitry can be some of the other circuitry that can communicate in some way, while other circuitry can be some of the other circuitry that can not be in communication with that particular circuitry.

Claims

1. A text summarization method, characterized in that: The method comprises the following steps: Converting each sentence in the text into a semantic vector, inputting the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence; Converting each sentence in the text into a syntax structure vector, inputting the syntax structure vector of each sentence into a first proofreading layer to output a proofread syntax structure vector of each sentence; wherein the first proofreading layer corrects the syntax structure vector of each sentence by using a difference degree vector; inputting the proofread syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence; Inputting the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector; Inputting the text content vector into a third recurrent neural network to output a target sentence as a text summary.

2. The method of claim 1, wherein, The method for converting each sentence in the text into a syntax structure vector comprises the following steps: For each sentence in the text, calculating the number of entities in the sentence, thereby generating a syntax vector of the sentence according to the number of entities in the sentence; For each sentence in the text, calculating the difference between the sentence and a baseline sentence, thereby generating a structure vector of the sentence according to the difference between the sentence and the baseline sentence; Inputting the syntax vector and the structure vector of the sentence into a syntax structure convolution layer to output a syntax structure vector of the sentence.

3. The method of claim 2, wherein, The method for calculating the difference between the sentence and the baseline sentence comprises the following steps: Numbering each sentence in the text in a positive order, and counting the number of sentences in the text as N; Taking the sentence corresponding to the number N / 2 as a baseline sentence, and calculating the difference between the number of the sentence and the number of the baseline sentence.

4. The method of claim 1, wherein, The method for converting each sentence in the text into a semantic vector comprises the following steps: Converting each sentence in the text into a semantic vector by using a BERT model.

5. The method of claim 1, wherein, The method for correcting the syntax structure vector of each sentence by using a difference degree vector comprises the following steps: Bitwise multiplying the syntax structure vector of each sentence with a difference degree vector.

6. The method of claim 1, wherein, The method for inputting the text content vector into a third recurrent neural network to output a target sentence as a text summary comprises the following steps: Inputting the text content vector into a second proofreading layer to output a proofread text content vector; wherein the second proofreading layer corrects the text content vector by using a difference degree vector; Inputting the proofread text content vector into a third recurrent neural network to output a target sentence as a text summary.

7. The method of claim 6, wherein, The method for correcting the text content vector by using a difference degree vector comprises the following steps: Bitwise multiplying the text content vector with a difference degree vector.

8. The method according to any one of claims 1-7, characterized in that, Before converting each sentence in the text into a semantic vector, the method further comprises the following steps: Converting the summary sentence of the text into a semantic vector by using a BERT model; Calculating a difference degree vector between the semantic vector of the summary sentence and the semantic vector of the target sentence output by the third recurrent neural network.

9. The method of claim 1, wherein, The first recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the second recurrent neural network is a bidirectional long short-term memory recurrent neural network; and / or, the third recurrent neural network is a unidirectional long short-term memory artificial neural network.

10. A text summarization extraction apparatus characterized by comprising: Comprising: A first representation module configured to convert each sentence in a text into a semantic vector, and input the semantic vector of each sentence into a first recurrent neural network to output a semantic representation vector of each sentence; A second representation module configured to convert each sentence in the text into a syntax structure vector, and input the syntax structure vector of each sentence into a first proofreading layer to output a proofread syntax structure vector of each sentence; wherein the first proofreading layer uses a difference degree vector to proofread the syntax structure vector of each sentence; and input the proofread syntax structure vector of each sentence into a second recurrent neural network to output a structure representation vector of each sentence; A text content module configured to input the semantic representation vector and the structure representation vector of each sentence into a semantic structure convolution layer to output a text content vector; An extraction module configured to input the text content vector into a third recurrent neural network to output a target sentence as a text summary.

11. An electronic device, comprising: Comprising: One or more processors; A storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-9.

12. A computer readable medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-9.

Images