Text abstract generation method and device, electronic equipment and storage medium

Abstract

The invention provides a text abstract generation method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The first similarity between sample sentences and abstract sentences is calculated, the sample sentence with the maximum first similarity is selected as a candidate sentence, and if the number of the candidate sentences is equal to the number of the abstract sentences, the abstract sentences are generated. If yes, determining a candidate text according to the candidate sentence, training according to the candidate text to obtain an extraction type abstract model, performing abstract extraction on the sample text through the extraction type abstract model to obtain a primary abstract, and performing abstract screening on the primary abstract according to the sample abstract to obtain a candidate abstract; and training according to the candidate abstract and the sample abstract to obtain a generative abstract model, performing abstract extraction on the target text through an extraction abstract model to obtain a coarsened abstract, and performing abstract refinement on the coarsened abstract through the generative abstract model to obtain the target abstract, so that the accuracy of text abstract generation can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text summary generation method, device, electronic device, and storage medium. Background Art

[0002] Text summarization is to extract key information from a longer text and generate a concise summary. In related technologies, a single extractive summary model or a generative summary model is used to generate text summaries. For texts in specific fields, such as legal texts, the texts cover a variety of closely related topics, are highly professional, have complex structures, and are long. A single extractive summary model or a generative summary model cannot accurately understand the text semantics of texts in specific fields, resulting in low accuracy in text summary generation. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a text summary generation method, device, electronic device, and storage medium, aiming to improve the accuracy of text summary generation.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a text summary generation method, the method comprising:

[0005] Acquire a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence;

[0006] Calculating a first similarity between the sample sentence and the summary sentence, selecting the sample sentence with the largest first similarity as a candidate sentence, and recording the number of the candidate sentences;

[0007] If the number of the sentences is equal to the number of the summary sentences in the sample summary, determining a candidate text according to the candidate sentences;

[0008] Training a preset summary extraction model according to the candidate text to obtain an extractive summary model;

[0009] Extracting a summary of the sample text by using an extractive summarization model to obtain a preliminary summary;

[0010] Screening the preliminary abstracts according to the sample abstracts to obtain candidate abstracts;

[0011] Training a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model;

[0012] Get the target text;

[0013] The target text is summarized and extracted by the extractive summary model to obtain a coarse summary, and the coarse summary is refined by the generative summary model to obtain a target summary.

[0014] In some embodiments, the extractive summarization model includes a first attention sub-model, a second attention sub-model, and a classification sub-model, and extracting a summary of the sample text by using the extractive summarization model to obtain a preliminary summary includes:

[0015] Performing semantic analysis on the sample text by using the first attention sub-model to obtain sentence semantic representation of the sample text;

[0016] Perform sentence encoding on the sentence semantic representation of the sample text by using the second attention sub-model to obtain sentence encoding features of each sample sentence;

[0017] For each of the sample sentences, the sentence encoding features are classified by the classification sub-model to obtain sentence importance categories; the sentence importance categories are used to indicate that the candidate sentences are important or the candidate sentences are not important;

[0018] The sample text is screened according to the important categories of the sentences to obtain the preliminary summary.

[0019] In some embodiments, the first attention sub-model includes a first network sub-layer and a second network sub-layer, the first network sub-layer has a query weight matrix, a first key weight matrix and a value weight matrix, and the semantic analysis of the sample text by the first attention sub-model to obtain the sentence semantic representation of the sample text includes:

[0020] In the first network sublayer, an original query feature of the first sample sentence is determined according to the query weight matrix and the first sample sentence, an original key feature of the first sample sentence is determined according to the first key weight matrix and the first sample sentence, and an original value feature of the first sample sentence is determined according to the value weight matrix and the first sample sentence; an attention transformation is performed on the original query feature, the original key feature, and the original value feature of the first sample sentence to obtain a sentence hidden state feature of the first sample sentence;

[0021] In the first network sublayer, starting from the second sample sentence, the previous sample sentence and the current sample sentence are concatenated to obtain a concatenated sentence, the original query feature of the current sample sentence is determined according to the query weight matrix and the current sample sentence, the original key feature of the current sample sentence is determined according to the first key weight matrix and the concatenated sentence, and the original value feature of the current sample sentence is determined according to the value weight matrix and the concatenated sentence; the original query feature, the original key feature and the original value feature of the current sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the current sample sentence;

[0022] In the second network sublayer, attention transformation is performed according to the sentence hidden state features of the first sample sentence to obtain the sentence semantic representation of the first sample sentence, and taking the second sample sentence as the starting point, attention transformation is performed according to the sentence hidden state features of the previous sample sentence and the sentence hidden state features of the current sample sentence to obtain the sentence semantic representation of the current sample sentence.

[0023] In some embodiments, the first network sublayer further has a second key weight matrix, a content bias matrix, and a position bias matrix, and the attention transformation is performed on the original query feature, the original key feature, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence, including:

[0024] Perform relative position encoding on the first sample sentence to obtain the first sentence position encoding;

[0025] Performing attention calculation according to the second key weight matrix, the content bias matrix, the position bias matrix, the original query feature of the first sample sentence, the original key feature and the first sentence position encoding to obtain an attention score;

[0026] Feature mapping is performed on the first sample sentence, the attention score, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence.

[0027] In some embodiments, the sentence encoding of the sentence semantic representation of the sample text by the second attention sub-model to obtain the sentence encoding feature of each sample sentence includes:

[0028] Performing absolute position encoding on the sentence semantic representation of the sample text to obtain a second sentence position encoding;

[0029] Performing feature fusion on the second sentence position code and the sentence semantic representation of the sample text to obtain a fused semantic feature;

[0030] The fused semantic features are feature encoded to obtain sentence encoding features of each sample sentence.

[0031] In some embodiments, screening the preliminary abstracts according to the sample abstracts to obtain candidate abstracts includes:

[0032] Calculating a second similarity between the sample abstract and the preliminary abstract;

[0033] If the second similarity is greater than a preset similarity threshold, the initially selected abstract is used as the candidate abstract.

[0034] In some embodiments, the step of training a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model includes:

[0035] Performing summary prediction on the candidate summary by using the preset summary generation model to obtain a target summary;

[0036] Perform cross entropy loss calculation based on the target summary and the sample summary to obtain target loss data;

[0037] The model parameters of the preset summary generation model are updated according to the target loss data to obtain the generative summary model.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a text summary generating device, the device comprising:

[0039] A first acquisition module is used to acquire a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence;

[0040] A calculation module, configured to calculate a first similarity between the sample sentence and the summary sentence, select the sample sentence with the largest first similarity as a candidate sentence, and record the number of the candidate sentences;

[0041] A determination module, configured to determine candidate texts according to the candidate sentences if the number of the sentences is equal to the number of the summary sentences in the sample summary;

[0042] A first training module is used to train a preset summary extraction model according to the candidate text to obtain an extractive summary model;

[0043] A summary extraction module, used to extract a summary of the sample text through an extractive summary model to obtain a preliminary summary;

[0044] An abstract screening module, used to screen the preliminary abstracts according to the sample abstracts to obtain candidate abstracts;

[0045] A second training module is used to train a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model;

[0046] The second acquisition module is used to acquire the target text;

[0047] The summary generation module is used to extract the summary of the target text through the extractive summary model to obtain a coarse summary, and to refine the coarse summary through the generative summary model to obtain a target summary.

[0048] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the text summary generation method of the above-mentioned first aspect when executing the computer program.

[0049] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the text summary generation method of the above-mentioned first aspect is implemented.

[0050] The text summary generation method, device, electronic device, and storage medium provided in the embodiment of the present application obtain a sample text and a sample summary of the sample text. The sample text contains sentences that are not related to the field where the sample summary is located. By calculating the first similarity between the sample sentence and the summary sentence, the sample sentence with the largest first similarity is selected as the candidate sentence to extract the sample sentence related to the field where the sample summary is located in the sample text, and remove the sample sentence that is not related to the field of the sample summary, thereby ensuring the accuracy of the sample text content. In order to ensure the integrity of the sample text content, the sample summary covers all the contents of the sample text, and the number of sentences in the candidate sentence is recorded. If the number of sentences is equal to the number of summary sentences in the sample summary, the candidate text is determined according to the candidate sentence, ensuring the balance between the integrity and accuracy of the candidate text in terms of information coverage, and obtaining high-quality text content, thereby improving the accuracy of summary generation. In order to improve the accuracy of summary generation of text in a specific field, the embodiment of the present application combines the extractive summary model and the generative summary model to generate a summary to accurately understand the text semantics of the candidate text. In order to improve the ability of the model to extract key information from the sample text, the preset summary extraction model is trained according to the sample text and the candidate text to obtain the extractive summary model. The sample text is summarized by the extractive summary model to extract the key text information in the sample text and obtain the preliminary summary. The extractive summary model and the generative summary model are heterogeneous models. In order to reduce the sample distribution difference when the heterogeneous models are fused and avoid the cascading errors generated by the extractive summary model from being brought into the next stage, the preliminary summary is screened according to the sample summary so that the preliminary summary is highly correlated with the sample summary to obtain the candidate summary to ensure the data quality of model training. The preset summary generation model is trained according to the candidate summary and the sample summary so that the preset summary generation model can learn more refined text semantics to obtain the generative summary model. The target text is obtained, and the target text is summarized by the extractive summary model to extract the key information in the target text to obtain the coarse summary. The coarse summary is refined by the generative summary model to ensure the accuracy of information extraction and the semantic coherence of the summary, thereby obtaining a more accurate target summary and improving the accuracy of text summary generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flowchart of a text summary generation method provided in an embodiment of the present application;

[0052] Figure 2 yes Figure 1 Flow chart of step S150 in FIG.

[0053] Figure 3 yes Figure 2 Flow chart of step S210 in FIG.

[0054] Figure 4 yes Figure 3 Flow chart of step S310 in FIG.

[0055] Figure 5 yes Figure 2 Flow chart of step S220 in FIG.

[0056] Figure 6 yes Figure 1 Flow chart of step S160 in FIG.

[0057] Figure 7 yes Figure 1 Flowchart of step S170 in FIG.

[0058] Figure 8 It is a structural schematic diagram of a text summary generating device provided in an embodiment of the present application;

[0059] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] Text summarization is to extract key information from a longer text and generate a concise summary. In related technologies, a single extractive summary model or a generative summary model is used to generate text summaries. For texts in specific fields, such as legal texts, the texts cover a variety of closely related topics, are highly professional, have complex structures, and are long. A single extractive summary model or a generative summary model cannot accurately understand the text semantics of texts in specific fields, resulting in low accuracy in text summary generation.

[0064] Based on this, the embodiments of the present application provide a text summary generation method, a text summary generation device, an electronic device and a computer-readable storage medium, aiming to improve the accuracy of text summary generation.

[0065] The text summary generation method, text summary generation device, electronic device and computer-readable storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the text summary generation method in the embodiments of the present application is described.

[0066] The text summary generation method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The text summary generation method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the text summary generation method, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0068] Figure 1 is an optional flowchart of a text summary generation method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S190.

[0069] Step S110, obtaining a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence;

[0070] Step S120, calculating the first similarity between the sample sentence and the summary sentence, selecting the sample sentence with the largest first similarity as the candidate sentence, and recording the number of the candidate sentences;

[0071] Step S130, if the number of sentences is equal to the number of summary sentences in the sample summary, determining candidate texts based on the candidate sentences;

[0072] Step S140, training a preset summary extraction model according to the sample text and the candidate text to obtain an extractive summary model;

[0073] Step S150, extracting a summary of the sample text through an extractive summarization model to obtain a preliminary summary;

[0074] Step S160, screening the preliminary abstracts according to the sample abstracts to obtain candidate abstracts;

[0075] Step S170, training a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model;

[0076] Step S180, obtaining the target text;

[0077] Step S190, extracting a summary of the target text through the extractive summary model to obtain a coarse summary, and refining the coarse summary through the generative summary model to obtain a target summary.

[0078] In step S110 of some embodiments, a sample text and a sample summary are obtained from a data set. The sample text is the text used in the text summary generation task, and the sample summary is a summary of the main content of the sample text. The sample text uses the source language, and the sample summary uses the target language. The source language and the target language can be the same or different. The sample text includes a first number of sample sentences, where the sample sentences are sentences in the sample text, and the first number is the number of sentences included in the sample text. The sample summary includes a second number of summary sentences, where the summary sentences are sentences in the sample summary, and the second number is the number of summary sentences included in the sample summary. Both the first number and the second number are greater than or equal to 1, and the first number is greater than or equal to the second number.

[0079] In step S120 of some embodiments, under the background of rapid development of information technology, text summarization technology has become the main means to cope with the explosion of massive data. Text summarization technology aims to compress lengthy content into concise and accurate text on the basis of retaining the core information of the text. Professionals in various industries generate a large amount of text data every day, and accurate summaries that can meet the needs of specific fields become crucial. Unlike general summaries that compress source texts based on information density, specific field summaries focus more on extracting and compressing the most representative and valuable information in the field. The sample text contains sample sentences that are irrelevant to the field where the sample summary is located. In order to obtain high-quality sample text, the embodiment of the present application is based on the field alignment matching strategy. At the sentence level, sample sentences in the sample text are reversely extracted according to the sample summary to remove sample sentences that are irrelevant to the field from the sample text and retain sample sentences related to the field. In order to extract text fragments that are highly relevant to the content of a specific field, for each sample sentence, the similarity between the sample sentence and each summary sentence is calculated to obtain a first similarity. The larger the first similarity, the more similar the sample sentence and the summary sentence are. The first similarity can be measured by cosine similarity, edit distance, and ROUGE value. The sample sentence with the largest first similarity is selected as the candidate sentence, and the summary sentence with the largest first similarity is selected as the sentence matching the sample sentence. In order to balance the completeness and accuracy of the sample summary in terms of information coverage, each summary sentence has a candidate sentence matching it, and the number of candidate sentences is recorded. It should be noted that there is a one-to-one correspondence between the candidate sentence and the summary sentence, and one summary sentence can only match one candidate sentence. If the sample sentence with the largest first similarity has been matched by other summary sentences, the sample sentence will no longer participate in the next round of matching.

[0080] For example, there are 5 sample sentences and 2 summary sentences. If the first similarity between the first summary sentence and the third sample sentence is the highest, the third sample sentence is used as a candidate sentence to match the first summary sentence. Since the third sample sentence has been matched by the first summary sentence, the third sample sentence is no longer involved in the matching of the second summary sentence.

[0081] In step S130 of some embodiments, if the number of sentences is equal to the number of summary sentences in the sample summary, that is, the second number, the obtained second number of candidate sentences are used as candidate texts to ensure the completeness of the candidate texts in terms of information coverage.

[0082] In specific fields such as the legal field, legal texts often cover diverse and closely related topics, are highly professional, complex in structure and long in length. The automated summary of legal texts must not only ensure the accuracy and factuality of the information, but also generate summaries with fluent content and strong relevance. Current text summarization technology mainly relies on a single extractive summary model or a generative summary model. The model cannot accurately understand the text semantics in a specific field, resulting in incoherent summaries generated using a single extractive summary model and inaccurate summaries generated using a single generative summary model, resulting in low quality of text summary generation. The embodiment of the present application proposes a two-stage fusion method of an extractive summary model and a generative summary model to generate text summaries. By combining the authenticity of the extractive summary model and the fluency of the generative summary model, the quality of text summary generation is improved. The sample text is summarized by the extractive summary model, and key information is extracted from the lengthy text while retaining the original meaning of the text to obtain a preliminary summary.

[0083] In step S140 of some embodiments, the extractive summary model is a model obtained after training a preset summary extraction model based on the sample text and the candidate text. The extractive summary model and the preset summary extraction model have the same model structure but different model parameters. Specifically, the sample text is summarized by the preset summary extraction model to obtain a predicted summary, and the loss of the predicted summary and the candidate text is calculated based on the loss function to obtain a loss value. The model parameters of the preset summary extraction model are adjusted to minimize the loss value to obtain the extractive summary model, so as to accurately extract key text information from the sample text through the extractive summary model.

[0084] See also Figure 2 In some embodiments, the extractive summarization model includes a first attention sub-model, a second attention sub-model, and a classification sub-model, and the first attention sub-model, the second attention sub-model, and the classification sub-model are cascaded. Step S150 may include, but is not limited to, steps S210 to S240:

[0085] Step S210, performing semantic analysis on the sample text through the first attention sub-model to obtain sentence semantic representation of the sample text;

[0086] Step S220, performing sentence encoding on the sentence semantic representation of the sample text through the second attention sub-model to obtain sentence encoding features of each sample sentence;

[0087] Step S230, for each sample sentence, classify the sentence encoding features through the classification sub-model to obtain a sentence importance category; the sentence importance category is used to indicate whether the sample sentence is important or not;

[0088] Step S240, screening the sample text according to important sentence categories to obtain a preliminary summary.

[0089] In step S210 of some embodiments, the sample text is encoded at the character level to obtain the encoded text. The first attention sub-model includes at least two network sub-layers, and the encoded text is semantically analyzed by the first attention sub-model until the output of the last network sub-layer is obtained to obtain the sentence semantic representation of each sample sentence.

[0090] The process of character-level encoding is as follows:

[0091] Assume that the sample text containing m sample sentences is represented as S = {s1,...,s i ,...,s m}, the i-th sample sentence containing t characters is the jth character of the i-th sample sentence. Add at the end of each sample sentence <sep>delimiter and add it at the end of the sample text <cls>delimiter, the encoded text is represented as {s1 <sep> ...s i <sep> ...s m <sep> <cls>}.

[0092] In step S220 of some embodiments, the sentence semantic representations of all sample sentences are sentence encoded by the second attention sub-model to promote the interaction of information between sentences, obtain sentence-level contextual information, and obtain sentence encoding features for each sample sentence. The sentence encoding feature is a sentence representation containing rich contextual information. The text in a specific field is long, and key information is scattered throughout the text. Through two-layer encoding at the character level and sentence level, the accuracy of local information extraction and the coherence of global semantics are guaranteed, so that the extractive summary model can process long texts, so that the generated summary has integrity and consistency.

[0093] In step S230 of some embodiments, the sentence encoding features of each candidate sentence are respectively input into the classification sub-model to perform binary classification on the importance of the candidate sentence, judge the importance of the candidate sentence in the candidate text, and obtain the sentence importance category of each sample sentence. The sentence importance category includes important or unimportant, which is used to indicate that the sample sentence is important or the sample sentence is unimportant. The classification sub-model can use a multi-layer perceptron. If the output value of the classification sub-model is 1, it means that the sample sentence is important; if the output value of the classification sub-model is 0, it means that the sample sentence is unimportant.

[0094] In step S240 of some embodiments, sample sentences whose important categories are important are screened out from the sample text to obtain a preliminary summary.

[0095] Through the above steps S210 to S240, the semantic relevance between sentences can be effectively improved, and the complete semantic encoding of the long text can be ensured, so as to extract key information from the long text and provide support for generating high-quality text summaries.

[0096] See also Figure 3 In some embodiments, the first attention sub-model includes a first network sub-layer and a second network sub-layer, the first network sub-layer has a query weight matrix, a first key weight matrix and a value weight matrix, and step S210 may include but is not limited to steps S310 to S330:

[0097] Step S310: at the first network sublayer, determining the original query feature of the first sample sentence according to the query weight matrix and the first sample sentence, determining the original key feature of the first sample sentence according to the first key weight matrix and the first sample sentence, and determining the original value feature of the first sample sentence according to the value weight matrix and the first sample sentence; performing attention transformation on the original query feature, the original key feature, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence;

[0098] Step S320: In the first network sublayer, starting from the second sample sentence, concatenate the previous sample sentence and the current sample sentence to obtain a concatenated sentence, determine the original query feature of the current sample sentence according to the query weight matrix and the current sample sentence, determine the original key feature of the current sample sentence according to the first key weight matrix and the concatenated sentence, and determine the original value feature of the current sample sentence according to the value weight matrix and the concatenated sentence; perform attention transformation on the original query feature, original key feature, and original value feature of the current sample sentence to obtain the sentence hidden state feature of the current sample sentence;

[0099] Step S330, in the second network sublayer, attention transformation is performed according to the sentence hidden state features of the first sample sentence to obtain the sentence semantic representation of the first sample sentence, and taking the second sample sentence as the starting point, attention transformation is performed according to the sentence hidden state features of the previous sample sentence and the sentence hidden state features of the current sample sentence to obtain the sentence semantic representation of the current sample sentence.

[0100] In step S310 of some embodiments, at the first network sublayer, the query weight matrix is ​​multiplied by the first sample sentence to obtain the original query feature of the first sample sentence, the first key weight matrix is ​​multiplied by the first sample sentence to obtain the original key feature of the first sample sentence, and the value weight matrix is ​​multiplied by the first sample sentence to obtain the original value feature of the first sample sentence. The original query feature, the original key feature, and the original value feature of the first sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the first sample sentence.

[0101] In step S320 of some embodiments, at the first network sublayer, starting from the second sample sentence, the previous sample sentence and the current sample sentence are concatenated to obtain a concatenated sentence, the query weight matrix is ​​multiplied by the current sample sentence to obtain the original query feature of the current sample sentence, the first key weight matrix is ​​multiplied by the concatenated sentence to obtain the original key feature of the current sample sentence, and the value weight matrix is ​​multiplied by the concatenated sentence to obtain the original value feature of the current sample sentence. The original query feature, the original key feature, and the original value feature of the current sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the current sample sentence. The above steps are repeated until the sentence hidden state feature of the last sample sentence is obtained.

[0102] In step S330 of some embodiments, the second network sublayer also has a query weight matrix, a first key weight matrix, and a value weight matrix. The query weight matrix, the first key weight matrix, and the value weight matrix of the second network sublayer are respectively multiplied with the sentence hidden state features of the first sample sentence to obtain the reference query features, reference key features, and reference value features of the first sample sentence, and the reference query features, reference key features, and reference value features of the first sample sentence are subjected to attention transformation to obtain the sentence semantic representation of the first sample sentence.

[0103] Taking the second sample sentence as the starting point, the sentence hidden state features of the previous sample sentence and the sentence hidden state features of the current sample sentence are concatenated to obtain the state concatenation features. Through the concatenation operation, the hidden state of the previous sample sentence can be introduced as an additional input into the attention mechanism of the current sample sentence, ensuring the contextual consistency across text sentences. The query weight matrix of the second network sublayer is multiplied with the sentence hidden state features of the current sample sentence to obtain the reference query features of the current sample sentence. The first key weight matrix and the value weight matrix of the second network sublayer are respectively multiplied with the state concatenation features to obtain the reference key features and reference value features of the current sample sentence. Attention transformation is performed based on the sentence hidden state features, reference query features, reference key features, and reference value features of the current sample sentence to obtain the sentence semantic representation of the current sample sentence, until the sentence semantic representation of the last sample sentence is obtained.

[0104] Through the above steps S310 to S330, the semantic relevance between different sample sentences in a long text can be captured more accurately, ensuring the contextual consistency across text fragments, thereby ensuring the accuracy and fluency of summary generation and avoiding the loss and fragmentation of text information caused by truncation methods or block methods.

[0105] See also Figure 4 In some embodiments, the first network sublayer further has a second key weight matrix, a content bias matrix, and a position bias matrix. Step S310 may include but is not limited to steps S410 to S430:

[0106] Step S410, performing relative position coding on the first sample sentence to obtain a first sentence position code;

[0107] Step S420, performing attention calculation according to the second key weight matrix, the content bias matrix, the position bias matrix, the original query feature of the first sample sentence, the original key feature and the first sentence position encoding to obtain an attention score;

[0108] Step S430, feature mapping is performed on the first sample sentence, the attention score, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence.

[0109] In step S410 of some embodiments, existing extractive summarization models usually limit the length of input data. For example, the BERT model can only encode 512 characters and cannot handle longer texts. In order to ensure the complete semantic encoding of long texts, the embodiments of the present application model the global relevance of sample texts from the sentence level, so that the extractive summarization model can learn key information that is highly relevant to the summary content from the complete long text.

[0110] The sample sentence includes multiple characters, each of which has a character index, which is used to indicate the position of the character in the sample sentence. If the jth character of the sample sentence is Then j is the character index of the character. Calculate the absolute value of the difference between the character indexes of any two characters in the sample sentence to obtain the target index. For example, if the character index of the jth character is j and the character index of the qth character is q, then the target index is |jq|. Get the encoding dimension. If the encoding dimension is an even number, encode the absolute value and the encoding dimension according to the sine function. If the encoding dimension is an odd number, encode the character index and the encoding dimension according to the cosine function until the encoding of the last character is completed, and the first sentence position encoding is obtained. The sine function is expressed as:

[0111]

[0112] Among them, pos represents the target index, i represents the encoding dimension, and d m Represents the feature dimension of the first sentence position encoding.

[0113] The cosine function is expressed as:

[0114]

[0115] In step S420 of some embodiments, a first sub-score is determined according to the original query feature and the original key feature of the first sample sentence, a second sub-score is determined according to the original query feature of the first sample sentence, the second key weight matrix and the first sentence position code, a third sub-score is determined according to the global content bias matrix and the original key feature of the first sample sentence, a fourth sub-score is determined according to the global position bias matrix, the second key weight matrix and the first sentence position code, and the first sub-score, the second sub-score, the third sub-score and the fourth sub-score are summed to obtain an attention score. The first sub-score, the second sub-score, the third sub-score and the fourth sub-score are used to comprehensively consider the content relevance and relative position relationship between the characters of the sentence, thereby improving the model's ability to understand the text.

[0116] The attention score is calculated as follows:

[0117]

[0118] Among them, A represents the attention score; T represents the transposition operation; q1 represents the original query feature of the first sample sentence; k1 represents the original key feature of the first sample sentence; represents the first sub-fraction; W k,R represents the second key weight matrix; R represents the first sentence position encoding; represents the second sub-score; u represents the content bias matrix; u T k1 represents the third sub-score; v represents the position bias matrix; v T W k,R R represents the fourth sub-fraction.

[0119] In step S430 of some embodiments, the attention score is activated by a softmax function point to obtain an activated feature, and the activated feature is linearized by a linear layer to obtain a linear feature. The linear feature is fused with the first sample sentence, and the fused feature is normalized to obtain a normalized feature. The normalized feature is feed-forwarded by a feed-forward network layer to obtain a sentence hidden state feature of the first sample sentence.

[0120] It should be noted that, referring to steps S410 to S430, attention transformation is performed according to the sentence hidden state feature, reference query feature, reference key feature and reference value feature of the current sample sentence to obtain the sentence semantic representation of the current sample sentence. The specific calculation process of the sentence semantic representation is as follows:

[0121]

[0122] a τ =softmax(A τ )v τ ,

[0123]

[0124] Among them, τ represents the τth sample sentence; T represents the transposition operation; A τ represents the attention score of the hidden state feature of the sentence of the τth sample sentence; q τ represents the reference query feature; k τ represents a reference key feature; represents the second key weight matrix of the second network sublayer; R τ represents the relative position encoding of the τth sample sentence; u 2 represents the global content bias matrix of the second network sublayer; v 2 represents the global position bias matrix of the second network sublayer; v τ Indicates reference value characteristics; is the hidden state feature of the sentence of the τth sample sentence; a τ , o τ is an intermediate variable; is the sentence semantic representation of the τth sample sentence.

[0125] Through the above steps S410 to S430, key information can be accurately extracted from a long text without being limited by the length of the long text.

[0126] See also Figure 5 In some embodiments, step S220 may include but is not limited to steps S510 to S530:

[0127] Step S510, performing absolute position coding on the sentence semantic representation of the sample text to obtain a second sentence position coding;

[0128] Step S520, performing feature fusion on the second sentence position code and the sentence semantic representation of the sample text to obtain a fused semantic feature;

[0129] Step S530, feature encoding is performed on the fused semantic features to obtain sentence encoding features of each sample sentence.

[0130] In step S510 of some embodiments, the second attention sub-model includes a position encoding layer and multiple feature encoding layers. The feature encoding layer is an encoder based on the transformer architecture, and the number of feature encoding layers can be 12. The network structure of each feature encoding layer is the same, including a multi-head self-attention mechanism, a normalization layer, and a feedforward network. The sentence semantics of all sample sentences are represented as SR={p1, p2,..., pm}, and the sentence semantic representations of all sample sentences can be absolutely position encoded through the position encoding layer to obtain global position information and obtain the second sentence position encoding.

[0131] In step S520 of some embodiments, the second sentence position code and the sentence semantic representation of all sample sentences are feature added to obtain the fused semantic features of the sample text. By integrating the sentence semantic representation into the absolute position code information, effective modeling of the context between sentences can be achieved.

[0132] In step S530 of some embodiments, the fused semantic features are randomly inactivated by dropout technology to obtain input features. The input features are feature encoded by multiple feature coding layers to promote the interaction of information between sentences, thereby obtaining sentence-level contextual information. Specifically, the input features are feature encoded by the first feature coding layer to obtain the output features of the first feature coding layer. Starting from the second feature coding layer, the output features of all feature coding layers before the current feature coding layer are feature concatenated to obtain the input features of the current feature coding layer, and the input features are feature encoded by the current feature coding layer to obtain the output features of the current feature coding layer, until the output features of the last feature coding layer are obtained, and the output features of the last feature coding layer are used as sentence coding features.

[0133] The feature encoding process of each feature encoding layer is the same. The multi-head self-attention mechanism can be used to perform multi-head attention calculation on the input features of the sample sentence to obtain the attention feature, and the attention feature and the input feature are added to obtain the first fusion feature. The first fusion feature is normalized through the normalization layer to obtain the normalized feature. The normalized feature is extracted through the feedforward network to obtain the feedforward feature, and the feedforward feature and the normalized feature are added to obtain the second fusion feature. The second fusion feature is normalized to obtain the output feature. The calculation method of the output feature of the zth feature encoding layer is as follows:

[0134]

[0135] Among them, MulHAtt is multi-head attention calculation; p i is the input feature of the i-th sample sentence of the z-th feature encoding layer; and They are the key vector, query vector, and value vector corresponding to the input features respectively; For attention characteristics; is the normalized feature; FF represents the feedforward network; Represents the output features.

[0136] See also Figure 6 In some embodiments, step S160 may include but is not limited to steps S610 to S620:

[0137] Step S610, calculating the second similarity between the sample abstract and the preliminary abstract;

[0138] Step S620: If the second similarity is greater than a preset similarity threshold, the initially selected abstract is used as a candidate abstract.

[0139] In step S610 of some embodiments, if the key information extracted by the extractive summary model in the first stage is inaccurate, it may introduce noise and hinder the learning process of the generative summary model in the second stage. The generative summary model will generate a target summary based on the incorrectly extracted sentences, resulting in an inaccurate target summary. If only the standard answer is used as the input of the second stage and teacher-forcing learning is performed, it may cause a difference in the reasoning process between the inaccurate information extracted by the first stage model and the standard summary learned in the second stage. In order to reduce the sample distribution difference of training-reasoning when the two-stage heterogeneous model is fused, and to avoid the cascading errors generated by the first stage model from being brought into the second stage model training process, the embodiment of the present application filters the preliminary summary of each sample text to ensure the integrity and relevance of the summary information, so as to improve the training sample quality of the generative summary model.

[0140] Specifically, the similarity between the sample abstract and the preliminary abstract is calculated to obtain a second similarity. The second similarity is calculated using a metric based on a text semantic similarity matching mechanism, such as ROUGE value, cosine similarity, edit distance, etc. The larger the second similarity, the more similar the sample abstract and the preliminary abstract are.

[0141] In step S620 of some embodiments, if the second similarity is greater than a preset similarity threshold, the preliminary abstract is used as a candidate abstract. If the second similarity is less than or equal to the preset similarity threshold, the preliminary abstract is removed and is not used as a sample for model training. The preset similarity threshold may be 0.5.

[0142] Through the above steps S610 to S620, high-quality samples with high semantic similarity to the sample summary can be screened out, ensuring the relevance of the two-stage training data, thereby improving the quality of summary generation.

[0143] See also Figure 7 In some embodiments, step S170 may include but is not limited to steps S710 to S730:

[0144] Step S710, performing summary prediction on the candidate summary by using a preset summary generation model to obtain a target summary;

[0145] Step S720, performing cross entropy loss calculation based on the target summary and the sample summary to obtain target loss data;

[0146] Step S730, updating the model parameters of the preset summary generation model according to the target loss data to obtain a generative summary model.

[0147] In step S710 of some embodiments, the preset summary generation model may adopt the T5 model, and the preset summary generation model includes an encoder and a decoder. The encoder performs summary encoding on the candidate summary to obtain a summary encoding representation vector. The summary encoding representation vector is a hidden state generated by the encoder. The word sequence output by the decoder before the current time step is obtained, and the decoder performs feature decoding on the summary encoding representation vector and the word sequence to obtain the target summary. If the current time step is the initial time step, the word sequence can be a zero vector. The feature decoding process is expressed as:

[0148] y t =Decoder(y ＜t ,h),

[0149] Among them, y t is the tth word in the generated target summary, y ＜t is the word sequence before the currently generated t-th word, and h is the summary encoding representation vector.

[0150] In step S720 of some embodiments, in order to minimize the difference between the target summary and the sample summary, a cross-entropy-based loss function is used to optimize the preset summary generation model. The loss function is defined as:

[0151]

[0152] Among them, L represents the target loss data; N is the number of sample summaries; T is the summary length of the target summary; git represents the true label of the tth word in the i-th sample summary; pit represents the probability that the tth word output by the preset summary generation model is the tth word in the sample summary; * represents a multiplication operation.

[0153] In step S730 of some embodiments, the model parameters of the preset summary generation model are updated. When the target loss data reaches the minimum, the training round reaches a preset number threshold, or the target loss data of adjacent iteration rounds does not change much, the preset summary generation model at this time is used as the generative summary model.

[0154] In the above steps S710 to S730, by selecting high-quality samples for model training, the model performance of the preset summary generation model is ensured, and the accuracy and fluency of summary generation are improved.

[0155] In step S180 of some embodiments, a target text is obtained from a data set, where the target text is the text for which a summary is to be generated.

[0156] In step S190 of some embodiments, a target text is abstracted by an extractive abstraction model to obtain a coarse abstract. Alternatively, the target text is abstracted by multiple extractive abstraction models to obtain multiple abstracts; the multiple abstracts are compared, and the multiple abstracts are voted, and the abstract with the most occurrences is selected as the coarse abstract. The coarse abstract is refined by the generative abstraction model to obtain the target abstract.

[0157] The embodiment of the present application performs model performance tests on two legal-related corpora, the CAIL 2022 legal public opinion summary dataset and the BillSum test dataset, and compares the text summary generation method of the embodiment of the present application with multiple baseline models. The embodiment of the present application shows significant advantages in terms of relevance, coherence, and comprehensiveness of information. The experimental results are shown in Tables 1 to 3. Table 1 shows the experimental results of each model on the CAIL 2022 dataset, Table 2 shows the experimental results of each model on the BillSum dataset, and Table 3 shows the average coherence score and average accuracy score of GPT4 for evaluating the output summary of the embodiment of the present application.

[0158]

[0159] Table 1

[0160]

[0161]

[0162] Table 2

[0163]

[0164] Table 3

[0165] See also Figure 8 The embodiment of the present application also provides a text summary generation device, which can implement the above-mentioned text summary generation method, and the text summary generation device includes:

[0166] The first acquisition module 810 is used to acquire a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence;

[0167] A calculation module 820 is used to calculate the first similarity between the sample sentence and the summary sentence, select the sample sentence with the largest first similarity as the candidate sentence, and record the number of candidate sentences;

[0168] A determination module 830 is used to determine candidate texts based on the candidate sentences if the number of sentences is equal to the number of summary sentences in the sample summary;

[0169] A first training module 840 is used to train a preset summary extraction model according to the sample text and the candidate text to obtain an extractive summary model;

[0170] A summary extraction module 850 is used to extract a summary of the sample text through an extractive summary model to obtain a preliminary summary;

[0171] An abstract screening module 860 is used to screen the preliminary abstracts according to the sample abstracts to obtain candidate abstracts;

[0172] A second training module 870 is used to train a preset summary generation model according to the candidate summaries and the sample summaries to obtain a generative summary model;

[0173] The second acquisition module 880 is used to acquire the target text;

[0174] The summary generation module 890 is used to extract a summary of the target text through an extractive summary model to obtain a coarse summary, and to refine the coarse summary through a generative summary model to obtain a target summary.

[0175] The specific implementation of the text summary generation device is basically the same as the specific implementation of the above-mentioned text summary generation method, and will not be repeated here.

[0176] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above text summary generation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0177] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0178] The processor 910 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0179] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 920, and the processor 910 calls and executes the text summary generation method of the embodiment of this application;

[0180] Input / output interface 930, used to implement information input and output;

[0181] Communication interface 940, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0182] bus 950 , which transmits information between the various components of the device (e.g., processor 910 , memory 920 , input / output interface 930 , and communication interface 940 );

[0183] The processor 910 , the memory 920 , the input / output interface 930 , and the communication interface 940 are connected to each other in communication within the device via a bus 950 .

[0184] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned text summary generation method is implemented.

[0185] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0186] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0187] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0188] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0190] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0191] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0192] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0193] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0196] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.< / cls> < / sep> < / sep> < / sep> < / cls> < / sep>

Claims

1. A text summary generation method, characterized in that: The method comprises: Acquire a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence; Calculating a first similarity between the sample sentence and the summary sentence, selecting the sample sentence with the largest first similarity as a candidate sentence, and recording the number of the candidate sentences; If the number of the sentences is equal to the number of the summary sentences in the sample summary, determining a candidate text according to the candidate sentences; Training a preset summary extraction model according to the sample text and the candidate text to obtain an extractive summary model; Extracting a summary of the sample text by using an extractive summarization model to obtain a preliminary summary; Screening the preliminary abstracts according to the sample abstracts to obtain candidate abstracts; Training a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model; Get the target text; The target text is summarized and extracted by the extractive summary model to obtain a coarse summary, and the coarse summary is refined by the generative summary model to obtain a target summary.

2. The method according to claim 1, characterized in that The extractive summarization model includes a first attention sub-model, a second attention sub-model and a classification sub-model. The extractive summarization model is used to extract a summary of the sample text to obtain a preliminary summary, including: Performing semantic analysis on the sample text by using the first attention sub-model to obtain sentence semantic representation of the sample text; Perform sentence encoding on the sentence semantic representation of the sample text by using the second attention sub-model to obtain sentence encoding features of each sample sentence; For each of the sample sentences, the sentence encoding features are classified by the classification sub-model to obtain a sentence importance category; the sentence importance category is used to indicate that the sample sentence is important or the sample sentence is not important; The sample text is screened according to the important categories of the sentences to obtain the preliminary summary.

3. The method according to claim 2, characterized in that The first attention sub-model includes a first network sub-layer and a second network sub-layer, the first network sub-layer has a query weight matrix, a first key weight matrix and a value weight matrix, and the semantic analysis of the sample text by the first attention sub-model is performed to obtain the sentence semantic representation of the sample text, including: In the first network sublayer, an original query feature of the first sample sentence is determined according to the query weight matrix and the first sample sentence, an original key feature of the first sample sentence is determined according to the first key weight matrix and the first sample sentence, and an original value feature of the first sample sentence is determined according to the value weight matrix and the first sample sentence; an attention transformation is performed on the original query feature, the original key feature, and the original value feature of the first sample sentence to obtain a sentence hidden state feature of the first sample sentence; In the first network sublayer, starting from the second sample sentence, the previous sample sentence and the current sample sentence are concatenated to obtain a concatenated sentence, the original query feature of the current sample sentence is determined according to the query weight matrix and the current sample sentence, the original key feature of the current sample sentence is determined according to the first key weight matrix and the concatenated sentence, and the original value feature of the current sample sentence is determined according to the value weight matrix and the concatenated sentence; the original query feature, the original key feature and the original value feature of the current sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the current sample sentence; In the second network sublayer, attention transformation is performed according to the sentence hidden state features of the first sample sentence to obtain the sentence semantic representation of the first sample sentence, and taking the second sample sentence as the starting point, attention transformation is performed according to the sentence hidden state features of the previous sample sentence and the sentence hidden state features of the current sample sentence to obtain the sentence semantic representation of the current sample sentence.

4. The method according to claim 3, characterized in that The first network sublayer also has a second key weight matrix, a content bias matrix, and a position bias matrix. The attention transformation is performed on the original query feature, the original key feature, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence, including: Perform relative position encoding on the first sample sentence to obtain the first sentence position encoding; Performing attention calculation according to the second key weight matrix, the content bias matrix, the position bias matrix, the original query feature of the first sample sentence, the original key feature and the first sentence position encoding to obtain an attention score; Feature mapping is performed on the first sample sentence, the attention score, and the original value feature of the first sample sentence to obtain the sentence hidden state feature of the first sample sentence.

5. The method according to claim 2, characterized in that: The sentence encoding of the sentence semantic representation of the sample text by the second attention sub-model to obtain the sentence encoding features of each sample sentence includes: Performing absolute position coding on the sentence semantic representation of the sample text to obtain a second sentence position coding; Performing feature fusion on the second sentence position code and the sentence semantic representation of the sample text to obtain a fused semantic feature; The fused semantic features are feature encoded to obtain sentence encoding features of each sample sentence.

6. The method according to any one of claims 1 to 5, characterized in that: The step of screening the preliminary abstracts according to the sample abstracts to obtain candidate abstracts includes: Calculating a second similarity between the sample abstract and the preliminary abstract; If the second similarity is greater than a preset similarity threshold, the initially selected abstract is used as the candidate abstract.

7. The method according to any one of claims 1 to 5, characterized in that: The step of training a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model includes: Performing summary prediction on the candidate summary by using the preset summary generation model to obtain a target summary; Perform cross entropy loss calculation based on the target summary and the sample summary to obtain target loss data; The model parameters of the preset summary generation model are updated according to the target loss data to obtain the generative summary model.

8. A text summary generation device, characterized in that: The device comprises: A first acquisition module is used to acquire a sample text and a sample summary of the sample text; the sample text includes a sample sentence, and the sample summary includes a summary sentence; A calculation module, configured to calculate a first similarity between the sample sentence and the summary sentence, select the sample sentence with the largest first similarity as a candidate sentence, and record the number of the candidate sentences; A determination module, configured to determine candidate texts according to the candidate sentences if the number of the sentences is equal to the number of the summary sentences in the sample summary; A first training module, used for training a preset summary extraction model according to the sample text and the candidate text to obtain an extractive summary model; A summary extraction module, used to extract a summary of the sample text through an extractive summary model to obtain a preliminary summary; An abstract screening module, used to screen the preliminary abstracts according to the sample abstracts to obtain candidate abstracts; A second training module is used to train a preset summary generation model according to the candidate summary and the sample summary to obtain a generative summary model; The second acquisition module is used to acquire the target text; The summary generation module is used to extract the summary of the target text through the extractive summary model to obtain a coarse summary, and to refine the coarse summary through the generative summary model to obtain a target summary.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images