Text abstract generation method and device, electronic equipment and storage medium

Abstract

The application provides a text abstract generation method and device, electronic equipment and storage medium, and belongs to the technical field of artificial intelligence. The first similarity between a sample sentence and an abstract sentence is calculated, the sample sentence with the maximum first similarity is selected as a candidate sentence, if the number of candidate sentences is equal to the number of abstract sentences, candidate texts are determined according to the candidate sentences, an extractive abstract model is obtained by training according to the candidate texts, the sample texts are abstracted by the extractive abstract model, and a preliminary abstract is obtained. The preliminary abstract is screened according to the sample abstract, and a candidate abstract is obtained. The generative abstract model is obtained by training according to the candidate abstract and the sample abstract. The target text is abstracted by the extractive abstract model, and a rough abstract is obtained. The target abstract is obtained by refining the rough abstract by the generative abstract model, and the accuracy of text abstract generation can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a text abstract generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] Text abstract generation is to extract key information from a long text and generate a concise abstract. In related technologies, a single extractive abstract model or generative abstract model is used for text abstract generation. For a text in a specific field such as a legal text, the text covers multiple and closely related topics, the text is highly professional and complex in structure, and the text is long. The single extractive abstract model or generative abstract model cannot accurately understand the text semantics of the text in the specific field, so that the accuracy of text abstract generation is low. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a text abstract generation method, device, electronic equipment and storage medium, which aims to improve the accuracy of text abstract generation.

[0004] To achieve the above purpose, a first aspect of the embodiments of the present application provides a text abstract generation method, which comprises:

[0005] obtaining a sample text and a sample abstract of the sample text; the sample text comprises sample sentences, and the sample abstract comprises abstract sentences;

[0006] calculating a first similarity between the sample sentences and the abstract sentences, selecting a sample sentence with the maximum first similarity as a candidate sentence, and recording the number of sentences of the candidate sentence;

[0007] if the number of sentences is equal to the number of abstract sentences in the sample abstract, determining a candidate text according to the candidate sentence;

[0008] training a preset abstract extraction model according to the candidate text to obtain an extractive abstract model;

[0009] extracting an abstract from the sample text by using the extractive abstract model to obtain a preliminary selected abstract;

[0010] selecting an abstract from the preliminary selected abstract according to the sample abstract to obtain a candidate abstract;

[0011] training a preset abstract generation model according to the candidate abstract and the sample abstract to obtain a generative abstract model;

[0012] obtaining a target text;

[0013] The target text is summarized by the extractive summary model to obtain a rough summary, and the rough summary is refined by the generative summary model to obtain a target summary.

[0014] In some embodiments, the extractive summary model includes a first attention sub-model, a second attention sub-model, and a classification sub-model, and the extractive summary model is used to summarize the sample text to obtain a preliminary summary, including:

[0015] The first attention sub-model is used to analyze the semantics of the sample text to obtain a sentence semantic representation of the sample text;

[0016] The second attention sub-model is used to encode the sentence semantic representation of the sample text to obtain a sentence encoding feature of each sample sentence;

[0017] For each sample sentence, the classification sub-model is used to classify the sentence encoding feature to obtain a sentence importance category; the sentence importance category is used to indicate that the candidate sentence is important or not important;

[0018] The sample text is filtered according to the sentence importance category to obtain the preliminary summary.

[0019] In some embodiments, the first attention sub-model includes a first network sub-layer and a second network sub-layer, the first network sub-layer has a query weight matrix, a first key weight matrix, and a value weight matrix, and the first attention sub-model is used to analyze the semantics of the sample text to obtain a sentence semantic representation of the sample text, including:

[0020] In the first network sub-layer, the query weight matrix and the first sample sentence are used to determine the original query feature of the first sample sentence, the first key weight matrix and the first sample sentence are used to determine the original key feature of the first sample sentence, and the value weight matrix and the first sample sentence are used to determine the original value feature of the first sample sentence; the original query feature, the original key feature, and the original value feature of the first sample sentence are transformed by attention to obtain a sentence hidden state feature of the first sample sentence;

[0021] In the first network sub-layer, starting with the second sample sentence, the previous sample sentence and the current sample sentence are concatenated to obtain a concatenated sentence. The original query features of the current sample sentence are determined based on the query weight matrix and the current sample sentence. The original key features of the current sample sentence are determined based on the first key weight matrix and the concatenated sentence. The original value features of the current sample sentence are determined based on the value weight matrix and the concatenated sentence. Attention transformation is applied to the original query features, original key features, and original value features of the current sample sentence to obtain the sentence hidden state features of the current sample sentence.

[0022] In the second sub-layer of the network, attention transformation is performed based on the hidden state features of the first sample sentence to obtain the semantic representation of the first sample sentence. Starting from the second sample sentence, attention transformation is performed based on the hidden state features of the previous sample sentence and the hidden state features of the current sample sentence to obtain the semantic representation of the current sample sentence.

[0023] In some embodiments, the first network sublayer further includes a second key weight matrix, a content bias matrix, and a position bias matrix. The attention transformation performed on the original query features, original key features, and original value features of the first sample sentence to obtain the sentence hidden state features of the first sample sentence includes:

[0024] The relative position encoding of the first sample sentence is performed to obtain the position encoding of the first sentence;

[0025] Attention is calculated based on the second key weight matrix, the content bias matrix, the position bias matrix, the original query features of the first sample sentence, the original key features, and the position code of the first sentence to obtain an attention score;

[0026] Feature mapping is performed on the first sample sentence, the attention score, and the original value features of the first sample sentence to obtain the sentence hidden state features of the first sample sentence.

[0027] In some embodiments, the step of encoding the sentence semantic representation of the sample text using the second attention sub-model to obtain the sentence encoding features of each sample sentence includes:

[0028] The absolute positional encoding of the sentence semantic representation of the sample text is performed to obtain the second sentence positional encoding;

[0029] The second sentence position encoding and the sentence semantic representation of the sample text are fused to obtain fused semantic features;

[0030] The fused semantic features are then encoded to obtain the sentence encoding features for each sample sentence.

[0031] In some embodiments, the abstract screening of the primary abstract according to the sample abstract obtains a candidate abstract, including:

[0032] calculating a second similarity between the sample abstract and the primary abstract;

[0033] if the second similarity is greater than a preset similarity threshold, taking the primary abstract as the candidate abstract.

[0034] In some embodiments, the training of the preset abstract generation model according to the candidate abstract and the sample abstract obtains a generative abstract model, including:

[0035] the abstract prediction of the candidate abstract by the preset abstract generation model obtains a target abstract;

[0036] the cross-entropy loss calculation according to the target abstract and the sample abstract obtains target loss data;

[0037] the updating of model parameters of the preset abstract generation model according to the target loss data obtains the generative abstract model.

[0038] To achieve the above object, a second aspect of the embodiment of the present application proposes a text abstract generation device, which comprises:

[0039] a first acquisition module configured to acquire a sample text and a sample abstract of the sample text; the sample text comprises sample sentences, and the sample abstract comprises abstract sentences;

[0040] a calculation module configured to calculate a first similarity between the sample sentences and the abstract sentences, select a sample sentence with the largest first similarity as a candidate sentence, and record a number of sentences of the candidate sentence;

[0041] a determination module configured to determine a candidate text according to the candidate sentence if the number of sentences is equal to a number of abstract sentences in the sample abstract;

[0042] a first training module configured to train a preset abstract extraction model according to the candidate text to obtain an extractive abstract model;

[0043] an abstract extraction module configured to perform abstract extraction on the sample text by the extractive abstract model to obtain a primary abstract;

[0044] an abstract screening module configured to perform abstract screening on the primary abstract according to the sample abstract to obtain a candidate abstract;

[0045] a second training module configured to train a preset abstract generation model according to the candidate abstract and the sample abstract, to obtain a generative abstract model;

[0046] a second obtaining module configured to obtain a target text;

[0047] an abstract generation module configured to perform abstract extraction on the target text by using the extractive abstract model to obtain a coarse abstract, and perform abstract refinement on the coarse abstract by using the generative abstract model to obtain a target abstract.

[0048] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the text abstract generation method of the first aspect when executing the computer program.

[0049] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the text abstract generation method of the first aspect.

[0050] The text summarization method, apparatus, electronic device, and storage medium provided in this application acquire sample text and sample summaries of the sample text. The sample text contains sentences unrelated to the domain of the sample summaries. By calculating the first similarity between the sample sentences and the summary sentences, the sample sentence with the highest first similarity is selected as a candidate sentence to extract sample sentences related to the domain of the sample summaries from the sample text and eliminate sample sentences unrelated to the domain of the sample summaries, thus ensuring the accuracy of the sample text content. To ensure the completeness of the sample text content and make the sample summaries cover all the content of the sample text, the number of candidate sentences is recorded. If the number of candidate sentences is equal to the number of summary sentences in the sample summaries, then candidate text is determined based on the candidate sentences, ensuring a balance between completeness and accuracy in information coverage of the candidate text, resulting in high-quality text content and thus improving the accuracy of summary generation. To improve the accuracy of summarization generation for texts in specific domains, this application combines an extractive summarization model and a generative summarization model for summary generation to accurately understand the textual semantics of the candidate texts. To enhance the model's ability to extract key information from sample texts, a preset summary extraction model is trained based on the sample texts and candidate texts to obtain an extractive summarization model. Extractive summarization models are used to extract summaries from sample texts to obtain key textual information, resulting in preliminary summaries. Since the extractive and generative summarization models are heterogeneous, to mitigate sample distribution differences during model fusion and prevent cascading errors from the extractive summarization model from being carried over to the next stage, the preliminary summaries are filtered based on the sample summaries to ensure high relevance between the preliminary and sample summaries, resulting in candidate summaries and guaranteeing data quality for model training. A pre-defined summarization model is trained using both candidate and sample summaries, enabling it to learn more refined textual semantics, resulting in a generative summarization model. The target text is then acquired, and the extractive summarization model extracts key information to obtain a coarse summary. The generative summarization model then refines this coarse summary, ensuring accuracy in information extraction and semantic coherence, thus yielding a more accurate target summary and improving the overall accuracy of text summarization generation. Attached Figure Description

[0051] Figure 1 This is a flowchart of the text digest generation method provided in the embodiments of this application;

[0052] Figure 2 yes Figure 1 The flowchart of step S150 in the middle;

[0053] Figure 3 yes Figure 2 The flowchart of step S210 in the middle;

[0054] Figure 4 is a flowchart of step S310 in Figure 3

[0055] Figure 5 is a flowchart of step S220 in Figure 2

[0056] Figure 6 is a flowchart of step S160 in Figure 1

[0057] Figure 7 is a flowchart of step S170 in Figure 1

[0058] Figure 8 is a structural schematic diagram of a text summary generation apparatus provided by an embodiment of the present application;

[0059] Figure 9 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0061] It should be noted that, although the functional modules are divided in the apparatus schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the apparatus or the order in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0063] Text summary generation is to extract key information from a long text and generate a concise summary. In the related art, a single extractive summary model or generative summary model is used for text summary generation. For a text in a specific field such as a legal text, the text covers multiple and closely related topics, the text is highly professional and complex in structure, and the text is long. A single extractive summary model or generative summary model cannot accurately understand the text semantics of the text in the specific field, resulting in low accuracy of text summary generation.

[0064] ​​​​Based on this, the embodiment of the present application provides a text summary generation method, a text summary generation device, an electronic device and a computer readable storage medium, aiming to improve the accuracy of text summary generation.

[0065] The text summary generation method, the text summary generation device, the electronic device and the computer readable storage medium provided by the embodiment of the present application are specifically explained by the following embodiment, first, the text summary generation method in the embodiment of the present application is described.

[0066] The text summary generation method provided by the embodiment of the present application relates to the field of artificial intelligence technology. The text summary generation method provided by the embodiment of the present application can be applied in a terminal, can be applied in a server end, and can also be software running in a terminal or a server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as an independent physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and basic cloud computing services such as big data and artificial intelligence platforms; the software can be an application that implements the text summary generation method, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0068] Figure 1 is an optional flowchart of a text summary generation method provided by the embodiment of the present application, Figure 1 The method in can include but is not limited to including steps S110 to S190.

[0069] Step S110, obtaining a sample text and a sample summary of the sample text; the sample text includes sample sentences, and the sample summary includes summary sentences;

[0070] In step S120, a first similarity between the sample sentences and the summary sentences is calculated, a sample sentence with the largest first similarity is selected as a candidate sentence, and a number of candidate sentences is recorded;

[0071] In step S130, if the number of sentences is equal to a number of summary sentences in the sample summary, a candidate text is determined according to the candidate sentences;

[0072] In step S140, a preset summary extraction model is trained according to the sample text and the candidate text, to obtain an extraction summary model;

[0073] In step S150, the sample text is summarized by the extraction summary model, to obtain a preliminary summary;

[0074] In step S160, the preliminary summary is screened according to the sample summary, to obtain a candidate summary;

[0075] In step S170, a preset summary generation model is trained according to the candidate summary and the sample summary, to obtain a generation summary model;

[0076] In step S180, a target text is obtained;

[0077] In step S190, the target text is summarized by the extraction summary model, to obtain a rough summary, and the rough summary is refined by the generation summary model, to obtain a target summary.

[0078] In step S110 of some embodiments, a sample text and a sample summary are obtained from a data set. The sample text is a text used in a text summary generation task, and the sample summary is a summary of main content of the sample text. The sample text uses a source language, and the sample summary uses a target language. The source language and the target language can be the same or different. The sample text includes a first number of sample sentences, the sample sentences are sentences in the sample text, the first number is a number of sentences contained in the sample text, the sample summary includes a second number of summary sentences, the summary sentences are sentences in the sample summary, and the second number is a number of summary sentences contained in the sample summary. The first number and the second number are greater than or equal to 1, and the first number is greater than or equal to the second number.

[0079] In step S120 of some embodiments, text summarization technology has become a primary means of coping with the explosion of massive amounts of data in the context of rapid development of information technology. Text summarization technology aims to compress lengthy content into concise and accurate text while preserving the core information. Professionals in various industries generate large amounts of text data daily, making accurate summaries that meet the needs of specific domains crucial. Unlike general summaries that compress source text based on information density, domain-specific summaries focus on extracting and compressing the most representative and valuable information within that domain. Sample texts may contain sample sentences unrelated to the domain of the sample summary. To obtain high-quality sample text, this embodiment uses a domain alignment matching strategy to extract sample sentences from the sample text at the sentence level, based on the sample summary, to remove domain-irrelevant sample sentences and retain domain-related sample sentences. To extract text fragments highly relevant to the content of a specific domain, for each sample sentence, the similarity between the sample sentence and each summary sentence is calculated to obtain a first similarity. The higher the first similarity, the more similar the sample sentence and the summary sentence are. The first similarity can be measured using cosine similarity, edit distance, or ROUGE value. The sample sentence with the highest first-order similarity is selected as the candidate sentence, and the summary sentence with the highest first-order similarity is selected as the sentence to match the sample sentence. To balance the completeness and accuracy of the sample summary in terms of information coverage, each summary sentence has a matching candidate sentence, and the number of candidate sentences is recorded. It should be noted that there is a one-to-one correspondence between candidate sentences and summary sentences; one summary sentence can only match one candidate sentence. If the sample sentence with the highest first-order similarity has already been matched by other summary sentences, then that sample sentence will not participate in the next round of matching.

[0080] For example, given 5 sample sentences and 2 summary sentences, if the first summary sentence and the third sample sentence have the highest similarity, then the third sample sentence is considered a candidate sentence to match the first summary sentence. Since the third sample sentence has already been matched by the first summary sentence, it will not be considered for matching the second summary sentence.

[0081] In step S130 of some embodiments, if the number of sentences is equal to the number of summary sentences in the sample summary, i.e., the second number, then the obtained second number of candidate sentences are used as candidate texts to ensure the integrity of the candidate texts in terms of information coverage.

[0082] In specific fields such as law, legal texts often cover diverse and closely related topics, are highly specialized, structurally complex, and lengthy. Automated summarization of legal texts must not only ensure the accuracy and factual validity of information but also generate fluent and highly relevant summaries. Current text summarization technologies mainly rely on single extractive summarization models or generative summarization models. These models cannot accurately understand the semantics of texts in specific domains, resulting in incoherent summaries generated using a single extractive summarization model and inaccurate summaries generated using a single generative summarization model, thus leading to low-quality text summarization. This application proposes a two-stage fusion method of extractive and generative summarization models for text summarization. By combining the realism of the extractive summarization model with the fluency of the generative summarization model, the quality of text summarization is improved. The extractive summarization model extracts summaries from sample texts, extracting key information from lengthy texts while preserving the original meaning, resulting in a preliminary summary.

[0083] In step S140 of some embodiments, the extractive summarization model is a model obtained after training a preset summarization extraction model based on sample text and candidate text. The extractive summarization model and the preset summarization extraction model have the same model structure but different model parameters. Specifically, the preset summarization extraction model extracts a summary from the sample text to obtain a predicted summary. A loss function is used to calculate the loss between the predicted summary and the candidate text to obtain a loss value. The model parameters of the preset summarization extraction model are adjusted by minimizing the loss value to obtain the extractive summarization model, which can accurately extract key text information from the sample text.

[0084] Please see Figure 2 In some embodiments, the extractive summarization model includes a first attention sub-model, a second attention sub-model, and a classification sub-model, which are cascaded. Step S150 may include, but is not limited to, steps S210 to S240:

[0085] Step S210: Perform semantic analysis on the sample text using the first attention sub-model to obtain the sentence semantic representation of the sample text;

[0086] Step S220: Sentence encoding is performed on the sentence semantic representation of the sample text using the second attention sub-model to obtain the sentence encoding features of each sample sentence;

[0087] Step S230: For each sample sentence, the sentence encoding features are classified using a classification sub-model to obtain the sentence importance category; the sentence importance category is used to indicate whether the sample sentence is important or not.

[0088] Step S240, screening the sample text according to the sentence importance category to obtain an initial selected summary.

[0089] In step S210 of some embodiments, the sample text is character-level encoded to obtain an encoded text. The first attention sub-model includes at least two network sub-layers, and the encoded text is subjected to semantic analysis through the first attention sub-model until the output of the last network sub-layer is obtained, thereby obtaining a sentence semantic representation of each sample sentence.

[0090] The process of character-level encoding is as follows:

[0091] Suppose the sample text containing m sample sentences is represented as S={s1,...,s i ,...,s m The i-th sample sentence contains t characters, and the j-th character of the i-th sample sentence is denoted as In the end of each sample sentence, a special character is added <sep>a separator, and adding <cls>Separator, resulting in an encoded text representation as {s1 <sep> ...s i <sep> ...s m <sep> <cls>}.

[0092] In step S220 of some embodiments, the sentence semantic representations of all sample sentences are encoded by a second attention sub-model to facilitate the interaction of information between sentences, obtain context information at the sentence level, and obtain sentence encoding features of each sample sentence. The sentence encoding features are sentence representations containing rich context information. Text in a specific field is long and key information is scattered throughout the text. By encoding at the character level and the sentence level, the accuracy of local information extraction and the coherence of global semantics are ensured, so that the extractive summarization model can process long text, and the generated summary has integrity and consistency.

[0093] In step S230 of some embodiments, the sentence encoding features of each candidate sentence are input into a classification sub-model to perform binary classification on the importance of the candidate sentence, determine the importance of the candidate sentence in the candidate text, and obtain the sentence importance category of each sample sentence. The sentence importance category includes important or unimportant, indicating that the sample sentence is important or unimportant. The classification sub-model can use a multi-layer perceptron. If the output value of the classification sub-model is 1, it indicates that the sample sentence is important; if the output value of the classification sub-model is 0, it indicates that the sample sentence is unimportant.

[0094] In step S240 of some embodiments, sample sentences with an important sentence importance category are selected from the sample text to obtain a preliminary selected summary.

[0095] Through the above steps S210 to S240, the semantic relevance between sentences can be effectively improved, the complete semantic encoding of long text is ensured, and thus the key information can be extracted from long text to support the generation of high-quality text summaries.

[0096] Please refer to Figure 3 In some embodiments, the first attention sub-model includes a first network sub-layer and a second network sub-layer, the first network sub-layer has a query weight matrix, a first key weight matrix, and a value weight matrix, and step S210 can include but is not limited to steps S310 to S330:

[0097] In step S310, in the first network sub-layer, the original query feature of the first sample sentence is determined according to the query weight matrix and the first sample sentence, the original key feature of the first sample sentence is determined according to the first key weight matrix and the first sample sentence, and the original value feature of the first sample sentence is determined according to the value weight matrix and the first sample sentence; the original query feature, the original key feature, and the original value feature of the first sample sentence are transformed by attention to obtain the sentence hidden state feature of the first sample sentence;

[0098] In step S320, at the first network sub-layer, the previous sample sentence and the current sample sentence are concatenated starting from the second sample sentence to obtain a concatenated sentence, the original query feature of the current sample sentence is determined according to the query weight matrix and the current sample sentence, the original key feature of the current sample sentence is determined according to the first key weight matrix and the concatenated sentence, and the original value feature of the current sample sentence is determined according to the value weight matrix and the concatenated sentence; and the original query feature, the original key feature and the original value feature of the current sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the current sample sentence.

[0099] In step S330, at the second network sub-layer, the sentence semantic representation of the first sample sentence is obtained by attention transformation according to the sentence hidden state feature of the first sample sentence, and the sentence semantic representation of the current sample sentence is obtained by attention transformation according to the sentence hidden state feature of the previous sample sentence and the sentence hidden state feature of the current sample sentence starting from the second sample sentence.

[0100] In step S310 of some embodiments, at the first network sub-layer, the query weight matrix is multiplied by the first sample sentence to obtain the original query feature of the first sample sentence, the first key weight matrix is multiplied by the first sample sentence to obtain the original key feature of the first sample sentence, and the value weight matrix is multiplied by the first sample sentence to obtain the original value feature of the first sample sentence. The original query feature, the original key feature and the original value feature of the first sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the first sample sentence.

[0101] In step S320 of some embodiments, at the first network sub-layer, the previous sample sentence and the current sample sentence are concatenated starting from the second sample sentence to obtain a concatenated sentence, the original query feature of the current sample sentence is determined according to the query weight matrix and the current sample sentence, the original key feature of the current sample sentence is determined according to the first key weight matrix and the concatenated sentence, and the original value feature of the current sample sentence is determined according to the value weight matrix and the concatenated sentence. The original query feature, the original key feature and the original value feature of the current sample sentence are subjected to attention transformation to obtain the sentence hidden state feature of the current sample sentence. The above steps are repeated until the sentence hidden state feature of the last sample sentence is obtained.

[0102] In step S330 of some embodiments, the second network sublayer also has a query weight matrix, a first key weight matrix, and a value weight matrix. The query weight matrix, the first key weight matrix, and the value weight matrix of the second network sublayer are respectively multiplied by the sentence hidden state feature of the first sample sentence to obtain a reference query feature, a reference key feature, and a reference value feature of the first sample sentence. The reference query feature, the reference key feature, and the reference value feature of the first sample sentence are subjected to attention transformation to obtain a sentence semantic representation of the first sample sentence.

[0103] The sentence hidden state feature of the previous sample sentence and the sentence hidden state feature of the current sample sentence are spliced to obtain a state splicing feature. Through the splicing operation, the hidden state of the previous sample sentence can be introduced as an additional input into the attention mechanism of the current sample sentence, ensuring the context consistency across the text sentences. The query weight matrix of the second network sublayer is multiplied by the sentence hidden state feature of the current sample sentence to obtain a reference query feature of the current sample sentence. The first key weight matrix and the value weight matrix of the second network sublayer are respectively multiplied by the state splicing feature to obtain a reference key feature and a reference value feature of the current sample sentence. The sentence hidden state feature, the reference query feature, the reference key feature, and the reference value feature of the current sample sentence are subjected to attention transformation to obtain a sentence semantic representation of the current sample sentence, until a sentence semantic representation of the last sample sentence is obtained.

[0104] Through the above steps S310 to S330, the semantic correlation between different sample sentences of a long text can be more accurately captured, ensuring the context consistency across the text segments, thereby ensuring the accuracy and fluency of the summary generation, and avoiding the loss and fragmentation of text information caused by the truncation method or the block method.

[0105] Please refer to Figure 4 In some embodiments, the first network sublayer also has a second key weight matrix, a content bias matrix, and a position bias matrix, and step S310 can include but is not limited to steps S410 to S430:

[0106] Step S410, relative position encoding is performed on the first sample sentence to obtain a first sentence position encoding;

[0107] Step S420, attention calculation is performed according to the second key weight matrix, the content bias matrix, the position bias matrix, the original query feature, the original key feature of the first sample sentence, and the first sentence position encoding to obtain an attention score;

[0108] Step S430, feature mapping is performed on the first sample sentence, the attention score, and the original value feature of the first sample sentence to obtain a sentence hidden state feature of the first sample sentence.

[0109] In step S410 of some embodiments, the existing extractive summary model usually limits the length of the input data, for example, the BERT model can only encode 512 characters and cannot deal with long texts. In order to ensure the complete semantic encoding of long texts, the embodiments of the present application model the global relevance of the sample text from the sentence level, so that the extractive summary model can learn the key information highly related to the summary content from the complete long text.

[0110] The sample sentence includes a plurality of characters, each character having a character index, the character index being used to indicate the position of the character in the sample sentence. If the jth character of the sample sentence is j is the character index of the character. The absolute value of the difference between the character indexes of any two characters in the sample sentence is calculated to obtain a target index. For example, if the character index of the jth character is j and the character index of the qth character is q, the target index is |j-q|. An encoding dimension is obtained, if the encoding dimension is even, the absolute value and the encoding dimension are encoded according to the sine function, if the encoding dimension is odd, the character index and the encoding dimension are encoded according to the cosine function, until the encoding of the last character is completed, to obtain a first sentence position encoding. The sine function is represented as:

[0111]

[0112] wherein pos represents the target index, i represents the encoding dimension, d m represents the feature dimension of the first sentence position encoding.

[0113] The cosine function is represented as:

[0114]

[0115] In step S420 of some embodiments, a first sub-score is determined according to the original query feature and the original key feature of the first sample sentence, a second sub-score is determined according to the original query feature, the second key weight matrix and the first sentence position encoding of the first sample sentence, a third sub-score is determined according to the global-based content bias matrix and the original key feature of the first sample sentence, and a fourth sub-score is determined according to the global-based position bias matrix, the second key weight matrix and the first sentence position encoding. The sum of the first sub-score, the second sub-score, the third sub-score and the fourth sub-score is obtained to obtain an attention score. The content relevance and the relative position relationship between the characters of the sentence can be considered comprehensively through the four components of the first sub-score, the second sub-score, the third sub-score and the fourth sub-score, and the understanding ability of the model to the text is improved.

[0116] The calculation method of the attention score is as follows:

[0117]

[0118] wherein A represents an attention score; T represents a transpose operation; q1 represents an original query feature of the first sample sentence; k1 represents an original key feature of the first sample sentence; represents a first sub-score; W k,R represents a second key weight matrix; R represents a first sentence position encoding; represents a second sub-score; u represents a content bias matrix; u T k1 represents a third sub-score; v represents a position bias matrix; v T W k,R R represents a fourth sub-score.

[0119] In step S430 of some embodiments, the attention score is activated by a softmax function to obtain an activated feature, and the activated feature is linearized by a linear layer to obtain a linear feature. The linear feature is fused with the first sample sentence to obtain a fused feature, and the fused feature is normalized by a layer normalization to obtain a normalized feature. The normalized feature is fed forward by a feedforward network layer to obtain a sentence hidden state feature of the first sample sentence.

[0120] It should be noted that, according to the sentence hidden state feature of the current sample sentence, the reference query feature, the reference key feature and the reference value feature, the attention transformation can be performed by referring to steps S410 to S430 to obtain a sentence semantic representation of the current sample sentence, and the specific calculation process of the sentence semantic representation is as follows:

[0121]

[0122] a τ = softmax(A τ )v τ ,

[0123]

[0124] wherein τ represents the τth sample sentence; T represents a transpose operation; A τ represents an attention score of the sentence hidden state feature of the τth sample sentence; q τ represents a reference query feature; k τ represents a reference key feature; represents a second key weight matrix of the second network sub-layer; R τ represents a relative position encoding of the τth sample sentence; u 2 represents a global-based content bias matrix of the second network sub-layer; v 2 represents a global-based position bias matrix of the second network sub-layer; v τ representing a reference value feature; is a sentence hidden state feature of the τ-th sample sentence; τ 、o τ is an intermediate variable; is a sentence semantic representation of the τ-th sample sentence.

[0125] Through the above steps S410 to S430, the key information can be accurately extracted from the long text without being limited by the length of the long text.

[0126] Please refer to Figure 5 In some embodiments, step S220 can include but is not limited to steps S510 to S530:

[0127] Step S510, performing absolute position encoding on the sentence semantic representations of the sample text to obtain second sentence position encodings;

[0128] Step S520, performing feature fusion on the second sentence position encodings and the sentence semantic representations of the sample text to obtain fused semantic features;

[0129] Step S530, performing feature encoding on the fused semantic features to obtain sentence encoding features of each sample sentence.

[0130] In step S510 of some embodiments, the second attention sub-model includes a position encoding layer and a plurality of feature encoding layers. The feature encoding layer is a transformer architecture-based encoder, and the number of feature encoding layers can be 12. The network structure of each feature encoding layer is the same, and each includes a multi-head self-attention mechanism, a normalization layer, and a feedforward network. The sentence semantic representations of all sample sentences are SR={p1, p2,..., pm}, and the position encoding layer can be used to perform absolute position encoding on the sentence semantic representations of all sample sentences to obtain global position information, thereby obtaining the second sentence position encodings.

[0131] In step S520 of some embodiments, feature addition is performed on the second sentence position encodings and the sentence semantic representations of all sample sentences to obtain the fused semantic features of the sample text. By integrating the sentence semantic representation into the absolute position encoding information, effective modeling of the context between sentences can be achieved.

[0132] In step S530 of some embodiments, the fused semantic features are randomly inactivated by a dropout technique to obtain input features. The input features are encoded by a plurality of feature encoding layers to facilitate the interaction of information between sentences, thereby obtaining context information at the sentence level. Specifically, the input features are encoded by a first feature encoding layer to obtain output features of the first feature encoding layer. Starting from a second feature encoding layer, the output features of all feature encoding layers before the current feature encoding layer are spliced to obtain input features of the current feature encoding layer, and the input features are encoded by the current feature encoding layer to obtain output features of the current feature encoding layer, until the output features of the last feature encoding layer are obtained, and the output features of the last feature encoding layer are taken as sentence encoding features.

[0133] The feature encoding process of each feature encoding layer is the same. The input features of the sample sentence can be calculated by a multi-head self-attention mechanism to obtain attention features. The attention features and the input features are added to obtain first fused features. The first fused features are normalized by a normalization layer to obtain normalized features. The normalized features are extracted by a feedforward network to obtain feedforward features. The feedforward features and the normalized features are added to obtain second fused features. The second fused features are normalized to obtain output features. The calculation method of the output features of the zth feature encoding layer is as follows:

[0134]

[0135] wherein MulHAtt is multi-head attention calculation; p i is the input feature of the i-th sample sentence of the zth feature encoding layer; and are the key vector, query vector and value vector corresponding to the input feature, respectively; is the attention feature; is the normalized feature; FF represents the feedforward network; represents the output feature.

[0136] Please refer to Figure 6 In some embodiments, step S160 can include but is not limited to steps S610 to S620:

[0137] Step S610, calculating a second similarity between the sample summary and the preliminary summary;

[0138] Step S620, if the second similarity is greater than a preset similarity threshold, taking the preliminary summary as a candidate summary.

[0139] In step S610 of some embodiments, if the key information extracted by the first-stage extractive summary model is inaccurate, noise can be introduced and hinder the learning process of the second-stage generative summary model, which generates the target summary according to the incorrectly extracted sentences, resulting in an inaccurate target summary. If only the standard answer is used as the input of the second stage and teacher-forcing learning is performed, the difference between the inaccurate information extracted by the first-stage model and the standard summary learned by the second stage can occur in the inference process. To alleviate the sample distribution difference between training and inference when the two-stage heterogeneous model is fused, and to avoid the cascading errors generated by the first-stage model from being brought into the training process of the second-stage model, the application embodiments filter the preliminary summary of each sample text to ensure the integrity and relevance of the summary information, so as to improve the training sample quality of the generative summary model.

[0140] Specifically, the similarity between the sample summary and the preliminary summary is calculated to obtain a second similarity. The second similarity is calculated using a measurement index based on a text semantic similarity matching mechanism, such as a ROUGE value, a cosine similarity, an edit distance, etc. The greater the second similarity, the more similar the sample summary and the preliminary summary.

[0141] In step S620 of some embodiments, if the second similarity is greater than a preset similarity threshold, the preliminary summary is taken as a candidate summary. If the second similarity is less than or equal to the preset similarity threshold, the preliminary summary is discarded and is not taken as a training sample of the model. The preset similarity threshold can be 0.5.

[0142] Through the above steps S610 to S620, high-quality samples with high semantic similarity to the sample summary can be screened out, ensuring the relevance of the two-stage training data, thereby improving the quality of the generated summary.

[0143] Please refer to Figure 7 In some embodiments, step S170 can include but is not limited to steps S710 to S730:

[0144] In step S710, a target summary is obtained by performing summary prediction on the candidate summary through a preset summary generation model.

[0145] In step S720, cross-entropy loss calculation is performed according to the target summary and the sample summary to obtain target loss data.

[0146] In step S730, the model parameters of the preset summary generation model are updated according to the target loss data to obtain a generative summary model.

[0147] In step S710 of some embodiments, the preset summary generation model can employ a T5 model, and the preset summary generation model includes an encoder and a decoder. The candidate summary is encoded by the encoder to obtain an encoded representation vector of the summary. The encoded representation vector of the summary is a hidden state generated by the encoder. A sequence of words output by the decoder before the current time step is obtained, and the encoded representation vector of the summary and the sequence of words are decoded by the decoder to obtain a target summary. If the current time step is an initial time step, the sequence of words can be a zero vector. The process of feature decoding is represented as:

[0148] y t = Decoder(y ＜t , h),

[0149] where y t is the t-th word in the generated target summary, y ＜t is a sequence of words before the t-th word currently generated, and h is the encoded representation vector of the summary.

[0150] In step S720 of some embodiments, in order to minimize the difference between the target summary and the sample summary, a loss function based on cross entropy is employed to optimize the preset summary generation model. The loss function is defined as:

[0151]

[0152] where L represents target loss data; N is the number of sample summaries; T is the summary length of the target summary; git represents the true label of the t-th word in the i-th sample summary; pit represents the probability that the t-th word output by the preset summary generation model is the t-th word in the sample summary; * represents multiplication.

[0153] In step S730 of some embodiments, the model parameters of the preset summary generation model are updated, and when the target loss data reaches a minimum, the training round reaches a preset number of thresholds, or the target loss data of adjacent iteration rounds changes little, the preset summary generation model at this time is taken as the generative summary model.

[0154] The steps S710 to S730 described above ensure the model performance of the preset summary generation model by selecting high-quality samples for model training, and improve the accuracy and fluency of summary generation.

[0155] In step S180 of some embodiments, a target text is obtained from the data set, and the target text is a text to be generated as a summary.

[0156] In step S190 of some embodiments, the target text is abstracted by an extractive summary model to obtain a rough summary. Alternatively, the target text is abstracted by multiple extractive summary models to obtain multiple extracted summaries; the multiple extracted summaries are compared, and the extracted summary with the highest number of occurrences is selected as the rough summary. The rough summary is refined by a generative summary model to obtain the target summary.

[0157] The model performance of the embodiments of the present application is tested on the two legal-related corpora of CAIL 2022 Law-related Public Opinion Summary Dataset and BillSum test dataset. The text summary generation method of the embodiments of the present application is compared with multiple baseline models. The embodiments of the present application show significant advantages in the relevance, coherence and information comprehensiveness of the summary. The experimental results are shown in Tables 1 to 3. Table 1 is the experimental results of each model on the CAIL 2022 dataset, Table 2 is the experimental results of each model on the BillSum dataset, and Table 3 is the average coherence score and average accuracy score of GPT4 evaluating the output summary of the embodiments of the present application.

[0158]

[0159] Table 1

[0160]

[0161]

[0162] Table 2

[0163]

[0164] Table 3

[0165] Please refer to Figure 8 The embodiments of the present application also provide a text summary generation device, which can implement the above-mentioned text summary generation method. The text summary generation device comprises:

[0166] The first acquisition module 810 is configured to acquire a sample text and a sample summary of the sample text. The sample text comprises a sample sentence, and the sample summary comprises a summary sentence.

[0167] The calculation module 820 is configured to calculate a first similarity between the sample sentence and the summary sentence, select a sample sentence with the maximum first similarity as a candidate sentence, and record the number of candidate sentences.

[0168] The determination module 830 is configured to determine a candidate text according to the candidate sentence if the number of sentences is equal to the number of summary sentences in the sample summary.

[0169] The first training module 840 is configured to train the preset extractive summary model according to the sample text and the candidate text, to obtain an extractive summary model.

[0170] The abstract extraction module 850 is configured to perform abstract extraction on the sample text by using the extractive summary model, to obtain a preliminary selected summary.

[0171] The abstract screening module 860 is configured to perform abstract screening on the preliminary selected summary according to the sample summary, to obtain a candidate summary.

[0172] The second training module 870 is configured to train the preset generative summary model according to the candidate summary and the sample summary, to obtain a generative summary model.

[0173] The second acquisition module 880 is configured to acquire a target text.

[0174] The abstract generation module 890 is configured to perform abstract extraction on the target text by using the extractive summary model, to obtain a rough abstract, and perform abstract refinement on the rough abstract by using the generative summary model, to obtain a target summary.

[0175] The specific implementation of the text abstract generation apparatus is basically the same as the specific implementation of the above-mentioned text abstract generation method, and thus will not be repeated here.

[0176] The embodiments of the present application further provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned text abstract generation method. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0177] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0178] The processor 910 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0179] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 920 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 920 and are called and executed by the processor 910 to implement the text summary generation method of the embodiments of the present application.

[0180] The input / output interface 930 is configured to realize information input and output.

[0181] The communication interface 940 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0182] The bus 950 is configured to transmit information between various components (for example, the processor 910, the memory 920, the input / output interface 930, and the communication interface 940) of the device.

[0183] The processor 910, the memory 920, the input / output interface 930, and the communication interface 940 are connected to each other through the bus 950 to realize the communication connection between the device.

[0184] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned text summary generation method.

[0185] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0186] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0187] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0189] Those skilled in the art can understand that all or some steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0190] The terms "first", "second", "third", "fourth" and the like in the description of the present application and the above-mentioned figures (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0191] It should be understood that in the present application, "at least one" means one or more, "multiple" means two or more. "And / or", used to describe the association between the associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A, B can be singular or plural. The character " / " generally represents a "or" relationship between the associated objects before and after. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0192] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0193] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0194] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0195] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0196] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.< / cls> < / sep> < / sep> < / sep> < / cls> < / sep>

Claims

1. A text summarization method, characterized in that, The method includes: Obtain sample text and sample summary of the sample text; the sample text includes a first number of sample sentences, and the sample summary includes a second number of summary sentences, wherein the first number is greater than or equal to the second number; Calculate the first similarity between the sample sentence and the summary sentence, select the sample sentence with the highest first similarity as the candidate sentence, and record the number of sentences in the candidate sentence; one candidate sentence matches one summary sentence; If the number of sentences is equal to the second number, then the second number of candidate sentences are taken as candidate texts; The preset summary extraction model is trained based on the sample text and the candidate text to obtain the extractive summary model; The sample text is extracted using the extractive summarization model to obtain a preliminary summary. Based on the sample abstract, the preliminary abstract is filtered to obtain candidate abstracts; The preset summary generation model is trained based on the candidate summary and the sample summary to obtain a generative summary model; Get the target text; The target text is summarized by the extractive summarization model to obtain a coarse summary, and the coarse summary is refined by the generative summarization model to obtain the target summary.

2. The method according to claim 1, characterized in that, The extractive summarization model includes a first attention sub-model, a second attention sub-model, and a classification sub-model. The process of extracting summaries from the sample text using the extractive summarization model to obtain preliminary summaries includes: The sample text is semantically analyzed using the first attention sub-model to obtain the sentence semantic representation of the sample text; The sentence encoding features of each sample sentence are obtained by encoding the sentence semantic representation of the sample text using the second attention sub-model. For each sample sentence, the sentence encoding features are classified using the classification sub-model to obtain a sentence importance category; the sentence importance category is used to indicate whether the sample sentence is important or not. The sample texts are filtered according to the importance category of the sentences to obtain the preliminary summary.

3. The method according to claim 2, characterized in that, The first attention sub-model includes a first network sub-layer and a second network sub-layer. The first network sub-layer has a query weight matrix, a first key weight matrix, and a value weight matrix. The step of performing semantic analysis on the sample text using the first attention sub-model to obtain the sentence semantic representation of the sample text includes: In the first network sub-layer, the original query features of the first sample sentence are determined based on the query weight matrix and the first sample sentence; the original key features of the first sample sentence are determined based on the first key weight matrix and the first sample sentence; and the original value features of the first sample sentence are determined based on the value weight matrix and the first sample sentence. Attention transformation is performed on the original query features, original key features, and original value features of the first sample sentence to obtain the sentence hidden state features of the first sample sentence. In the first network sub-layer, starting with the second sample sentence, the previous sample sentence and the current sample sentence are concatenated to obtain a concatenated sentence. The original query features of the current sample sentence are determined based on the query weight matrix and the current sample sentence. The original key features of the current sample sentence are determined based on the first key weight matrix and the concatenated sentence. The original value features of the current sample sentence are determined based on the value weight matrix and the concatenated sentence. Attention transformation is applied to the original query features, original key features, and original value features of the current sample sentence to obtain the sentence hidden state features of the current sample sentence. In the second sub-layer of the network, attention transformation is performed based on the hidden state features of the first sample sentence to obtain the semantic representation of the first sample sentence. Starting from the second sample sentence, attention transformation is performed based on the hidden state features of the previous sample sentence and the hidden state features of the current sample sentence to obtain the semantic representation of the current sample sentence.

4. The method according to claim 3, characterized in that, The first network sublayer also has a second key weight matrix, a content bias matrix, and a position bias matrix. The attention transformation performed on the original query features, original key features, and original value features of the first sample sentence to obtain the sentence hidden state features of the first sample sentence includes: The relative position encoding of the first sample sentence is performed to obtain the position encoding of the first sentence; Attention is calculated based on the second key weight matrix, the content bias matrix, the position bias matrix, the original query features of the first sample sentence, the original key features, and the position code of the first sentence to obtain an attention score; Feature mapping is performed on the first sample sentence, the attention score, and the original value features of the first sample sentence to obtain the sentence hidden state features of the first sample sentence.

5. The method according to claim 2, characterized in that, The step of encoding the sentence semantic representation of the sample text using the second attention sub-model to obtain the sentence encoding features of each sample sentence includes: The absolute positional encoding of the sentence semantic representation of the sample text is performed to obtain the second sentence positional encoding; The second sentence position encoding and the sentence semantic representation of the sample text are fused to obtain fused semantic features; The fused semantic features are then encoded to obtain the sentence encoding features for each sample sentence.

6. The method according to any one of claims 1 to 5, characterized in that, The step of filtering the initial summary based on the sample summary to obtain candidate summaries includes: Calculate the second similarity between the sample summary and the initial selection summary; If the second similarity is greater than the preset similarity threshold, then the preliminary summary is used as the candidate summary.

7. The method according to any one of claims 1 to 5, characterized in that, The step of training a preset summary generation model based on the candidate summary and the sample summary to obtain a generative summary model includes: The target summary is obtained by performing summary prediction on the candidate summaries using the preset summary generation model. Cross-entropy loss is calculated based on the target summary and the sample summary to obtain the target loss data; The model parameters of the preset summary generation model are updated based on the target loss data to obtain the generative summary model.

8. A text summarization generation apparatus, characterized in that, The device includes: The first acquisition module is used to acquire sample text and a sample summary of the sample text; the sample text includes a first number of sample sentences, and the sample summary includes a second number of summary sentences, wherein the first number is greater than or equal to the second number; The calculation module is used to calculate the first similarity between the sample sentence and the summary sentence, select the sample sentence with the largest first similarity as a candidate sentence, and record the number of sentences in the candidate sentence; one candidate sentence matches one summary sentence; The determining module is configured to, if the number of sentences is equal to the second number, select the second number of candidate sentences as candidate text; The first training module is used to train the preset summary extraction model based on the sample text and the candidate text to obtain the extractive summary model. The abstract extraction module is used to extract abstracts from the sample text using the extractive abstraction model to obtain a preliminary abstract. The abstract filtering module is used to filter the initially selected abstracts based on the sample abstracts to obtain candidate abstracts; The second training module is used to train the preset summary generation model based on the candidate summary and the sample summary to obtain a generative summary model; The second acquisition module is used to acquire the target text; The summary generation module is used to extract a summary from the target text using the extractive summarization model to obtain a coarse summary, and then refine the coarse summary using the generative summarization model to obtain the target summary.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images