Abstract generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an abstract generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-processed text; inputting the to-be-processed text into an abstract generation model, wherein the abstract generation model comprises a key sentence extractor and an abstract generator; calling the key sentence extractor to process the to-be-processed text to obtain candidate sentences corresponding to the to-be-processed text and sentence scores corresponding to the candidate sentences, and screening first K candidate sentences with the highest sentence scores from the candidate sentences, K being a positive integer; and calling the abstract generator to process a combined text of the first K candidate statements and the to-be-processed text based on a built-in prompt word template, and generating a text abstract corresponding to the to-be-processed text. According to the embodiment of the invention, the problem of incomplete summarization or wrong understanding in the abstract generation process can be solved, and the accuracy and efficiency of the generated abstract are improved.

Description

Technical Field

[0001] The present application relates to the technical field of abstract generation, and in particular to an abstract generation method, device, electronic device, and storage medium. Background Art

[0002] In today's era of explosive growth in information, the processing, summarization and analysis of text data have become crucial. Currently, there are two common methods for generating text summaries: extractive and abstract.

[0003] Extractive approaches rely on extracting several parts from a text, such as phrases and sentences, and combining them to form a summary. Abstractive approaches, on the other hand, apply deep learning or large-scale modeling techniques to generate a new summary based on the model's understanding of the text content. While both approaches have achieved considerable maturity, they still present certain challenges.

[0004] Extractive text summarization methods have limited understanding of semantics and struggle to deeply explore text context. Consequently, they may lose key information during the abstraction process, resulting in incomplete summaries. Abstractive text summarization methods, on the other hand, analyze and summarize existing content during the large model generation process. However, this can be influenced by previously learned content, leading to misunderstandings and other issues in the results. Summary of the Invention

[0005] The technical problem to be solved by the embodiments of the present application is to provide a summary generation method, device, electronic device and storage medium to effectively solve the problems of incomplete summarization or misunderstanding in the summary generation process, and improve the accuracy and efficiency of the generated summary.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a summary, the method comprising:

[0007] Get the text to be processed;

[0008] Inputting the text to be processed into a summary generation model, wherein the summary generation model includes: a key sentence extractor and a summary generator;

[0009] Calling the key sentence extractor to process the text to be processed, obtaining candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and screening out top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer;

[0010] The summary generator is called to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

[0011] Optionally, the key sentence extractor includes: a basic network layer and a scoring network layer,

[0012] The calling of the key sentence extractor to process the text to be processed to obtain candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences includes:

[0013] Calling the basic network layer to process the text to be processed to obtain a full-text feature matrix of the text to be processed and a sentence feature matrix of each candidate sentence in the text to be processed;

[0014] The scoring network layer is called to process the full-text feature matrix and the sentence feature matrix to obtain sentence scores corresponding to the candidate sentences.

[0015] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0016] The calling of the scoring network layer to process the full-text feature matrix and the sentence feature matrix to obtain a sentence score corresponding to the candidate sentence includes:

[0017] Calling the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features;

[0018] Calling the pooling layer to perform global mean pooling processing and global maximum pooling processing on the attention fusion feature, respectively, to obtain corresponding mean pooling features and maximum pooling features;

[0019] Calling the feature splicing layer to perform splicing processing on the mean pooling feature and the maximum pooling feature to obtain a first splicing feature;

[0020] The regression layer is called to process the first concatenated feature to obtain a sentence score corresponding to the candidate sentence.

[0021] Optionally, the attention fusion layer includes: a first attention fusion module, a splicing module and a second attention fusion module.

[0022] The calling of the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features includes:

[0023] Calling the first attention fusion module to process the full-text feature matrix and the sentence feature matrix respectively to obtain corresponding full-text features and sentence features;

[0024] Calling the splicing module to perform splicing processing on the full-text feature and the sentence feature to obtain a second splicing feature;

[0025] The second attention fusion module is called to process the second splicing feature to obtain the attention fusion feature.

[0026] Optionally, before inputting the to-be-processed text into the summary generation model, the method further includes:

[0027] Constructing the first summary dataset;

[0028] Based on the first summary dataset, training the key sentence extractor;

[0029] Constructing a second summary dataset based on the first summary dataset and the key sentence extractor;

[0030] Based on the second summary data set, training the summary generator;

[0031] The key sentence extractor and the summary generator are combined to obtain a summary generation model.

[0032] Optionally, the first summary data set includes: text identifier, text content and text summary,

[0033] The step of training the key sentence extractor based on the first summary dataset includes:

[0034] Using a mask training method, a basic network layer is trained based on the first summary data set;

[0035] Constructing a scoring data set based on the text identifier, the text content, the sentence content corresponding to the text content and the text summary, and the sentence score corresponding to the sentence content;

[0036] A scoring network layer is obtained by training based on the scoring data set;

[0037] The base network layer and the scoring network layer are combined to obtain the key sentence extractor.

[0038] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0039] The scoring network layer is obtained by training based on the scoring dataset, comprising:

[0040] Converting the text content and the sentence content respectively to obtain text segmentation information corresponding to the text content and sentence segmentation information corresponding to the sentence content;

[0041] Calling the attention fusion layer to perform a first round of attention mechanism processing on the text segmentation information and the sentence segmentation information respectively to obtain full-text initial features and sentence initial features, and performing a second round of attention mechanism processing on the concatenated features of the full-text initial features and the sentence initial features to obtain fused features;

[0042] Calling the pooling layer to perform global mean pooling and global maximum pooling on the fusion features respectively to obtain two corresponding pooling features;

[0043] Calling the regression layer to process the two pooled features to obtain a predicted sentence score;

[0044] Calculating a loss value based on the predicted sentence score and the sentence score;

[0045] When the loss value is within a preset range, the scoring network layer is obtained.

[0046] Optionally, the second summary dataset includes: a text identifier, target text content, and a text summary, wherein the target text content is obtained by combining the top K candidate sentences with the highest scores in descending order of scores to form additional content, and appending the resultant content to the beginning of the original text in the text content field of the first dataset;

[0047] The step of training the summary generator based on the second summary dataset includes:

[0048] A hard prompt fine-tuning method is adopted to guide the training of the generator according to the target text content and the text summary to obtain the summary generator.

[0049] In a second aspect, an embodiment of the present application provides a summary generation device, the device comprising:

[0050] A text acquisition module is used to obtain the text to be processed;

[0051] A text input module, configured to input the text to be processed into a summary generation model, wherein the summary generation model includes: a key sentence extractor and a summary generator;

[0052] a sentence acquisition module, configured to call the key sentence extractor to process the text to be processed, obtain candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and screen out top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer;

[0053] The summary generation module is used to call the summary generator to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

[0054] Optionally, the key sentence extractor includes: a basic network layer and a scoring network layer,

[0055] The statement acquisition module includes:

[0056] A feature matrix acquisition unit, configured to call the basic network layer to process the text to be processed, and obtain a full-text feature matrix of the text to be processed and a sentence feature matrix of each candidate sentence in the text to be processed;

[0057] The sentence score acquisition unit is used to call the scoring network layer to process the full-text feature matrix and the sentence feature matrix to obtain the sentence score corresponding to the candidate sentence.

[0058] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0059] The sentence scoring acquisition unit includes:

[0060] A fusion feature acquisition subunit is used to call the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features;

[0061] A pooling feature acquisition subunit is used to call the pooling layer to perform global mean pooling processing and global maximum pooling processing on the attention fusion feature, respectively, to obtain corresponding mean pooling features and maximum pooling features;

[0062] A first feature acquisition subunit is configured to call the feature splicing layer to perform splicing processing on the mean pooling feature and the maximum pooling feature to obtain a first splicing feature;

[0063] The sentence score acquisition subunit is used to call the regression layer to process the first splicing feature to obtain the sentence score corresponding to the candidate sentence.

[0064] Optionally, the attention fusion layer includes: a first attention fusion module, a splicing module and a second attention fusion module.

[0065] The fusion feature acquisition subunit includes:

[0066] a feature acquisition subunit, configured to call the first attention fusion module to process the full-text feature matrix and the sentence feature matrix respectively to obtain corresponding full-text features and sentence features;

[0067] A second feature acquisition subunit is configured to call the splicing module to perform splicing processing on the full-text feature and the sentence feature to obtain a second splicing feature;

[0068] The attention feature acquisition subunit is used to call the second attention fusion module to process the second splicing feature to obtain the attention fusion feature.

[0069] Optionally, the device further comprises:

[0070] A first data set construction module, used to construct a first summary data set;

[0071] An extractor training module, configured to train the key sentence extractor based on the first summary dataset;

[0072] A second data set construction module, configured to construct a second summary data set based on the first summary data set and the key sentence extractor;

[0073] A generator training module, configured to train the summary generator based on the second summary dataset;

[0074] The model acquisition module is used to combine the key sentence extractor and the summary generator to obtain a summary generation model.

[0075] Optionally, the first summary data set includes: text identifier, text content and text summary,

[0076] The extractor training module includes:

[0077] A base layer training unit, configured to train a base network layer based on the first summary data set using a mask training method;

[0078] a scoring set construction unit, configured to construct a scoring data set based on the text identifier, the text content, the sentence content corresponding to the text content and the text summary, and the sentence score corresponding to the sentence content;

[0079] A scoring layer training unit, configured to train a scoring network layer based on the scoring data set;

[0080] An extractor acquisition unit is used to combine the basic network layer and the scoring network layer to obtain the key sentence extractor.

[0081] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0082] The scoring layer training unit includes:

[0083] A word segmentation information acquisition subunit is used to convert the text content and the sentence content respectively to obtain text word segmentation information corresponding to the text content and sentence word segmentation information corresponding to the sentence content;

[0084] a feature fusion subunit, configured to call the attention fusion layer to perform a first round of attention mechanism processing on the text segmentation information and the sentence segmentation information respectively to obtain full-text initial features and sentence initial features, and perform a second round of attention mechanism processing on the concatenated features of the full-text initial features and the sentence initial features to obtain fused features;

[0085] A feature pooling subunit, configured to call the pooling layer to perform global mean pooling and global maximum pooling on the fused features, respectively, to obtain two corresponding pooling features;

[0086] A prediction score acquisition subunit, configured to call the regression layer to process the two pooled features to obtain a prediction sentence score;

[0087] a loss value calculation subunit, configured to calculate a loss value based on the predicted sentence score and the sentence score;

[0088] The scoring layer acquisition subunit is used to obtain the scoring network layer when the loss value is within a preset range.

[0089] Optionally, the second summary dataset includes: a text identifier, target text content, and a text summary, wherein the target text content is obtained by combining the top K candidate sentences with the highest scores in descending order of scores to form additional content, and appending the resultant content to the beginning of the original text in the text content field of the first dataset;

[0090] The generator training module includes:

[0091] The generator acquisition unit is used to adopt a hard prompt fine-tuning method to guide the training of the generator according to the target text content and the text summary to obtain the summary generator.

[0092] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0093] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned methods for generating a summary is implemented.

[0094] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the above-described summary generation methods.

[0095] Compared with the prior art, the embodiments of the present application have the following advantages:

[0096] In an embodiment of the present application, a text to be processed is obtained. The text to be processed is input into a summary generation model, and the summary generation model includes: a key sentence extractor and a summary generator. The key sentence extractor is called to process the text to be processed, and the candidate sentences corresponding to the text to be processed and the sentence scores corresponding to the candidate sentences are obtained, and the top K candidate sentences with the highest sentence scores are screened out from the candidate sentences, where K is a positive integer. The summary generator is called to process the combined text of the top K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed. The embodiment of the present application solves the problems of incomplete summarization or misunderstanding in the summary generation process by combining the extraction type and the abstract type summary generation methods. At the same time, compared with the traditional extraction type or abstract type summary extraction algorithm, it not only avoids the hallucination problem of the large model, but also improves the accuracy and efficiency of the generated summary.

[0097] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 A flowchart of the steps of a summary generation method provided in an embodiment of the present application;

[0099] Figure 2 A schematic diagram of an Extractor structure provided in an embodiment of the present application;

[0100] Figure 3 A schematic diagram of an attention fusion processing flow provided in an embodiment of the present application;

[0101] Figure 4 A schematic diagram of a feature construction and scoring model training process provided in an embodiment of the present application;

[0102] Figure 5 A schematic diagram of an Extractor-Generator hybrid summary generation model provided in an embodiment of the present application;

[0103] Figure 6 A schematic diagram of data flow provided in an embodiment of the present application;

[0104] Figure 7 A schematic diagram of the structure of a summary generation device provided in an embodiment of the present application;

[0105] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0106] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0107] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0108] Reference Figure 1 , shows a flowchart of the steps of a summary generation method provided by an embodiment of the present application, such as Figure 1 As shown, the summary generation method may include: step 101, step 102, step 103 and step 104.

[0109] Step 101: Obtain the text to be processed.

[0110] In this embodiment, the text to be processed refers to the text for which an abstract needs to be generated. In this example, the text to be processed can be a paper, or the content of a chapter of a novel, etc., which is not limited in this embodiment.

[0111] When it is necessary to generate a summary of the text to be processed, the text to be processed may be obtained.

[0112] After the text to be processed is obtained, step 102 is executed.

[0113] Step 102: Input the text to be processed into a summary generation model, which includes a key sentence extractor and a summary generator.

[0114] A summary generation model is a pre-trained model used to generate summaries of text. In this example, the summary generation model may include a key sentence extractor and a summarizer. In actual training, the key sentence extractor and summarizer can be trained separately and then combined to form the summary generation model. The model training process will be described in detail in the following embodiments and is not limited in this embodiment.

[0115] After obtaining the text to be processed, the text to be processed can be input into the summary generation model.

[0116] After the text to be processed is input into the summary generation model, step 103 is performed.

[0117] Step 103: calling the key sentence extractor to process the text to be processed, obtaining candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and filtering out the top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer.

[0118] After inputting the text to be processed into the summary generation model, the key sentence extractor can be called to process the text to obtain candidate sentences corresponding to the text and the sentence scores of the candidate sentences. Then, the top K (K is a positive integer) candidate sentences with the highest sentence scores can be selected from the candidate sentences. For example, if there are 10 candidate sentences, the order of score from high to low is: sentence 1, sentence 2, sentence 3, ... sentence 10, and K = 2. In this case, two candidate sentences, sentence 1 and sentence 2, are selected, and so on. This implementation process can be described in detail in conjunction with the specific implementation method below.

[0119] In a specific implementation of the present application, the above step 103 may include:

[0120] Sub-step A1: calling the basic network layer to process the text to be processed, and obtaining a full-text feature matrix of the text to be processed and a sentence feature matrix of each candidate sentence in the text to be processed.

[0121] In this embodiment, the key sentence extractor may include: a basic network layer and a scoring network layer.

[0122] After the text to be processed is input into the summary generation model, the basic network layer of the key sentence extractor can be called to process the text to be processed, thereby obtaining a full-text feature matrix for the text to be processed and a sentence feature matrix for each candidate sentence in the text to be processed. Specifically, the text to be processed can be composed of multiple sentences, and the text to be processed can be segmented according to punctuation marks in the text to be processed to obtain a number of candidate sentences. Furthermore, the basic network layer can be used to process the text to be processed and the candidate sentences in the text to be processed, thereby obtaining a full-text feature matrix for the text to be processed and a sentence feature matrix for each candidate sentence in the text to be processed.

[0123] After obtaining the full-text feature matrix of the text to be processed and the sentence feature matrix of each candidate sentence in the text to be processed, sub-step A2 is executed.

[0124] Sub-step A2: calling the scoring network layer to process the full-text feature matrix and the sentence feature matrix to obtain sentence scores corresponding to the candidate sentences.

[0125] After obtaining the full-text feature matrix of the document to be processed and the sentence feature matrix for each candidate sentence in the document to be processed, the scoring network layer can be called to process the full-text feature matrix and the sentence feature matrix to obtain the sentence scores corresponding to the candidate sentences. The processing of the scoring network layer is described in detail in conjunction with the specific implementation method below.

[0126] In a specific implementation of the present application, the above sub-step A2 may include:

[0127] Sub-step B1: calling the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features.

[0128] In this embodiment, the scoring network layer may include: an attention fusion layer, a pooling layer, a feature concatenation layer, and a regression layer.

[0129] After obtaining the full-text feature matrix and the sentence feature matrix, the attention fusion layer can be called to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features. In this example, the attention fusion layer may include: a first attention fusion module, a splicing module, and a second attention fusion module. The process of attention mechanism processing may be: calling the first attention fusion module to process the full-text feature matrix and the sentence feature matrix respectively to obtain the corresponding full-text features and sentence features. Then calling the splicing module to perform splicing processing on the full-text features and sentence features to obtain the second splicing features. Finally, the second attention fusion module may be called to process the second splicing features to obtain attention fusion features.

[0130] After obtaining the attention fusion feature, perform sub-step B2.

[0131] Sub-step B2: Call the pooling layer to perform global mean pooling and global maximum pooling on the attention fusion features to obtain corresponding mean pooling features and maximum pooling features.

[0132] After obtaining the attention fusion features, the pooling layer can be called to perform global mean pooling and global maximum pooling on the attention fusion features to obtain the corresponding mean pooling features and maximum pooling features.

[0133] Sub-step B3: calling the feature splicing layer to perform splicing processing on the mean pooling feature and the maximum pooling feature to obtain a first splicing feature.

[0134] After obtaining the corresponding mean pooling features and maximum pooling features, the feature splicing layer can be called to splice the mean pooling features and the maximum pooling features to obtain the first splicing features.

[0135] Sub-step B4: calling the regression layer to process the first concatenated features to obtain a sentence score corresponding to the candidate sentence.

[0136] After obtaining the first concatenated feature, the regression layer may be called to process the first concatenated feature to obtain a sentence score corresponding to the candidate sentence.

[0137] This embodiment of the application uses a key sentence extractor to deeply mine text semantics and grammatical information, providing rich and accurate features for the scoring network layer. The scoring network layer scores based on these high-quality features. Compared with traditional methods such as simple word frequency statistics, it can more accurately measure the importance of candidate sentences in the full text and improve scoring accuracy.

[0138] After calling the key sentence extractor to process the text to be processed, obtaining candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and filtering out the top K candidate sentences with the highest sentence scores from the candidate sentences, step 104 is executed.

[0139] Step 104: calling the summary generator to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

[0140] After selecting the top K candidate sentences with the highest sentence scores from the candidate sentences, the summary generator can be called to process the combined text of the top K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

[0141] The embodiments of the present application combine extractive and abstractive summary generation methods to solve problems such as incomplete summarization or misunderstanding in the summary generation process. At the same time, compared with traditional extractive or abstractive summary extraction algorithms, it not only avoids the hallucination problem of large models, but also improves the accuracy and efficiency of the generated summary.

[0142] Next, the training process of the summary generation model is described in detail with reference to specific embodiments.

[0143] In a specific implementation of this application, it may also include:

[0144] Step C1: Construct the first summary dataset.

[0145] In this embodiment, a first summary dataset can be constructed first. Specifically, an open source text summary dataset can be obtained and named OriginalDataset to serve as the basis for building this model. To meet the specific requirements of the present invention, OriginalDataset should contain the following key fields: (1) Text ID (id): Assign a unique identifier to each text sample to ensure the uniqueness of the sample in the dataset, which not only facilitates data management, cleaning and preprocessing, but also helps to track samples during model training. (2) Text content (article): This is the core of model training and contains text data used to train the algorithm framework. (3) Text summary (highlights): This field contains summary information manually summarized for each text content, providing a learning target for the model. After obtaining the text summary dataset, the dataset needs to be thoroughly cleaned to ensure the quality and consistency of the data. Data cleaning is mainly divided into two stages: (1) Global cleaning, first perform a data set integrity check, identify and delete samples containing null values, this step ensures that the data used in model training is complete and intact. The dataset is then deduplicated based on the text ID, ensuring that each text ID appears only once in the dataset, thus avoiding data redundancy that could lead to bias in the model training process. (2) Local cleaning: Based on global cleaning, the text content is further purified. This includes removing hypertext tags, special characters, and irrelevant formatting information from the text content. These steps help reduce potential interference with model performance.

[0146] After the first summary dataset is constructed, step C2 is performed.

[0147] Step C2: Based on the first summary dataset, train the key sentence extractor.

[0148] After constructing the first summary dataset, a key sentence extractor can be trained based on the first summary dataset. In this example, the key sentence extractor consists of two parts: a base network layer and a scoring network layer. The training process of the key sentence extractor can be described in detail in conjunction with the following specific implementation method.

[0149] In another specific implementation of the present application, the above step C2 may include:

[0150] Sub-step D1: adopting a mask training method to train a basic network layer based on the first summary data set.

[0151] In this embodiment, when training the key sentence extractor, a mask training method may be first adopted to train a basic network layer based on the first summary data set.

[0152] Masked Language Model (MLM): A masked language model is a self-supervised learning method that randomly masks words in the input text and then trains the model to predict these masked words using the unmasked contextual information. This method does not rely on explicitly annotated data, but instead learns the semantic relationships and contextual dependencies of words through the model's own understanding and reasoning of the text. This training strategy effectively improves the model's performance in various natural language processing (NLP) tasks, enabling it to more accurately capture and understand the complexity of language.

[0153] In the specific implementation, the training of the basic model in the Extractor key sentence extractor. In this step, the basic module responsible for outputting the high-dimensional feature matrix in the Extractor key sentence extractor is trained. The Extractor key sentence extractor consists of two main parts, and its structure is as follows: Figure 2 As shown. Since the main task of the base model (i.e., the basic network layer in this example) is to extract high-dimensional features of the text, in this example, an encoder model can be chosen to complete this task. Among the many training methods, the masked language model (MLM) is selected because of its outstanding performance in capturing the deep semantic relationships of the text. Specifically, the training data set of this step comes from the text content prepared in the above step. The word segmentation tool of the large model can be used to convert the text into tokens, and then a portion of the tokens are randomly selected and masked, which are used as input for model training. This masking strategy not only prompts the model to learn the contextual information of the text, but also enhances its ability to capture key information. Through masked language model training, the base model can learn rich text features, providing strong feature support for subsequent key sentence extraction. When performing mask training, the mask ratio can be set to 15%, that is, a portion of the tokens are randomly selected and masked, which are used as input for model training. The preprocessed tokens are input into the DeBERTa model (i.e., the base model in this embodiment) for masked language model training. Through this process, a learned base model is obtained. DeBERTa is a pre-trained language model developed by Microsoft. It significantly improves the model's language understanding and processing capabilities by introducing two innovative technologies: an attention decoupling mechanism and an enhanced mask decoder. The attention decoupling mechanism separates word representations into two vectors: content and position. This allows the model to more finely distinguish the relationships between words when calculating attention weights. The enhanced mask decoder optimizes the prediction of mask tokens by incorporating absolute position information into the decoding layer, thereby improving the model's understanding of sentence structure. These innovations enable DeBERTa to demonstrate outstanding performance across a variety of natural language processing tasks.

[0154] Tokens: In natural language processing, tokens are the process of breaking text into smaller units, such as words, subwords, characters, or other specific symbols. Tokenization is an important step in NLP, converting raw text into a format that computers can process.

[0155] After the basic network layer is obtained by training based on the first summary data set using the mask training method, sub-step D2 is performed.

[0156] Sub-step D2: constructing a scoring data set based on the text identifier, the text content, the sentence content corresponding to the text content and the text summary, and the sentence score corresponding to the sentence content.

[0157] After the basic network layer is trained based on the first summary dataset using a mask training method, a scoring dataset can be constructed based on the text identifier, text content, sentence content corresponding to the text content and text summary, and sentence scores corresponding to the sentence content.

[0158] The process of constructing a dataset for training the scoring model in Extractor can be as follows: The dataset consists of the following parts: (1) Text ID (id): The text ID from the OriginalDataset dataset. (2) Text content (article): The text content from the OriginalDataset dataset. (3) Sentence content (sentence): The text content is divided into multiple sentences according to the punctuation marks (period, question mark, exclamation mark) in the text content field, and each sentence is used as an independent sample. (4) Sentence score (score): The scoring model ( Figure 2 The Scoder Model in the dataset (i.e., the scoring network layer in this example) is essentially a regression model, so a supervisory signal is needed to train the model. Here, the sentence content field of the dataset and the text summary field of the OriginalDataset dataset are used to evaluate the similarity between the sentence and the summary using the ROUGE-1, ROUGE-2, and ROUGE-L calculation methods, and the average of these evaluation values is obtained. Each sentence has a text id to which it belongs. This id must be the same as the id corresponding to the highlights field in the OriginalDataset, indicating that the sentence and summary are from the same sample. Through the above steps, a structured dataset is constructed and named ExtractorDataset, as shown in Table 1 below, which provides the necessary data support for the training of the scoring model.

[0159] Table 1: ExtractorDataset

[0160]

[0161] In practical applications, sentence types are classified by judging the punctuation marks at the end of the sentence. Exclamatory sentences are represented by -1, declarative sentences are represented by 0, and interrogative sentences are represented by 1.

[0162] After constructing the scoring dataset, perform sub-step D3.

[0163] Sub-step D3: training a scoring network layer based on the scoring dataset.

[0164] After constructing the scoring dataset, the scoring network layer can be trained based on the scoring dataset to obtain a scoring network layer. In this example, the scoring network layer may include: an attention fusion layer, a pooling layer, a feature splicing layer, and a regression layer. Specifically, the training process may be: converting the text content and the sentence content respectively to obtain the text segmentation information corresponding to the text content and the sentence segmentation information corresponding to the sentence content. The attention fusion layer is called to perform a first round of attention mechanism processing on the text segmentation information and the sentence segmentation information respectively to obtain the full-text initial features and the sentence initial features, and a second round of attention mechanism processing is performed on the splicing features of the full-text initial features and the sentence initial features to obtain fusion features. The pooling layer is called to perform global mean pooling and global maximum pooling on the fusion features respectively to obtain the corresponding two pooling features. The regression layer is called to process the two pooling features to obtain the predicted sentence score. Based on the predicted sentence score and the sentence score, the loss value is calculated. When the loss value is within the preset range, the scoring network layer is obtained.

[0165] The specific training process can be: extracting sentence content features based on the attention fusion mechanism. The algorithm implementation process is as follows: Figure 3 As shown. First, through the basic network layer trained in the above steps, the text content field and sentence content field in the ExtractorDataset dataset are segmented and converted into tokens. It should be added here that for the segmentation of the text content field, the maximum sequence length limit can be set to 512, while the maximum sequence length of the sentence content field is set to 128. Although this setting has a different output shape, it maximizes the acquisition of relevant information about the text content and saves the computing resources required for training. The tokens in the text content field and the sentence content field are input into the basic model to obtain the initial feature F of the full text. arti_fea and sentence initial features F sen_fea After the two pass through the first round of attention mechanism, they are spliced along the sequence dimension to obtain the initial fusion feature F arti_sen_fea , and then the feature is processed by the second round of attention mechanism to obtain the attention fusion feature Fatt_arti_sen_fea , and extract the feature part corresponding to the sentence according to the sequence length and perform global mixed pooling processing. Here, we first need to perform two types of pooling on the feature, global mean pooling and global maximum pooling, and then splice the two pooled features to form the final sentence content feature F sen_content_fea , the calculation formula is as follows:

[0166] F arti_sen_fea =[Attention(F arti_fea ),Attention(F sen_fea )]

[0167] F att_arti_sen_fea =[Attention(F arti_sen_fea )

[0168] F sen_content_fea =[GlobalAveragePooling(F att_arti_sen_fea ),GlobalMaxPooling(F att_arti_sen_fea ),]

[0169] In the above steps, the first round of attention allows the initial features of the full text and the initial features of the sentence to highlight their respective key information. The second round of attention, performed after the two are merged, further emphasizes the relevance and importance of the sentence features to the features of the full text. This hierarchical feature learning method can enhance the model's ability to represent input data, reduce the risk of overfitting, and improve model accuracy. The global hybrid pooling method in this step also combines the advantages of global mean pooling, which relatively well preserves all information in the region and is suitable for tasks requiring more contextual information, with the advantages of global maximum pooling, which, by selecting the maximum value, can better capture important features and is less susceptible to interference from small amounts of noise.

[0170] Sentence structure feature construction and scoring model training in Extractor. In this step, the basic structural features of the sentence itself are also incorporated into the subsequent calculations to enhance the model's ability to judge the importance of the sentence. The basic algorithm flow can be as follows: Figure 4As shown. This feature contains three dimensions of values: (1) Sentence length. This part calculates the length of the word list obtained after the word segmentation operation. For shorter sentences, the important information contained is less. (2) Sentence type. This part uses the last punctuation mark of the sentence as the basis for division. Compared with interrogative sentences, exclamatory sentences, declarative sentences tend to describe objective facts. (3) Sentence relative position. This value is calculated as the ratio of the position number of each sentence to the number of sentences in the text. For key summary sentences, they often appear at the beginning and end of the text, so this value also needs to be used as one of the structural features. The structural features finally formed are normalized and dimensionless, and spliced with the output features in the above steps to form a sentence comprehensive feature input to the regression module. The module consists of an input layer, a hidden layer, an output layer, and a ReLU activation layer. By inputting the features of the specified dimension, the output layer finally outputs the regression value of the feature. This value is used to calculate the loss value in the sentence scoring field in the ExtractorDataset of the above step, and backpropagation is performed to update the parameters of the regression module and attention fusion module in the scoring model.

[0171] In practical applications, the relative position of a sentence is determined by the ratio of the number of sentences in the text to the number of sentences in the text. Since sentence length can be large, while other values range from -1 to 1 and from 0 to 1, it is necessary to normalize the sentence length dimension. Here, we use the minimum and maximum normalization method, as shown in the following formula:

[0172]

[0173] Through the above formula, the length of the sentence is mapped to between 0 and 1. And the processed sentence structure feature F struc_fea , and F in the above steps sen_content_fea Splicing to get the sentence comprehensive feature F final_sen_fea, the feature length is the sum of the two feature lengths, that is, 2051 dimensions. This feature is used as the input of the regression module in the scoring model. The module consists of the following parts: (1) Input layer, which needs to receive a 2051-dimensional feature and output a 1024-dimensional feature. (2) ReLU activation layer, which introduces nonlinearity, improves computational efficiency and promotes sparsity. (3) Hidden layer, the input is a 1024-dimensional feature and the output is a 512-dimensional feature. (4) Output layer, the input is a 512-dimensional feature and the output is a 1-dimensional value, that is, the predicted regression value. Finally, the above parts are combined into a regression module in the form of input layer-ReLU activation layer-hidden layer-ReLU activation layer-output layer. During the training of the scoring model, Adam is used to jointly update the multi-head attention mechanism parameters and regression module parameters in the attention fusion module, and SmoothL1 is used as the regression loss function. The loss function formula is as follows:

[0174]

[0175] where x n Represents the predicted output value of the scoring model, y n It represents the true label value corresponding to the sentence, that is, the value corresponding to the score field. Beta defaults to 1, l n The scoring model is trained using the training set of ExtractorDataset until the loss function no longer decreases, and then the validation set of ExtractorDataset is used for verification, and the mean square error is used as the judgment standard for the prediction accuracy of the scoring model.

[0176] Sub-step D4: combining the base network layer and the scoring network layer to obtain the key sentence extractor.

[0177] After training to obtain the base network layer and the scoring network layer, the base network layer and the scoring network layer can be combined to obtain a key sentence extractor.

[0178] After the key sentence extractor is trained, step C3 is performed.

[0179] Step C3: constructing a second summary dataset based on the first summary dataset and the key sentence extractor.

[0180] After the key sentence extractor is trained, a second summary dataset can be constructed based on the first summary dataset and the key sentence extractor.

[0181] After the second summary data set is constructed, step C4 is performed.

[0182] Step C4: Based on the second summary data set, the summary generator is trained.

[0183] After constructing the second summary dataset, a summary generator can be trained based on the second summary dataset. Specifically, the second summary dataset may include: a text identifier, target text content, and a text summary. The target text content is the text content obtained by combining the top K candidate sentences with the highest scores in descending order of score to form additional content, and appending it to the beginning of the original text in the text content field of the first dataset. In this example, a hard prompt fine-tuning method can be used to guide the generator training based on the target text content and text summary to obtain a summary generator.

[0184] The dataset construction and training of the Generator summary generator. This step adopts the hard prompt fine-tuning method (Hard Prompt) in the large model Prompt-Tuning fine-tuning technology and makes innovative improvements. In this step, the ROUGE score is calculated using each sentence in the original text and the corresponding label summary field of the original text. The K sentences with the highest scores are used to form a key sentence group and place it at the beginning of the original text to achieve data reinforcement. At the same time, the hard prompt fine-tuning method is used to guide the training of the generator, allowing the model to generate a summary after the key sentence group at the beginning of the associated text, thereby avoiding the illusion of the large model while increasing the accuracy of summary generation.

[0185] Before training the Generator, a training set GeneratorDataset for training the summary generator must be specially constructed. The dataset must include the following parts: (1) Text ID, which comes from the text ID field of OriginalDataset. (2) New text content, through the sentence content and corresponding sentence scores of the ExtractorDataset dataset in step S3, the top-k sentences are obtained in descending order according to the scores, where the value of k can be flexibly set according to the length of the text. The k sentences are combined to form additional content and attached to the beginning of the original text in the text content field of ExtractorDataset to form new text content. (3) Text summary, which comes from the text summary field of OriginalDataset. After building the proprietary dataset, a variety of different hard prompt templates can be designed and selected based on the training effect.

[0186] Step C5: combining the key sentence extractor and the summary generator to obtain a summary generation model.

[0187] After training the key sentence extractor and summary generator, the key sentence extractor and summary generator can be combined to obtain a summary generation model. Specifically, the Extractor key sentence extractor and the Generator summary content generator are combined and tested and evaluated. The trained Extractor key sentence extractor and the Generator summary generator are combined, and the structure is as follows Figure 5 As shown. In the whole process, the data will first enter the basic model of the extractor, and the basic model will output the full text and the initial feature matrix of the sentence. Then the two feature matrices will enter the scoring model to calculate the score of each sentence. The top k sentences with the highest scores are selected as the key sentence group by sorting them in descending order. They are concatenated with the original text and placed at the beginning of the text to form new text content and pushed to the summary generator. At the same time, the prompt template in the above steps must be combined to guide the generation of the summary results. Finally, ROUGE-1, ROUGE-2, and ROUGE-L indicators are used to evaluate the summary effect. The overall data flow is as follows Figure 6 shown.

[0188] Among them, ROUGE scoring is a method for automatically evaluating the performance of text generation or summary systems. The score is calculated by comparing the similarity between the text generated by the system and the reference standard text. It is mainly used in fields such as machine translation, automatic summarization, and question-answer generation to measure the degree of match between the generated results and the standard results. Commonly used ones include ROUGE-1, ROUGE-2, and ROUGE-L, among which ROUGE-1 and ROUGE-2 calculate the overlap of 1-gram and 2-gram respectively, that is, the degree of overlap between a single word and two adjacent words. ROUGE-L is calculated based on the longest common subsequence, focusing on the length of the longest common subsequence between the output text and the target. Specifically, the ROUGE score is calculated as the average of the three values of ROUGE-1, ROUGE-2, and ROUGE-L. The three sentences with the highest scores are then combined to form a key sentence group, and spliced to form additional content as shown below:

[0189] "Key Sentences:(1)Sentence1(2)Sentence2(3)Sentence3"

[0190] The additional content is then spliced into the beginning of the article to form new text content, and "Article:" is added before the article to assist the output of the summary generator. (3) Highlights: This field directly references the highlights field in the OriginalDataset dataset. The fine-tuning method for the large language model this time adopts the hard prompt method in Prompt-Tuning, so it is necessary to set a variety of hard prompt templates for fine-tuning the model. The template design needs to focus on two points. One is to let the model pay attention to the key sentence group at the beginning of the article, and the other is to let the model perform the summary generation task. Therefore, two templates are set according to the above requirements:

[0191] (1)“Use the Key Sentences and generate a summary:”

[0192] (2) "According to Key Sentences,Summarize Article:"

[0193] Finally, the newly constructed text content is fed into the model using the training set, with the highlights field as the supervisory signal. The model is fine-tuned using the two prompt templates described above to reduce the loss. The validation set is then used to evaluate the model's training performance. Note that the model evaluation criterion still uses the mean of ROUGE-1, ROUGE-2, and ROUGE-L.

[0194] The embodiment of the present application generates high-dimensional features for the full text and sentences through an encoder model, and performs fusion splicing after the first round of attention mechanism, and then performs a second round of attention mechanism to associate the sentence features with the full text features, and then takes out the part of the sentence from the features in the fusion splicing and performs global hybrid pooling processing to obtain the final sentence content features. The algorithm first uses a dual-branch structure to set different maximum cutoff values to process the full text and sentences. This method can obtain all the content features of the full text as much as possible to avoid missing important information. At the same time, a smaller maximum cutoff value to process sentences can greatly save computing resources and thus improve computing efficiency. Therefore, the attention fusion mechanism algorithm and the global hybrid pooling algorithm can be better combined with the full text content in the process of obtaining sentence content features, highlighting the important parts of the sentence relative to the full text, and also improving the calculation speed of the model.

[0195] At the same time, an innovative algorithm for scoring the importance of sentences to texts was also disclosed. In addition to the relevance and importance of sentence content relative to the full text, it was also observed that the basic structural features of sentences also play an important role in the full text. Therefore, the sentence content features and sentence structure features output by the attention fusion module are spliced together to form a comprehensive sentence feature, which is used as the input of the regression module in the scoring model. At the same time, ROUGE calculates the score of each sentence and summary as a supervision signal for the regression module. This method not only significantly improves the accuracy of the model's regression prediction of sentences, but also accelerates the convergence speed during training.

[0196] Furthermore, a hybrid summary generation model architecture is disclosed. Compared with traditional extractive and abstract summary generation methods, this model has the advantages of both. The key sentence extractor outputs the most important K sentences to form a sentence group, and this sentence group is placed at the beginning of the original text as a prompt content for the summary generator to generate a text summary. This method not only utilizes the large model to connect the context to output more comprehensive results, but also the key sentence group output by the key sentence extractor makes the results output by the large model more closely related to the topic, and also avoids the problem of large model hallucination, that is, the generated content contains parts that are not mentioned in the original text. It should be noted that under this architecture, the fine-tuning method of the summary generator needs to be further improved. Unlike the traditional Hard-Prompt, the set prompt template not only requires the model to guide the summary task of the original text, but also requires the model to first combine the content of the key sentence group. Therefore, the first step is to splice the key sentence group and the original text to form new text content, and then set the prompt template and perform fine-tuning training.

[0197] The summary generation method provided in the embodiment of the present application obtains the text to be processed. The text to be processed is input into the summary generation model, and the summary generation model includes: a key sentence extractor and a summary generator. The key sentence extractor is called to process the text to be processed, and the candidate sentences corresponding to the text to be processed and the sentence scores corresponding to the candidate sentences are obtained, and the top K candidate sentences with the highest sentence scores are screened out from the candidate sentences, where K is a positive integer. The summary generator is called to process the combined text of the top K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed. The embodiment of the present application solves the problems of incomplete summarization or misunderstanding in the summary generation process by combining the extraction type and abstract type summary generation methods. At the same time, compared with the traditional extraction type or abstract type summary extraction algorithm, it not only avoids the hallucination problem of the large model, but also improves the accuracy and efficiency of the generated summary.

[0198] Reference Figure 7 , which shows a schematic structural diagram of a summary generation device provided in an embodiment of the present application, such as Figure 7As shown, the summary generating apparatus 700 may include the following modules:

[0199] A text acquisition module 710 is used to acquire the text to be processed;

[0200] A text input module 720 is used to input the text to be processed into a summary generation model, wherein the summary generation model includes: a key sentence extractor and a summary generator;

[0201] The sentence acquisition module 730 is configured to call the key sentence extractor to process the text to be processed, obtain candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and select the top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer;

[0202] The summary generation module 740 is configured to call the summary generator to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template, and generate a text summary corresponding to the text to be processed.

[0203] Optionally, the key sentence extractor includes: a basic network layer and a scoring network layer,

[0204] The statement acquisition module includes:

[0205] A feature matrix acquisition unit, configured to call the basic network layer to process the text to be processed, and obtain a full-text feature matrix of the text to be processed and a sentence feature matrix of each candidate sentence in the text to be processed;

[0206] The sentence score acquisition unit is used to call the scoring network layer to process the full-text feature matrix and the sentence feature matrix to obtain the sentence score corresponding to the candidate sentence.

[0207] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0208] The sentence scoring acquisition unit includes:

[0209] A fusion feature acquisition subunit is used to call the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features;

[0210] A pooling feature acquisition subunit is used to call the pooling layer to perform global mean pooling processing and global maximum pooling processing on the attention fusion feature, respectively, to obtain corresponding mean pooling features and maximum pooling features;

[0211] A first feature acquisition subunit is configured to call the feature splicing layer to perform splicing processing on the mean pooling feature and the maximum pooling feature to obtain a first splicing feature;

[0212] The sentence score acquisition subunit is used to call the regression layer to process the first splicing feature to obtain the sentence score corresponding to the candidate sentence.

[0213] Optionally, the attention fusion layer includes: a first attention fusion module, a splicing module and a second attention fusion module.

[0214] The fusion feature acquisition subunit includes:

[0215] a feature acquisition subunit, configured to call the first attention fusion module to process the full-text feature matrix and the sentence feature matrix respectively to obtain corresponding full-text features and sentence features;

[0216] A second feature acquisition subunit is configured to call the splicing module to perform splicing processing on the full-text feature and the sentence feature to obtain a second splicing feature;

[0217] The attention feature acquisition subunit is used to call the second attention fusion module to process the second splicing feature to obtain the attention fusion feature.

[0218] Optionally, the device further comprises:

[0219] A first data set construction module, used to construct a first summary data set;

[0220] An extractor training module, configured to train the key sentence extractor based on the first summary dataset;

[0221] A second data set construction module, configured to construct a second summary data set based on the first summary data set and the key sentence extractor;

[0222] A generator training module, configured to train the summary generator based on the second summary dataset;

[0223] The model acquisition module is used to combine the key sentence extractor and the summary generator to obtain a summary generation model.

[0224] Optionally, the first summary data set includes: text identifier, text content and text summary,

[0225] The extractor training module includes:

[0226] A base layer training unit, configured to train a base network layer based on the first summary data set using a mask training method;

[0227] a scoring set construction unit, configured to construct a scoring data set based on the text identifier, the text content, the sentence content corresponding to the text content and the text summary, and the sentence score corresponding to the sentence content;

[0228] A scoring layer training unit, configured to train a scoring network layer based on the scoring data set;

[0229] An extractor acquisition unit is used to combine the basic network layer and the scoring network layer to obtain the key sentence extractor.

[0230] Optionally, the scoring network layer includes: an attention fusion layer, a pooling layer, a feature splicing layer and a regression layer.

[0231] The scoring layer training unit includes:

[0232] A word segmentation information acquisition subunit is used to convert the text content and the sentence content respectively to obtain text word segmentation information corresponding to the text content and sentence word segmentation information corresponding to the sentence content;

[0233] a feature fusion subunit, configured to call the attention fusion layer to perform a first round of attention mechanism processing on the text segmentation information and the sentence segmentation information respectively to obtain full-text initial features and sentence initial features, and perform a second round of attention mechanism processing on the concatenated features of the full-text initial features and the sentence initial features to obtain fused features;

[0234] A feature pooling subunit, configured to call the pooling layer to perform global mean pooling and global maximum pooling on the fused features, respectively, to obtain two corresponding pooling features;

[0235] A prediction score acquisition subunit, configured to call the regression layer to process the two pooled features to obtain a prediction sentence score;

[0236] a loss value calculation subunit, configured to calculate a loss value based on the predicted sentence score and the sentence score;

[0237] The scoring layer acquisition subunit is used to obtain the scoring network layer when the loss value is within a preset range.

[0238] Optionally, the second summary dataset includes: a text identifier, target text content, and a text summary, wherein the target text content is obtained by combining the top K candidate sentences with the highest scores in descending order of scores to form additional content, and appending the resultant content to the beginning of the original text in the text content field of the first dataset;

[0239] The generator training module includes:

[0240] The generator acquisition unit is used to adopt a hard prompt fine-tuning method to guide the training of the generator according to the target text content and the text summary to obtain the summary generator.

[0241] The summary generation device provided in the embodiment of the present application obtains the text to be processed. The text to be processed is input into the summary generation model, and the summary generation model includes: a key sentence extractor and a summary generator. The key sentence extractor is called to process the text to be processed, and the candidate sentences corresponding to the text to be processed and the sentence scores corresponding to the candidate sentences are obtained, and the top K candidate sentences with the highest sentence scores are screened out from the candidate sentences, where K is a positive integer. The summary generator is called to process the combined text of the top K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed. The embodiment of the present application solves the problems of incomplete summarization or misunderstanding in the summary generation process by combining the extraction type and abstract type summary generation methods. At the same time, compared with the traditional extraction type or abstract type summary extraction algorithm, it not only avoids the hallucination problem of the large model, but also improves the accuracy and efficiency of the generated summary.

[0242] An embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned summary generation method when executed by the processor.

[0243] Figure 8 FIG. 8 is a schematic structural diagram of an electronic device 800 according to an embodiment of the present invention. Figure 8 As shown, the electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0244] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, a microphone, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0245] The various processes and procedures described above may be executed by the processing unit 801. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more actions in the method described above may be performed.

[0246] In addition, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned summary generation method when executed by a processor.

[0247] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0248] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0249] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminals (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal generate instructions for implementing the steps in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0250] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0251] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal so that a series of operational steps are executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable terminal for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0252] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0253] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal comprising the element.

[0254] The above describes in detail a summary generation method, a summary generation device, an electronic device, and a computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core concept of the present application. At the same time, for those skilled in the art, based on the concept of the present application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting the present application.

Claims

1. A method for generating a summary, characterized in that: The method comprises: Get the text to be processed; Inputting the text to be processed into a summary generation model, wherein the summary generation model includes: a key sentence extractor and a summary generator; Calling the key sentence extractor to process the text to be processed, obtaining candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and screening out top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer; The summary generator is called to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

2. The method according to claim 1, characterized in that The key sentence extractor includes: a basic network layer and a scoring network layer, The calling of the key sentence extractor to process the text to be processed to obtain candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences includes: Calling the basic network layer to process the text to be processed to obtain a full-text feature matrix of the text to be processed and a sentence feature matrix of each candidate sentence in the text to be processed; The scoring network layer is called to process the full-text feature matrix and the sentence feature matrix to obtain sentence scores corresponding to the candidate sentences.

3. The method according to claim 2, characterized in that The scoring network layer includes: attention fusion layer, pooling layer, feature splicing layer and regression layer. The calling of the scoring network layer to process the full-text feature matrix and the sentence feature matrix to obtain a sentence score corresponding to the candidate sentence includes: Calling the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features; Calling the pooling layer to perform global mean pooling processing and global maximum pooling processing on the attention fusion feature, respectively, to obtain corresponding mean pooling features and maximum pooling features; Calling the feature splicing layer to perform splicing processing on the mean pooling feature and the maximum pooling feature to obtain a first splicing feature; The regression layer is called to process the first concatenated feature to obtain a sentence score corresponding to the candidate sentence.

4. The method according to claim 3, characterized in that The attention fusion layer includes: a first attention fusion module, a splicing module and a second attention fusion module. The calling of the attention fusion layer to perform attention mechanism processing on the full-text feature matrix and the sentence feature matrix to obtain attention fusion features includes: Calling the first attention fusion module to process the full-text feature matrix and the sentence feature matrix respectively to obtain corresponding full-text features and sentence features; Calling the splicing module to perform splicing processing on the full-text feature and the sentence feature to obtain a second splicing feature; The second attention fusion module is called to process the second splicing feature to obtain the attention fusion feature.

5. The method according to claim 1, wherein Before inputting the to-be-processed text into the summary generation model, the method further includes: Constructing the first summary dataset; Based on the first summary dataset, training the key sentence extractor; Constructing a second summary dataset based on the first summary dataset and the key sentence extractor; Based on the second summary data set, training the summary generator; The key sentence extractor and the summary generator are combined to obtain a summary generation model.

6. The method according to claim 5, characterized in that The first summary data set includes: text identifier, text content and text summary, The step of training the key sentence extractor based on the first summary dataset includes: Using a mask training method, a basic network layer is trained based on the first summary data set; Constructing a scoring data set based on the text identifier, the text content, the sentence content corresponding to the text content and the text summary, and the sentence score corresponding to the sentence content; A scoring network layer is obtained by training based on the scoring data set; The base network layer and the scoring network layer are combined to obtain the key sentence extractor.

7. The method according to claim 6, characterized in that The scoring network layer includes: attention fusion layer, pooling layer, feature splicing layer and regression layer. The scoring network layer is obtained by training based on the scoring dataset, comprising: Converting the text content and the sentence content respectively to obtain text segmentation information corresponding to the text content and sentence segmentation information corresponding to the sentence content; Calling the attention fusion layer to perform a first round of attention mechanism processing on the text segmentation information and the sentence segmentation information respectively to obtain full-text initial features and sentence initial features, and performing a second round of attention mechanism processing on the concatenated features of the full-text initial features and the sentence initial features to obtain fused features; Calling the pooling layer to perform global mean pooling and global maximum pooling on the fusion features respectively to obtain two corresponding pooling features; Calling the regression layer to process the two pooled features to obtain a predicted sentence score; Calculating a loss value based on the predicted sentence score and the sentence score; When the loss value is within a preset range, the scoring network layer is obtained.

8. The method according to claim 5, characterized in that The second summary dataset includes: a text identifier, target text content, and a text summary. The target text content is obtained by combining the top K candidate sentences with the highest scores in descending order of scores to form additional content, and appending it to the beginning of the original text in the text content field of the first dataset. The step of training the summary generator based on the second summary dataset includes: A hard prompt fine-tuning method is adopted to guide the training of the generator according to the target text content and the text summary to obtain the summary generator.

9. A summary generation device, characterized in that: The device comprises: A text acquisition module is used to obtain the text to be processed; A text input module, configured to input the text to be processed into a summary generation model, wherein the summary generation model includes: a key sentence extractor and a summary generator; a sentence acquisition module, configured to call the key sentence extractor to process the text to be processed, obtain candidate sentences corresponding to the text to be processed and sentence scores corresponding to the candidate sentences, and screen out top K candidate sentences with the highest sentence scores from the candidate sentences, where K is a positive integer; The summary generation module is used to call the summary generator to process the combined text of the first K candidate sentences and the text to be processed based on the built-in prompt word template to generate a text summary corresponding to the text to be processed.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the abstract generation method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the abstract generation method according to any one of claims 1 to 8.

Images