Text abstract generation method and device, equipment and storage medium
By inserting preset markers into the BART model and performing importance scoring, the problem of insufficient inter-sentence connection ability of the BART model in multi-sentence text summarization is solved, thus improving the accuracy of the generated summaries.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN POLYTECHNIC UNIVERSITY
- Filing Date
- 2022-07-28
- Publication Date
- 2026-07-24
AI Technical Summary
The existing BART model is not outstanding in capturing the relationships between sentences in multi-sentence text summarization tasks because it does not modify the embedding method of the BERT model, resulting in inaccurate text summaries.
By inserting preset markers into the input text, multiple sentences are segmented into target sentences. The target sentences are then input into the BART model to generate representation vectors. After the vectors are scored for importance, sentences with high importance are selected to generate summary text.
It improves the accuracy of text summarization, ensuring that the generated summary is more closely related to the most important statements in the input text, resulting in a more accurate summary.
Smart Images
Figure CN115422941B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a text summarization method, apparatus, device, and storage medium. Background Technology
[0002] The rapid development of the Internet in the 21st century has led to a surge in massive amounts of text data, making it difficult for users to quickly extract the main information from texts. How to quickly and efficiently extract the useful information needed from massive amounts of text has become an urgent problem to be solved.
[0003] The pre-trained language model BART, proposed by Mike Lewis et al., is a variant of BERT. If BERT is a pre-trained language model using only the Transformer-Encoder structure, then BART uses the entire Transformer model structure. Compared to BERT, it doesn't degrade performance on natural language understanding tasks and shows significant improvement on natural language generation tasks. However, it doesn't modify the embedding method of the traditional BERT model. Therefore, for text summarization tasks with multiple sentences, its ability to capture inter-sentence connections is not outstanding, resulting in less accurate results.
[0004] Application content
[0005] The main objective of this application is to provide a text summarization method, apparatus, device, and storage medium, which aims to improve the accuracy of generated text summaries.
[0006] To achieve the above objectives, this application provides a text summarization method, the method comprising:
[0007] Obtain the input text; the input text includes multiple statements;
[0008] Multiple preset flags are inserted into multiple statements to obtain multiple target statements;
[0009] Multiple target statements are input into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART;
[0010] Based on the multiple representation vectors, an importance score is obtained for each of the representation vectors;
[0011] Based on the importance score and the target statement, a summary text is generated.
[0012] Optionally, multiple preset flags are inserted into multiple statements to obtain multiple target statements, including:
[0013] Insert the first preset flag before the first character of each of the statements;
[0014] The second preset flag is inserted after the last character of each statement to obtain multiple target statements.
[0015] Optionally, the BART model includes an embedding layer, which takes multiple target statements as input to the pre-trained BART model to obtain multiple representation vectors output by the pre-trained BART model, including:
[0016] Multiple target statements are input into the embedding layer of the pre-trained model BART;
[0017] The embedding layer performs word embedding operations on the target statement to obtain multiple representation vectors.
[0018] Optionally, the embedding layer performs word embedding operations on the target statement to obtain multiple representation vectors, including:
[0019] Determine whether the target statement is an odd-numbered target statement;
[0020] If so, the first segmentation embedding method is selected to perform word embedding on the target statement to obtain multiple representation vectors.
[0021] Optionally, after determining whether the target statement is an odd-numbered representation vector, the method further includes:
[0022] If not, then the second segmented embedding method is selected to embed the target statement, resulting in multiple representation vectors.
[0023] Optionally, the step of inputting multiple target statements into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART includes:
[0024] Multiple target statements are input into the pre-trained model BART to obtain multiple representation vectors and multiple context information vectors output by the pre-trained model BART;
[0025] The step of obtaining an importance score for each of the multiple representation vectors includes:
[0026] Based on the multiple representation vectors and the multiple context information vectors, the overall information of the input text is obtained;
[0027] For each statement, the representation vector is compared with the overall information to obtain the correlation comparison result, and the correlation comparison result is used as the importance score of each representation vector.
[0028] Optionally, based on the importance score and the target statement, a summary text is generated, including:
[0029] The target statements corresponding to the representation vectors whose importance scores are greater than or equal to a preset threshold are selected to obtain the filtered statements;
[0030] The summary text is obtained based on all the filtering statements.
[0031] Furthermore, to achieve the above objectives, this application also provides a text summarization generation apparatus, the apparatus comprising:
[0032] The text acquisition module is used to acquire input text; the input text includes multiple statements.
[0033] The flag insertion module is used to insert multiple preset flags into multiple statements to obtain multiple target statements;
[0034] The vector generation module is used to input multiple target statements into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART.
[0035] The scoring module is used to obtain an importance score for each of the multiple representation vectors.
[0036] The summary generation module is used to generate summary text based on the importance score and the target statement.
[0037] This application also provides a text summarization generation device, including: a processor, a memory, and a text summarization generation program stored in the memory, wherein when the text summarization generation program is executed by the processor, it implements the steps described in any possible implementation of the first aspect.
[0038] This application also provides a computer-readable storage medium storing a text summarization program thereon, which, when executed by a processor, implements the steps described in any possible implementation of the first aspect.
[0039] This application proposes a text summarization method. It involves acquiring multiple sentences from the input text, inserting preset markers into these sentences to obtain multiple target sentences, inputting these target sentences into a BART model to obtain multiple representation vectors corresponding to the target sentences, assigning importance scores to these representation vectors, and then generating a summary text based on the importance scores and the corresponding target sentences. In other words, this application assigns importance scores to all representation vectors to determine the importance of each sentence in the input text, and then generates a summary text based on the importance scores of the representation vectors and the target sentences. This results in a higher correlation between the generated summary text and the target sentences with higher importance in the input text, thus improving the accuracy of the generated text summary. Attached Figure Description
[0040] Figure 1 A schematic diagram of the structure of the text digest generation device for the hardware operating environment involved in this application;
[0041] Figure 2 This is a flowchart illustrating the first embodiment of the text digest generation method of this application;
[0042] Figure 3 This is a flowchart illustrating the second embodiment of the text digest generation method of this application;
[0043] Figure 4 This is a schematic diagram of the functional modules of the text summarization device involved in this application.
[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0045] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0046] Because existing technologies directly use BART for text summarization extraction, and BART does not modify the embedding method of the traditional BERT model, the results are not accurate enough for text summarization tasks with multiple sentences because its ability to capture the connections between sentences is not outstanding.
[0047] To address this, this application proposes a text summarization method. This method involves acquiring multiple sentences from the input text, inserting preset markers into these sentences to obtain multiple target sentences, inputting these target sentences into a BART model to obtain multiple representation vectors corresponding to the target sentences, assigning importance scores to these representation vectors, and then generating a summary text based on the importance scores and the corresponding target sentences. By assigning importance scores to the representation vectors, the importance of each sentence in the input text is determined, thus improving the accuracy of obtaining the text summary.
[0048] like Figure 1 As shown, the text summarization device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.
[0049] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the text summarization device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0050] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include a control system, a data acquisition module, a data calculation module, a data judgment module, and a text summary generation program.
[0051] like Figure 1 As shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the server of the present invention can be set in the server, and the server calls the text digest generation program stored in the memory 1005 through the processor 1001 and executes the text digest generation method provided in the embodiment of the present invention.
[0052] Based on, but not limited to, the above hardware structure, this invention provides a first embodiment of a text summarization generation method. (Refer to...) Figure 2 , Figure 2 This is a schematic flowchart of the first embodiment of the text summarization method of the present invention.
[0053] Reference Figure 2 The first embodiment of this application provides a text summarization method, the method comprising:
[0054] S10: Obtain the input text; the input text includes multiple statements;
[0055] The input text can be an article or a piece of text.
[0056] S20: Insert multiple preset flags into multiple statements to obtain multiple target statements;
[0057] Preset flags are used to split multiple sentences in the input text, embedding the flags in each sentence.
[0058] The preset markers are inserted into the continuous sentences in the input text to split the continuous sentences into individual sentences, thus obtaining the target sentences. For example, the sentence "sent one, sent two" is obtained by inserting the preset markers [C] and [S] into the sentence, resulting in "[C]sent one[S], [C]sent two[S]".
[0059] S30: Input multiple target statements into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART;
[0060] The pre-trained model BART is a pre-trained model based on a denoising autoencoder seq2seq structure. In the pre-training stage, the core pre-training process is to use an arbitrary noise function to corrupt the original text input to the BART model during pre-training. In the encoder part of the BART model, the corrupted input text is encoded, calculated, and features are extracted. The decoder uses cross-attention and the hidden state results of the last layer of the encoder to perform attention aggregation calculation, and then restores and reconstructs the corrupted document in an autoregressive manner.
[0061] A representation vector is a vector that can represent a target statement, and one representation vector corresponds to only one target statement.
[0062] All target statements are sent to the pre-trained model BART. The BART model generates a corresponding representation vector for each target statement. For example, the statements “[C]sent one[S],[C]sent two[S]” will generate representation vector A for “[C]sent two[S]” and representation vector B for “[C]sent two[S]” after passing through the pre-trained model BART.
[0063] S40: Based on multiple representation vectors, obtain the importance score of each representation vector;
[0064] The importance score is calculated by comparing the overall information that can represent the input text with the sentences represented by the representation vector. Each sentence is scored based on how much overall information it can contain. The resulting score is the importance score of the representation vector, which indicates the importance of the target sentence in the entire input text.
[0065] It is important to understand that the importance score of each representation vector can be calculated through the function of the transform model in the pre-trained BART model.
[0066] S50: Generate summary text based on importance score and target statement;
[0067] Specifically, in the process of generating summary text, target sentences with higher importance scores can be given priority consideration, that is, target sentences with higher importance scores have higher weight, while target sentences with lower importance scores are given secondary consideration or not considered, that is, target sentences with lower importance scores have lower weight. As a result, the final generated summary text has a higher correlation with the target sentences with higher importance scores, so that the final generated summary text can more accurately reflect the input text.
[0068] For example, based on the importance scores of the representation vectors, the target sentences corresponding to the representation vectors with high scores can be obtained. All target sentences with high importance scores can be combined to generate summary text. For instance, if the importance score of representation vector A1 is 90 points, the importance score of representation vector B1 is 85 points, and the importance score of representation vector A2 is 40 points, then the target sentences corresponding to representation vector A1 and representation vector B1 can be combined to generate summary text.
[0069] This application embodiment obtains multiple sentences from the input text, inserts preset flags into the multiple sentences to obtain multiple target sentences, inputs the multiple target sentences into the BART model to obtain multiple representation vectors corresponding to the target sentences, scores the importance of the sentences in the entire input text based on the multiple representation vectors, and then generates summary text based on the importance scores and the target sentences corresponding to the representation vectors, thereby improving the accuracy of obtaining text summaries.
[0070] In this embodiment, by inserting preset flags into the target sentences input to the pre-trained BART, multiple consecutive sentences are segmented, further enhancing the uniqueness of each sentence. After generating representation vectors, the importance of each representation vector is scored. Based on the score of each representation vector, the target sentences corresponding to the high-scoring representation vectors are combined to generate summary text. This solves the problem of inaccurate summary extraction caused by the BART model's poor ability to distinguish multiple sentences in the prior art.
[0071] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the text digest generation method of this application, based on the above. Figure 2 The illustrated embodiment presents a second embodiment of the text summarization generation method of the present invention.
[0072] In this embodiment, step S20 includes:
[0073] S21: Insert the first preset flag before the first character of each statement;
[0074] The first preset flag is the flag [CLS], which is used to separate two consecutive statements. It is placed before the first character of the statement. For example, for the statement "sent one", after inserting the first preset flag, it becomes "[CLS]sent one".
[0075] S22: Insert the second preset flag after the last character of each statement to obtain the target statement;
[0076] The second preset flag is the [SEP] flag, which is used to separate two consecutive statements. It is placed before the last character of the statement. For example, for the statement "sent one", after inserting the first preset flag and the second preset flag, we get "[CLS]sent one[SEP]".
[0077] In this embodiment, step S30 includes:
[0078] S31: Determine if the target statement is the odd-numbered target statement;
[0079] S32: If so, then select the first segmentation embedding method to perform word embedding on the target sentence to obtain multiple representation vectors;
[0080] S33, if not, then select the second segmentation embedding method to perform word embedding on the target sentence to obtain multiple representation vectors;
[0081] The first sentence in the input text is taken as the first sentence, the second sentence as the second sentence, and so on. The process is repeated to determine whether the identified sentence is the odd-numbered or even-numbered sentence. It is understood that the Segment Embedding layer only has two vector representations. Therefore, if it is the odd-numbered sentence, the first embedding method is selected, and if it is the even-numbered sentence, the second embedding method is selected. For example, if the input sentence set is [sent1, sent2, sent3, sent4], if the current sentence is determined to be the odd-numbered sentence, the segment embedding vector Ea is selected, and if the current sentence is the even-numbered sentence, the segment embedding vector Eb is selected. Therefore, the embedded vector is [Ea, Eb, Ea, Eb].
[0082] S34: Input multiple target statements into the pre-trained model BART to obtain multiple representation vectors and multiple context information vectors output by the pre-trained model BART;
[0083] Contextual information can be obtained through the transform model or the Long Short-Term Memory (LSTM) network.
[0084] If LSTM is used to obtain the contextual information of a sentence, the obtained target sentence is used as the input of LSTM, and the contextual semantics of the sentence are extracted at the sentence level using LSTM.
[0085] If the Transform model is used to obtain contextual information, the attention mechanism in Transform is used. The attention mechanism maps the word-embedded representation vectors to three new subspace representations, q, k, and v. Based on the similarity between each target vector q and each k, the weight coefficient of each k is obtained. Then, the weight coefficients are weighted and summed with v to obtain the correlation value of each vector. Based on this correlation value, the contextual information is obtained.
[0086] The embedding layer of the pre-trained model BART is used to embed words in the input sentence, converting each word in the input sentence into a representation vector. The embedding layer is divided into Token embedding, Segment Embedding and Position Embedding.
[0087] The token embedding layer converts each word in the input text into a fixed-dimensional vector. For example, “[CLS]sent one[SEP][CLS]sent two[SEP]” generates vectors E[cls], Esent, Eone, E[sep], E[cls], Esent, Etwo, E[sep].
[0088] The Segment Embedding layer is used to distinguish between two consecutive sentences. This layer has only two vector representations: the first vector assigns vector Ea to each token of the first sentence, and the second vector assigns vector Eb to each token of the second sentence. For example, if the input text is "[CLS]sent one[SEP][CLS]sent two[SEP]", then the generated vectors are Ea, Ea, Ea, Ea, Eb, Eb, Eb, Eb.
[0089] The Position Embedding layer is used to represent the order of vectors. For example, if the input text is “[CLS]sent one[SEP][CLS]sent two[SEP]”, then the generated vectors are E1, E2, E3, E4, E5, E6, E7, and E8.
[0090] The sum of all vectors generated by the statement input to the embedding layer is used as the representation vector of the target statement. In other words, the sum of all vectors generated by the statement in the Token embedding layer, Segment Embedding layer, and Position Embedding layer is used as the representation vector of the target statement.
[0091] In this embodiment, step S40 includes:
[0092] S41: Based on multiple representation vectors and multiple context information vectors, obtain the overall information of the input text;
[0093] Overall information can represent the main theme of the entire input text.
[0094] S42: For each statement, compare the relevance of the representation vector with the overall information to obtain the relevance comparison result, and use the relevance comparison result as the importance score of each representation vector;
[0095] The relevance is the proportion of overall information in the target statement corresponding to the vector.
[0096] It can be understood that the classifier used in this embodiment scores the degree of relevance between the target sentence corresponding to the representation vector and the overall information. The classifier can be one of the following three types: Simple classifier, Inter-sentence Transformer plus linear classifier, and Recurrent Neural Network plus linear classifier.
[0097] If a simple linear classifier is used for importance scoring, the Sigmoid function is used to obtain the predicted score, which is denoted as .
[0098]
[0099] Actually, the middle T i This represents the vector corresponding to the [CLS] flag of the i-th sentence in the top-level BART, which will serve as the sentence representation of the i-th sentence.
[0100] If an inter-sentence transformer combined with a linear classifier is used for importance scoring, the input vector will first be processed by the transformer.
[0101]
[0102]
[0103] Where h 0 It is obtained by position embedding the input sentence vector T. LN refers to the layer normalization operation, MHAtt is the multi-head attention mechanism of the transformer, and l refers to the depth of the stack.
[0104] The output layer is still a Sigmoid classifier:
[0105]
[0106] in This represents the sentence vector of the i-th sentence in the L-th layer of the transformer.
[0107] If a recurrent neural network combined with a linear classifier is used for importance scoring, then a Long Short-Term Memory (LSTM) network is used to capture sentence information:
[0108]
[0109] C i =σ(F i )⊙C i-1 +σ(I i )⊙tanh(G i-1 )
[0110] h i =σ(O t )⊙tanh(LN c (C t ))
[0111] Among them I i O i ,F i These correspond to the input gate, output gate, and forget gate of an LSTM, respectively. G i This refers to the hidden layer vector, C i It is a memory vector, h iIt is the output vector. Its final output layer is also a Sigmoid classifier:
[0112]
[0113] In this embodiment, step S50 includes:
[0114] S51: Filter out the target statements corresponding to the representation vectors whose importance scores are greater than or equal to the preset threshold, and obtain the filtered statements;
[0115] Understandably, the representation vectors can be sorted according to their importance scores in order to filter out the representation vectors with high importance scores, thus providing a basis for subsequently selecting target statements with high importance scores.
[0116] In this embodiment, a preset threshold, known as the "golden tag," is set. This means that the importance score of each representation vector is compared to the preset threshold, and representation vectors with importance scores greater than or equal to the threshold are selected. Their corresponding statements are then used as the filtering statements. For example, if the golden tag value is set to 60, and representation vector E1 has an importance score of 90, representation vector E2 has an importance score of 85, representation vector E3 has an importance score of 20, and representation vector E4 has an importance score of 30, then they are sorted according to their importance scores: E1 is first, E2 is second, E4 is third, and E3 is fourth. The target statement represented by representation vector E1 is sent1, that of representation vector E2 is sent2, that of representation vector E3 is sent3, and that of representation vector E4 is sent4. Therefore, representation vectors with importance scores greater than or equal to 60 include E1 and E2, and the filtering statements are sent1 and sent2.
[0117] S52: Generate summary text based on all filter statements;
[0118] Target statements with importance scores greater than or equal to a preset threshold are combined, and a summary text is generated based on the combined statements. For example, if the obtained filter statements are "sent one" and "sent two", they are combined to obtain "sent one, sent two". Then, a summary text is generated based on "sent one, sent two". It is worth mentioning that those skilled in the art know how to generate summary text based on provided statements, and the key to this application is to focus on the importance of each statement in the input text and to remove statements with low relevance based on importance when generating the summary text. However, there is no improvement to the specific summary generation process. Therefore, how to generate summary text based on filter statements will not be described here. In this embodiment, the summary text is generated only based on statements with high relevance to the overall information of the input text. By removing statements with high relevance to the overall information of the input text during the summary generation process, the accuracy of the generated summary text can be further improved.
[0119] This embodiment obtains target sentences by inserting a first preset flag and a second preset flag before and after each sentence, respectively. The target sentences are then input into the embedding layer of the pre-trained BART model. The embedding method is determined based on whether the target sentence is an odd-numbered target sentence, resulting in representation vectors that represent the target sentences. These representation vectors are then input into the summarization layer, where a classifier scores the importance of each representation vector within the input text. The representation vectors are then sorted from highest to lowest importance based on their scores. Target sentences corresponding to representation vectors with importance scores greater than or equal to a preset threshold are selected as filter sentences. Finally, a summary text is generated based on all the filter sentences. This embodiment, by converting the input text into representation vectors using the pre-trained BART model and then further scoring the representation vectors for importance, can determine the importance of each target sentence within the input text. By selecting all highly important target sentences based on their importance scores to generate the summary text, the accuracy of text summarization extraction is improved.
[0120] See Figure 4 , Figure 4 A schematic diagram of a text summarization device provided in this application is shown. The device specifically includes:
[0121] The text acquisition module is used to acquire the input text; the input text includes multiple statements.
[0122] The flag insertion module is used to insert multiple preset flags into multiple statements to obtain multiple target statements;
[0123] The vector generation module is used to input multiple target statements into the pre-trained model BART and obtain multiple representation vectors output by the pre-trained model BART.
[0124] The scoring module is used to obtain an importance score for each representation vector based on multiple representation vectors.
[0125] The summary generation module is used to generate summary text based on importance scores and target statements.
[0126] The technical solution of this embodiment, through the cooperation between various functional modules, achieves the following: by acquiring multiple sentences in the input text, inserting preset markers into the multiple sentences to obtain multiple target sentences, inputting the multiple target sentences into the BART model to obtain multiple representation vectors corresponding to the target sentences, scoring the importance of the multiple representation vectors, and then generating summary text based on the importance scores and the target sentences corresponding to the representation vectors, thereby improving the accuracy of obtaining text summaries.
[0127] Furthermore, embodiments of this application also propose a computer storage medium storing a text summarization generation program. When executed by a processor, the text summarization generation program implements the steps of the text summarization method described above. Therefore, it will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed to execute on a single computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.
[0128] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0129] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.
[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.
[0131] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A text summarization method, characterized in that, The method includes: Obtain the input text; the input text includes multiple statements; Multiple preset flags are inserted into multiple statements to obtain multiple target statements; Multiple target statements are input into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART; Based on the multiple representation vectors, an importance score is obtained for each of the representation vectors; Based on the importance score and the target statement, a summary text is generated; The step of inputting multiple target statements into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART includes: Multiple target statements are input into the pre-trained model BART to obtain multiple representation vectors and multiple context information vectors output by the pre-trained model BART; The step of obtaining an importance score for each of the multiple representation vectors includes: Based on the multiple representation vectors and the multiple context information vectors, the overall information of the input text is obtained; For each statement, the representation vector is compared with the overall information to obtain the correlation comparison result, and the correlation comparison result is used as the importance score of each representation vector.
2. The text summarization method according to claim 1, characterized in that, The step of inserting multiple preset flags into multiple statements to obtain multiple target statements includes: Insert the first preset flag before the first character of each of the statements; The second preset flag is inserted after the last character of each statement to obtain multiple target statements.
3. The text summarization method according to claim 1, characterized in that, The BART model includes an embedding layer, which takes multiple target sentences as input to the pre-trained BART model to obtain multiple representation vectors output by the pre-trained BART model, including: Multiple target statements are input into the embedding layer of the pre-trained model BART; The embedding layer performs word embedding operations on the target statement to obtain multiple representation vectors.
4. The text summarization method according to claim 3, characterized in that, The embedding layer performs word embedding operations on the target statement to obtain multiple representation vectors, including: Determine whether the target statement is an odd-numbered target statement; If so, the first segmentation embedding method is selected to perform word embedding on the target statement to obtain multiple representation vectors.
5. The text summarization method according to claim 4, characterized in that, After determining whether the target statement is an odd-numbered representation vector, the method further includes: If not, then the second segmented embedding method is selected to embed the target statement, resulting in multiple representation vectors.
6. The text summarization method according to claim 1, characterized in that, The step of generating summary text based on the importance score and the target statement includes: The target statements corresponding to the representation vectors whose importance scores are greater than or equal to a preset threshold are selected to obtain the filtered statements; The summary text is obtained based on all the filtering statements.
7. A text summarization generation device, characterized in that, The device includes: The text acquisition module is used to acquire input text; the input text includes multiple statements. The flag insertion module is used to insert multiple preset flags into multiple statements to obtain multiple target statements; The vector generation module is used to input multiple target statements into the pre-trained model BART to obtain multiple representation vectors output by the pre-trained model BART. The scoring module is used to obtain an importance score for each of the multiple representation vectors. The summary generation module is used to generate summary text based on the importance score and the target statement; The vector generation module is also used to input multiple target statements into the pre-trained model BART to obtain multiple representation vectors and multiple context information vectors output by the pre-trained model BART. The vector generation module is further configured to obtain an importance score for each of the multiple representation vectors, including: The vector generation module is further configured to obtain the overall information of the input text based on the multiple representation vectors and the multiple context information vectors; The vector generation module is further configured to compare the relevance of the representation vector with the overall information for each statement, obtain the relevance comparison result, and use the relevance comparison result as the importance score of each representation vector.
8. A text summarization generation device, characterized in that, include: A processor, a memory, and a text summarization generation program stored in the memory, wherein the text summarization generation program is executed by the processor to implement the steps of the text summarization generation method as claimed in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a text summarization program, which, when executed by a processor, implements the text summarization method as described in any one of claims 1 to 6.