A text summary generation method

Abstract

The present invention relates to the technical field of text summary generation, and more particularly to a method for generating a text summary, comprising the following steps: randomly combining at least two predetermined articles to generate a tree-shaped balanced binary tree, wherein a leaf node of the tree represents an article; and layer-by-layer fusing connected articles within the tree-shaped balanced binary tree to generate a target text summary that incorporates key information from the at least two articles. It is understood that fusing key information from multiple articles to generate a target text summary allows readers to quickly obtain key information from multiple articles, thereby improving reading efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of text summary generation, and in particular to a text summary generation method. Background Art

[0002] Professional research reports, such as industry development reports and securities analysis reports, are an important source of high-quality information. Due to their logical and specialized nature, research reports often contain a wealth of information. Furthermore, numerous specialized institutions and experts often cover the same event or subject, leading to the need to read numerous reports to understand the subject being analyzed. For example, financial investors need to read all relevant information on a specific target company to find the answers they need and make more accurate decisions. Faced with information overload, improving professional research report reading and information processing skills is essential for improving work efficiency.

[0003] Traditional intelligent research report reading systems focus on collecting and categorizing information, such as using keyword clustering algorithms to aggregate related research reports on the same company, making it easier for people to read and find information. However, a research report often contains tens of thousands of words, and simple information aggregation can no longer meet people's needs for quickly obtaining core information of concern. On the other hand, due to the limitations of the development and application of natural language processing technology, the technologies used in existing intelligent research report reading are mainly based on N-gram matching algorithms, such as keyword-based content retrieval and clustering algorithms. On this basis, people often need to read the entire research report to understand the content in order to find the answers to the core questions they are concerned about. This process of finding key information requires human effort and is time-consuming. Summary of the Invention

[0004] In order to solve the problem of being unable to quickly obtain the core information of a research report when reading a research report, the present invention provides a text summary generation method.

[0005] The solution to the technical problem of the present invention is to provide a text summary generation method, which specifically includes the following steps:

[0006] Randomly combine at least two preset articles to generate a tree-balanced binary tree, where the leaf nodes of the tree represent an article;

[0007] The connected articles in the tree-balanced binary tree are fused layer by layer to generate a target text summary that incorporates the key information of at least two articles.

[0008] Preferably, the tree-balanced binary tree includes nodes from the 1st to the Nth layer, the Nth layer nodes are leaf nodes, and N is a positive integer. The step of performing pairwise fusion of connected articles in the tree-balanced binary tree layer by layer includes:

[0009] Merge the N-th layer node articles into the N-1th layer node articles. Each node in the N-1th layer represents an N-1th layer node article. Each N-1th layer node article is generated by the fusion of the N-th layer node articles connected to the same N-1th layer node.

[0010] Layer-by-layer fusion is performed from layer N-1 to layer 1 to generate layer-1 node articles, where the layer-1 node articles are fused from layer-2 node articles connected to layer-1 nodes, and the layer-1 node articles are the target text summaries.

[0011] Preferably, the pairwise fusion of articles specifically includes the following steps:

[0012] Determine the two connected articles through the same upper-level node;

[0013] Use named entity recognition technology to identify the demand information in two connected articles and extract key sentences from them. Key sentences are sentences containing key information in the articles.

[0014] The extracted key sentences are screened based on the similarity between the key sentences, where the similarity between the key sentences is determined by the similarity calculated based on the anchor points and the cosine similarity calculated based on the semantics;

[0015] The filtered key sentences are spliced ​​together to obtain a set of key sentences after the two articles are fused.

[0016] Preferably, the step of identifying demand information in two connected articles by using named entity recognition technology and extracting key sentences therefrom specifically includes the following steps:

[0017] Identify the demand information in two connected articles through named entity recognition technology and fill it into the preset question template to generate multiple questions;

[0018] Using the Mencius-BERT model, answer fragments are extracted from each article for each question, and the obtained answer fragments are the key sentences.

[0019] Preferably, the step of screening the extracted key sentences based on the similarity between the key sentences specifically includes the following steps:

[0020] Match the elements in the two key sentence sets pairwise based on similarity to form a bipartite graph;

[0021] A greedy algorithm is used to select a set of key sentences that contains the most information but the least number of sentences.

[0022] Preferably, the similarity calculation formula between key sentences is ;

[0023] To calculate the cosine similarity based on semantics, we first calculate the vector representation of each key sentence through the Mencius pre-training model. , among which is the key sentence number, x is a positive integer, and then the cosine similarity between vectors is calculated. The specific formula is

[0024] ;

[0025] The similarity calculated based on the anchor point can be confirmed by digital comparison or character comparison. Indicates the weight coefficient of the similarity calculated based on the anchor point.

[0026] Preferably, the text summary generation method further comprises the following steps:

[0027] After completing the fusion of all articles, the Mencius-BERT pre-trained model is used to sort the key sentences in the final fused article to obtain the initial target text summary; the Mencius-T5 model is used to generate transition text between the key sentences of the initial target text summary to obtain the target text summary with transition text.

[0028] Preferably, the step of generating transition text between key sentences of the initial target text summary by using the Mencius-T5 model, thereby obtaining the target text summary with transition text, specifically comprises the following steps:

[0029] Set masks between the key sentences of the initial target text summary;

[0030] By predicting the content of the mask through the Mengzi-T5 model, the generated transition text can be obtained, thereby obtaining the target text summary with the transition text. The transition text is used to improve the logical relationship between adjacent key sentences in the initial target text summary.

[0031] Preferably, the text summary generation method further comprises the following steps:

[0032] Generate an overview and / or summary text for the target text summary with transition text, and the summary text is a paragraph or a sentence, so as to obtain a final target text summary.

[0033] Preferably, generating an overview and / or summary text for the target text summary with transition text, wherein the summary text is a paragraph or a sentence, thereby obtaining a final target text summary, specifically comprises the following steps:

[0034] Generate a topic word for each key sentence in sequence through the Mengzi-T5 model and label it;

[0035] Design a prompt question-and-answer template to inquire about the main content of each topic vocabulary in the target text summary with transition text. Then set the answer to the corresponding question as a mask and use the Mencius-T5 model to predict the masked content to obtain an overview and / or summary text.

[0036] The overview and / or summary text and the target text summary with transition text are combined to obtain a final target text summary.

[0037] Compared with the prior art, the text summary generation method of the present invention has the following advantages:

[0038] 1. The text summary generation method of the present invention specifically comprises the following steps: randomly combining at least two predetermined articles to generate a tree-balanced binary tree, where each leaf node of the tree represents an article; and layer-by-layer fusing connected articles within the tree-balanced binary tree to generate a target text summary that incorporates key information from at least two articles. As will be appreciated, fusing key information from multiple articles to generate a target text summary allows readers to quickly access key information from multiple articles, thereby improving reading efficiency.

[0039] 2. In the present invention, a tree-balanced binary tree includes nodes from the 1st to the Nth level, wherein the Nth level nodes are leaf nodes, and N is a positive integer. The method of layer-by-layer fusion of connected articles in the tree-balanced binary tree comprises: fusing the Nth level node articles into the N-1th level node articles, wherein each node in the N-1th level represents an N-1th level node article, and each N-1th level node article is generated by fusing the Nth level node articles connected to the same N-1th level node; and fusing layer-by-layer from the N-1th level to the 1st level to generate the 1st level node articles, wherein the 1st level node articles are generated by fusing the 2nd level node articles connected to the 1st level node, and the 1st level node articles are the target text summary. The summary body obtained by this method obtains the final key set by layer-by-layer fusion of two connected articles in the tree-balanced binary tree, thereby ensuring that the final key set contains all key information. In addition, some less important information can be removed during the layer-by-layer fusion, thereby reducing information redundancy while ensuring comprehensive information coverage, thereby improving the reading experience.

[0040] 3. In the present invention, the pairwise fusion of articles specifically includes the following steps: determining two connected articles through the same upper-level node; identifying the demand information in the two connected articles using named entity recognition technology, and extracting key sentences from them, where the key sentences are sentences containing key information in the articles; screening the extracted key sentences based on the similarity between the key sentences, where the similarity between the key sentences is determined by both similarity calculated based on anchor points and cosine similarity calculated based on semantics; and concatenating the filtered key sentences to obtain a set of key sentences after the two articles are fused. It can be understood that screening and concatenating the key sentences based on the similarity between the key sentences helps ensure that the obtained set of key sentences contains all important information.

[0041] 4. In this invention, named entity recognition technology is used to identify the demand information in two connected articles and fill it into a preset question template, thereby generating multiple questions. The Mencius-BERT model is used to extract answer fragments from each article for each question, and the obtained answer fragments are the key sentences. As can be understood, the preset question templates can control the generated questions, which helps users introduce their preferences for key topics or entities. On the other hand, the use of an extractive question-answering model facilitates the identification of more accurate key sentence collection fragments in a paragraph.

[0042] 5. In this invention, the elements of the two key sentence sets are matched pairwise based on similarity to form a bipartite graph. A greedy algorithm is then used to select the key sentence set with the most information but the fewest sentences as the key sentence set for the fused multiple articles. It is understood that the more concise the key sentence set after the fusion of multiple articles, the better. This ensures comprehensiveness while reducing the amount of reading, helping users quickly obtain information.

[0043] 6. In this invention, the similarity between key sentences is determined by combining similarity calculated based on anchor points and cosine similarity calculated based on semantics. This design helps to enhance the reliability of the similarity comparison results between key sentences, thereby improving the readability of the sorted text and enhancing the user's reading experience.

[0044] 7. In the present invention, transition text is also generated between key sentences in the initial target text summary using the Mencius-T5 model, thereby obtaining a target text summary with transition text. It is understandable that, after the previous steps, the initial target text summary already contains essentially all the necessary information for an article summary. However, this information is simply directly spliced ​​together, resulting in poor coherence and readability when reading. Generating transition text between key sentences in the initial target text summary can further improve the readability of the target text summary, thereby further enhancing the user's reading experience.

[0045] 8. In the present invention, the method of generating transition text is to set masks between the key sentences of the initial target text summary; the generated transition text can be obtained by predicting the content of the mask through the Mencius-T5 model, thereby obtaining the target text summary with transition text, and the transition text is used to improve the logical relationship between adjacent key sentences in the initial target text summary. After the previous algorithm steps, the initial target text summary has basically included all the information necessary for a research report summary. Therefore, the transition text is often a relatively short logical phrase, conjunction or subtitle, etc., and does not contain too much useful information text. Then, the problem of generating transition text can be simplified to the problem of generating logical phrases, conjunctions or subtitles between sentences. The Mencius-T5 model uses 300GB of massive data for pre-training, and stores rich prior knowledge, so it can achieve a higher generation effect under the setting of small sample fine-tuning.

[0046] 9. The text summary generation method of the present invention further includes the following steps: generating an overview and / or summary text for the target text summary with transition text, wherein the summary text is a paragraph or sentence, and combining the overview and / or summary text with the target text summary with transition text to obtain a final target text summary. It will be appreciated that generating the overview and / or summary text makes the final target text summary more convenient to read, helping users to more quickly obtain information and further improving the user's reading experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 4 is a flowchart of a text summary generation method provided by the first embodiment of the present invention.

[0049] Figure 2 4 is a flowchart of step S2 of the text summary generation method provided by the first embodiment of the present invention.

[0050] Figure 3 4 is a flowchart of step S22 of the text summary generation method provided by the first embodiment of the present invention.

[0051] Figure 4 This is a flowchart of step S221 of the text summary generation method provided by the first embodiment of the present invention.

[0052] Figure 54 is a flowchart of step S23 of the text summary generation method provided by the first embodiment of the present invention.

[0053] Figure 6 3 is a schematic diagram of coherence sorting in the text summary generation method provided by the first embodiment of the present invention.

[0054] Figure 7 4 is another flow chart of the text summary generation method provided by the first embodiment of the present invention.

[0055] Figure 8 4 is a flowchart of step S3 of the text summary generation method provided by the first embodiment of the present invention.

[0056] Figure 9 4 is a flowchart of step S4 of the text summary generation method provided by the first embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] The terms “vertical,” “horizontal,” “left,” “right,” “up,” “down,” “upper left,” “upper right,” “lower left,” “lower right,” and similar expressions used herein are for illustrative purposes only.

[0059] See also Figure 1 The first embodiment of the present invention provides a method for generating a text summary, which specifically includes the following steps:

[0060] Step S1: randomly combine at least two preset articles to generate a tree-balanced binary tree, where the leaf node of the tree represents an article;

[0061] Step S2: Layer by layer, the connected articles in the tree-balanced binary tree are fused pairwise to generate a target text summary that incorporates key information from at least two articles.

[0062] It is understandable that by integrating the key information of multiple articles to generate a target text summary, readers can quickly obtain the key information of multiple articles and improve their reading efficiency.

[0063] Specifically, in this embodiment, the article uses a research report.

[0064] Furthermore, the tree-balanced binary tree includes nodes from the 1st to the Nth layer, the Nth layer node is a leaf node, and N is a positive integer. The connected articles in the tree-balanced binary tree are merged layer by layer, specifically including the following steps:

[0065] Merge the N-th layer node articles into the N-1th layer node articles. Each node in the N-1th layer represents an N-1th layer node article. Each N-1th layer node article is generated by the fusion of the N-th layer node articles connected to the same N-1th layer node.

[0066] Layer-by-layer fusion from the N-1th layer to the 1st layer generates a layer-1 node article, wherein the layer-1 node article is generated by fusing the layer-2 node articles connected to the layer-1 nodes, and the layer-1 node article is the target text summary. Specifically, the layer-N-1 node article is fused into the layer-N-2 node article, each node of the layer-N-2 represents an N-2 node article, and each layer-N-2 node article is generated by fusing the layer-N-1 node articles connected to the same layer-N-2 node; ... the layer-3 node articles are fused into the layer-2 node article, each node of the layer-2 represents a layer-2 node article, and each layer-2 node article is generated by fusing the layer-3 node articles connected to the same layer-2 node; the layer-2 node articles are fused into the layer-1 node article, and the layer-1 node article is the target text summary.

[0067] The summary body obtained by this method obtains the final key set by fusing two connected articles in the tree-balanced binary tree layer by layer, which can ensure that the final key set contains all key information. It can also remove some less important information in the layer-by-layer fusion, reducing information redundancy while ensuring comprehensive information coverage, which is conducive to improving the reading experience.

[0068] Please combine Figure 1 and Figure 2 , the pairwise fusion of articles in step 2 specifically includes the following steps:

[0069] Step S21: Determine two connected articles through the same upper-level node;

[0070] Step S22: identifying the demand information in the two connected articles through named entity recognition technology, and extracting key sentences from them, where the key sentences are sentences containing key information in the articles;

[0071] Step S23: Screening the extracted key sentences based on the similarity between the key sentences, where the similarity between the key sentences is determined by the similarity calculated based on the anchor points and the cosine similarity calculated based on the semantics;

[0072] Step S24: splicing the selected key sentences to obtain a set of key sentences after the two articles are fused.

[0073] It can be understood that screening and concatenating key sentences based on the similarity between key sentences is conducive to ensuring that the obtained key sentence set can contain all important information.

[0074] Please combine Figures 1 to 3 , step S22 specifically includes the following steps:

[0075] Step S221: identifying the demand information in the two connected articles through named entity recognition technology, and filling it into a preset question template to generate multiple questions;

[0076] Step S222: Using the Mencius-BERT model, extract answer fragments from each article for each question, and the obtained answer fragments are key sentences.

[0077] It can be understood that the generated questions can be controlled by the preset question templates, which is conducive to the user introducing his or her preferences for key topics or entities; on the other hand, by using the extractive question-answering model, it is easier to find a more accurate set of key sentence fragments in the paragraph.

[0078] It should be noted that named entity recognition technology is an existing technology. For details, please see: Che, W., Feng, Y., Qin, L., & Liu, T. (2021). N-LTP: An Open-source Neural Language Technology Platform for Chinese. EMNLP.

[0079] Please combine Figures 1 to 4 Step S221 specifically includes the following steps:

[0080] Step S2211: Identify the demand information in each article using named entity recognition technology and fill it into a preset question template to generate multiple questions; that is, the demand information is determined based on the vacancies in the question template;

[0081] Step S2212: Use the Mencius-BERT model to extract answer fragments from each key sentence for each question, and the obtained answer fragments are the key sentences.

[0082] It can be understood that the generated questions can be controlled by the preset question templates, which is conducive to the user introducing his or her preferences for key topics or entities; on the other hand, by using the extractive question-answering model, it is easier to find a more accurate set of key sentence fragments in the paragraph.

[0083] Furthermore, a large number of questions can be prepared in advance in the preset question template, such as "What is the business model of [Company X]?", and then the entity names of several companies in the paragraph are identified through named entity recognition technology and filled into the question template to generate a question.

[0084] Furthermore, when extracting answer fragments, the several questions generated by each research report are recorded as Q1, Q2...QY in turn, where Y is a positive integer; for example, the two questions generated by research report A are recorded as A.Q1 and A.Q2, and the corresponding answers are recorded as A1 and A2, so as to facilitate subsequent matching.

[0085] Please refer to the picture Figure 2 and Figure 5 Step S23 specifically includes the following steps:

[0086] Step S231: Match the elements in the two key sentence sets pairwise based on similarity to form a bipartite graph;

[0087] Step S232: A set of key sentences containing the most information but the least number of sentences is selected through a greedy algorithm as the set of key sentences after merging multiple articles.

[0088] It is understandable that the more concise the set of key sentences after integrating multiple articles is, the better. While reducing the reading volume, the content must be comprehensive to help users quickly obtain information.

[0089] It should be noted that the matching algorithm of the bipartite graph can refer to the existing technology. For details, please refer to Wang Junli, Zhou Qing, and Yang Yaxing. A method for analyzing text semantic similarity [P]. Shanghai: CN106547739B, 2019-04-02. However, the present invention uses key sentences rather than topics as nodes. That is, if the similarity between two key sentences is greater than the set threshold, then the two sentences are connected by an edge. After iteration, a bipartite graph is finally generated. The nodes of this bipartite graph represent the sentence numbers in the research report, and the edges represent the similarity relationship between the two sentences. The present invention first calculates the out-degree of each node, that is, the number of edges with the node as the endpoint, and then uses it as a representation of the amount of information contained in the corresponding sentence. Based on this, a greedy algorithm is used to select a set of key sentences with the most information but the least number of sentences as the set of key sentences after the fusion of multiple research reports.

[0090] Furthermore, the similarity calculation formula between key sentences is ;

[0091] To calculate the cosine similarity based on semantics, we first calculate the vector representation of each key sentence through the Mencius pre-training model. , among which is the key sentence number, x is a positive integer, and then the cosine similarity between vectors is calculated. The specific formula is

[0092] ;

[0093] The similarity calculated based on the anchor point can be confirmed by digital comparison or character comparison. Indicates the weight coefficient of the similarity calculated based on the anchor point.

[0094] Understandably, this design helps to enhance the reliability of the similarity comparison results between key sentences, thereby enhancing the readability of the sorted text and thus improving the user's reading experience.

[0095] Furthermore, in the part of calculating similarity based on anchor points, for the method of number comparison, only when numbers are an important part of the article content can they be confirmed by number comparison. If two sentences contain the same numbers, then the two sentences are similar.

[0096] Furthermore, in the part of calculating similarity based on anchor points, for the character comparison method, the edit distance between two sentences, the longest common subsequence length, or N-Gram similarity can be used to calculate the similarity.

[0097] Furthermore, in step S2, the arrangement of key sentences is completed by the Mencius-BERT pre-trained model; the Mencius-BERT pre-trained model can calculate the probability of the key sentence being arranged in the i-th position and sort the key sentences based on the calculation result, where i is a positive integer.

[0098] The Mengzi-BERT pre-training model adopts the pre-training task of sentence order prediction, so it can adapt well to the downstream task of inter-sentence coherence sorting.

[0099] See also Figure 6 , give an example of the arrangement of key sentences, given three sentences , and then randomly shuffled and arranged into ,The input sample of constructing the Mencius encoder is shown in the figure, where "[CLS]" represents the input sample start symbol, " represents the start symbol of the sentence, "..." represents the text content of the corresponding sentence, and "[SEP]" represents the delimiter (which can be regarded as the end symbol of the input sample). Then use the Mencius encoder to encode it and get

[0100] The hidden vectors are ;set up

[0101] , use it as the key vector K and value vector V of the Mencius decoder, and As the query vector Q of Mengzi Decoder, it is decoded to ; Finally, we can get , represents the probability that the j-1th sentence is ranked in the i-th position. In this way, the coherence ranking of the three sentences is completed; it can be understood that the ranking of more than three sentences is similar.

[0102] For details on how to use the Mengzi-BERT pre-trained model to perform coherence sorting, see Lee, H., Hudson, DA, Lee, K., & Manning, CD (2020). SLM: Learning a DiscourseLanguage Representation with Sentence Unshuffling. EMNLP.

[0103] See also Figure 7 , the text summary generation method further includes the following steps:

[0104] Step S3: After completing the fusion of all articles, the Mencius-BERT pre-trained model is used to sort the key sentences in the final fused article to obtain the initial target text summary; the Mencius-T5 model is used to generate transition text between the key sentences of the initial target text summary to obtain the target text summary with transition text.

[0105] It is understandable that after the previous steps, the initial target text summary has basically contained all the information necessary for an article summary, but this information has only been directly spliced ​​together, so when reading, the coherence and readability are poor; generating transition text between the key sentences of the initial target text summary can further improve the readability of the target text summary, thereby further improving the user's reading experience.

[0106] Please combine Figure 7 and Figure 8 , step S3 specifically includes the following steps:

[0107] Step S31: setting masks between key sentences of the initial target text summary;

[0108] Step S32: The content of the mask is predicted by the Mencius-T5 model to obtain the generated transition text, thereby obtaining the target text summary with the transition text. The transition text is used to improve the logical relationship between adjacent key sentences in the initial target text summary.

[0109] It is understandable that after the previous algorithm steps, the initial target text summary has basically contained all the information necessary for a research report summary. Therefore, the transition text is often a relatively short logical phrase, conjunction or subtitle, etc., and does not contain too much useful information text. Then, the problem of generating transition text can be simplified to the problem of generating logical phrases, conjunctions or subtitles between sentences. The Mengzi-T5 model uses 300GB of massive data for pre-training and stores rich prior knowledge, so it can achieve higher generation effects under the setting of small sample fine-tuning.

[0110] For example, input three sorted key sentences, recorded as: 、 and , the input template is set as: Finally, the Mengzi-T5 model predicts the contents of <mask 1>, <mask 2>, and so on to generate the generated transition text. \s represents the input terminator. The target text summary with the transition text is denoted as Sum.

[0111] Furthermore, the Mengzi-T5 model needs to be fine-tuned using a fine-tuning dataset before use. The construction of the fine-tuning dataset is based on part-of-speech tagging and / or punctuation recognition and / or subtitles.

[0112] It is understandable that the problem of generating transitional text can be simplified to the problem of generating logical phrases, conjunctions or subtitles between sentences. Therefore, the targeted construction of fine-tuning datasets can effectively ensure the generation effect of the Mencius-T5 model.

[0113] Specifically, the application scenario of constructing a fine-tuning dataset based on part-of-speech tagging is as follows: there are usually many words with parts of speech such as prepositions or conjunctions in an article. Prepositions, conjunctions, etc. are replaced with masks. When fine-tuning the model, the replaced words are used as labels, and the text at the masked position is generated as the training task.

[0114] The application scenario for building a fine-tuning dataset based on punctuation recognition is as follows: Generally, research reports often contain paragraphs formatted like "Investment advice:...", where the text before the colon summarizes the subsequent content. Therefore, a mask is used to replace the text before the colon, and the fine-tuning task is also to generate text at the masked position.

[0115] The application scenario of constructing a fine-tuned dataset based on subtitles is as follows: there are often a large number of subtitles in research reports. The data is reconstructed as "subtitle text: paragraph text" and the data can be constructed using the same method as that used to construct a fine-tuned dataset based on punctuation recognition.

[0116] Please continue reading Figure 7 , the text summary generation method further includes the following steps:

[0117] Step S4: Generate an overview and / or summary text for the target text summary with transition text, and the summary text is a paragraph or a sentence, so as to obtain a final target text summary.

[0118] It is understandable that generating an overview and / or summary text can help users obtain information more quickly and further improve the user's reading experience.

[0119] Please combine Figure 7 and Figure 9 , step S4 specifically includes the following steps:

[0120] Step S41: Generate a topic word for each key sentence in sequence through the Mengzi-T5 model and mark it;

[0121] Step S42: Designing a prompt question-answering template for inquiring about the main content of each topic vocabulary in the target text summary with transition text, then setting the answer to the corresponding question as a mask, and predicting the masked content using the Mencius-T5 model to obtain an overview and / or summary text;

[0122] Step S43: combining the overview and / or summary text and the target text summary with transition text to obtain a final target text summary.

[0123] It is understandable that generating an overview and / or summary text can make the final target text summary more convenient to read, help users obtain information more quickly, and further improve the user's reading experience.

[0124] Specifically, taking the overview as an example, input three key sentences, which are recorded as follows: 、 and First, the Mencius-T5 model generates a topic vocabulary for each key sentence in sequence, which are recorded as: 、 and , then design the prompt template as: "Sum. Please ask: This paragraph is about, and What is the main content of the article? "Answer: <mask>". Finally, the Mencius-T5 model is used to predict the content of <mask> to obtain the summary text. This step requires fine-tuning the model using a small amount of data. The dataset sources include DOU ZY, LIU P, HAYASHI H, et al. 2021. GSum: AGeneral Framework for Guided NeuralAbstractive Summarization. abs / 2010.08014. and HE J, KRYSCINSKI W, MCCANN B, et al. 2020. CTRLsum: Towards GenericControllable TextSummarization. abs / 2012.04281.

[0125] Specifically, the dataset for topic generation is constructed as follows: First, for a paragraph and its corresponding subheading in a research report, the largest common subsequence is extracted as a candidate topic word, and the paragraph is used as a key sentence to construct a large amount of training data. A prompt template is designed as: "Key sentence: paragraph text; topic word: <mask>". Using this training data, the Mencius-T5 model is fine-tuned to generate the corresponding topic word at the "<mask>" position.

[0126] The answer-generated dataset was constructed as follows: This dataset is essentially the same as the topic-generated dataset. However, given that each paragraph in the topic-generated dataset can only correspond to one subheading and one topic word, we merged multiple paragraphs, their corresponding subheadings, and keywords. We then replaced the merged subheadings with mask tags to construct a large amount of pseudo data. Finally, we used this pseudo data to fine-tune the Mengzi-T5 model, generating the corresponding summary or summary text at the "<mask>" position.

[0127] Generally, the content of an overview and a summary is not much different. You can choose to generate an overview and / or a summary.

[0128] Specifically, in this embodiment, it is selected to generate only an overview.

[0129] Compared with the prior art, the text summary generation method of the present invention has the following advantages:

[0130] 1. The text summary generation method of the present invention specifically comprises the following steps: randomly combining at least two predetermined articles to generate a tree-balanced binary tree, where each leaf node of the tree represents an article; and layer-by-layer fusing connected articles within the tree-balanced binary tree to generate a target text summary that incorporates key information from at least two articles. As will be appreciated, fusing key information from multiple articles to generate a target text summary allows readers to quickly access key information from multiple articles, thereby improving reading efficiency.

[0131] 2. In the present invention, a tree-balanced binary tree includes nodes from the 1st to the Nth level, wherein the Nth level nodes are leaf nodes, and N is a positive integer. The method of layer-by-layer fusion of connected articles in the tree-balanced binary tree comprises: fusing the Nth level node articles into the N-1th level node articles, wherein each node in the N-1th level represents an N-1th level node article, and each N-1th level node article is generated by fusing the Nth level node articles connected to the same N-1th level node; and fusing layer-by-layer from the N-1th level to the 1st level to generate the 1st level node articles, wherein the 1st level node articles are generated by fusing the 2nd level node articles connected to the 1st level node, and the 1st level node articles are the target text summary. The summary body obtained by this method obtains the final key set by layer-by-layer fusion of two connected articles in the tree-balanced binary tree, thereby ensuring that the final key set contains all key information. In addition, some less important information can be removed during the layer-by-layer fusion, thereby reducing information redundancy while ensuring comprehensive information coverage, thereby improving the reading experience.

[0132] 3. In the present invention, the pairwise fusion of articles specifically includes the following steps: determining two connected articles through the same upper-level node; identifying the demand information in the two connected articles using named entity recognition technology, and extracting key sentences from them, where the key sentences are sentences containing key information in the articles; screening the extracted key sentences based on the similarity between the key sentences, where the similarity between the key sentences is determined by both similarity calculated based on anchor points and cosine similarity calculated based on semantics; and concatenating the filtered key sentences to obtain a set of key sentences after the two articles are fused. It can be understood that screening and concatenating the key sentences based on the similarity between the key sentences helps ensure that the obtained set of key sentences contains all important information.

[0133] 4. In this invention, named entity recognition technology is used to identify the demand information in two connected articles and fill it into a preset question template, thereby generating multiple questions. The Mencius-BERT model is used to extract answer fragments from each article for each question, and the obtained answer fragments are the key sentences. As can be understood, the preset question templates can control the generated questions, which helps users introduce their preferences for key topics or entities. On the other hand, the use of an extractive question-answering model facilitates the identification of more accurate key sentence collection fragments in a paragraph.

[0134] 5. In this invention, the elements of the two key sentence sets are matched pairwise based on similarity to form a bipartite graph. A greedy algorithm is then used to select the key sentence set with the most information but the fewest sentences as the key sentence set for the fused multiple articles. It is understood that the more concise the key sentence set after the fusion of multiple articles, the better. This ensures comprehensiveness while reducing the amount of reading, helping users quickly obtain information.

[0135] 6. In this invention, the similarity between key sentences is determined by combining similarity calculated based on anchor points and cosine similarity calculated based on semantics. This design helps to enhance the reliability of the similarity comparison results between key sentences, thereby improving the readability of the sorted text and enhancing the user's reading experience.

[0136] 7. In the present invention, transition text is also generated between key sentences in the initial target text summary using the Mencius-T5 model, thereby obtaining a target text summary with transition text. It is understandable that, after the previous steps, the initial target text summary already contains essentially all the necessary information for an article summary. However, this information is simply directly spliced ​​together, resulting in poor coherence and readability when reading. Generating transition text between key sentences in the initial target text summary can further improve the readability of the target text summary, thereby further enhancing the user's reading experience.

[0137] 8. In the present invention, the method of generating transition text is to set masks between the key sentences of the initial target text summary; the generated transition text can be obtained by predicting the content of the mask through the Mencius-T5 model, thereby obtaining the target text summary with transition text, and the transition text is used to improve the logical relationship between adjacent key sentences in the initial target text summary. After the previous algorithm steps, the initial target text summary has basically included all the information necessary for a research report summary. Therefore, the transition text is often a relatively short logical phrase, conjunction or subtitle, etc., and does not contain too much useful information text. Then, the problem of generating transition text can be simplified to the problem of generating logical phrases, conjunctions or subtitles between sentences. The Mencius-T5 model uses 300GB of massive data for pre-training, and stores rich prior knowledge, so it can achieve a higher generation effect under the setting of small sample fine-tuning.

[0138] 9. The text summary generation method of the present invention further includes the following steps: generating an overview and / or summary text for the target text summary with transition text, wherein the summary text is a paragraph or sentence, and combining the overview and / or summary text with the target text summary with transition text to obtain a final target text summary. It will be appreciated that generating the overview and / or summary text makes the final target text summary more convenient to read, helping users to more quickly obtain information and further improving the user's reading experience.

[0139] The above is a detailed introduction to a text summary generation method disclosed in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention. Any modifications, equivalent replacements and improvements made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A text summary generation method, characterized in that: The specific steps include: Randomly combine at least two preset articles to generate a tree-balanced binary tree, where the leaf nodes of the tree represent an article; The connected articles in the tree-balanced binary tree are fused layer by layer to generate a target text summary that incorporates the key information of at least two articles; The step of fusing two connected articles in the tree-balanced binary tree layer by layer specifically includes the following steps: Determine the two connected articles through the same upper-level node; Use named entity recognition technology to identify the demand information in two connected articles and extract key sentences from them. Key sentences are sentences containing key information in the articles. The extracted key sentences are screened based on the similarity between the key sentences, where the similarity between the key sentences is determined by the similarity calculated based on the anchor points and the cosine similarity calculated based on the semantics; The filtered key sentences are spliced ​​together to obtain a set of key sentences after the two articles are fused.

2. The text summary generation method according to claim 1, wherein: The tree-balanced binary tree includes nodes from the 1st to the Nth layer, the nodes in the Nth layer are leaf nodes, and N is a positive integer. The step of performing pairwise fusion of connected articles in the tree-balanced binary tree layer by layer includes: Merge the N-th layer node articles into the N-1th layer node articles. Each node in the N-1th layer represents an N-1th layer node article. Each N-1th layer node article is generated by the fusion of the N-th layer node articles connected to the same N-1th layer node. Layer-by-layer fusion is performed from layer N-1 to layer 1 to generate layer-1 node articles, where the layer-1 node articles are fused from layer-2 node articles connected to layer-1 nodes, and the layer-1 node articles are the target text summaries.

3. The text summary generation method according to claim 1, wherein: The method of identifying the demand information in two connected articles by using named entity recognition technology and extracting key sentences therefrom specifically includes the following steps: Identify the demand information in two connected articles through named entity recognition technology and fill it into the preset question template to generate multiple questions; Using the Mencius-BERT model, answer fragments are extracted from each article for each question, and the obtained answer fragments are the key sentences.

4. The text summary generation method according to claim 1, wherein: The method of screening the extracted key sentences based on the similarity between the key sentences specifically includes the following steps: Match the elements in the two key sentence sets pairwise based on similarity to form a bipartite graph; A greedy algorithm is used to select a set of key sentences that contains the most information but the least number of sentences.

5. The text summary generation method according to claim 1, wherein: The similarity calculation formula between key sentences is: ; To calculate the cosine similarity based on semantics, we first calculate the vector representation of each key sentence through the Mencius pre-training model. , among which is the key sentence number, x is a positive integer, and then the cosine similarity between vectors is calculated. The specific formula is ； The similarity calculated based on the anchor point can be confirmed by digital comparison or character comparison. Indicates the weight coefficient of the similarity calculated based on the anchor point.

6. The text summary generation method according to claim 1, wherein: The following steps are also included: After completing the fusion of all articles, the Mencius-BERT pre-trained model is used to sort the key sentences in the final fused article to obtain the initial target text summary; the Mencius-T5 model is used to generate transition text between the key sentences of the initial target text summary to obtain the target text summary with transition text.

7. The text summary generation method according to claim 6, wherein: The method of generating transition text between key sentences of the initial target text summary by using the Mencius-T5 model, thereby obtaining a target text summary with transition text, specifically includes the following steps: Set masks between the key sentences of the initial target text summary; By predicting the content of the mask through the Mengzi-T5 model, the generated transition text can be obtained, thereby obtaining the target text summary with the transition text. The transition text is used to improve the logical relationship between adjacent key sentences in the initial target text summary.

8. The text summary generation method according to claim 6, wherein: The following steps are also included: Generate an overview and / or summary text for the target text summary with transition text, and the summary text is a paragraph or a sentence, so as to obtain a final target text summary.

9. The text summary generation method according to claim 8, wherein: The method of generating an overview and / or summary text for the target text summary with transition text, wherein the summary text is a paragraph or a sentence, thereby obtaining a final target text summary, specifically includes the following steps: Generate a topic word for each key sentence in sequence through the Mengzi-T5 model and label it; Design a prompt question-and-answer template to inquire about the main content of each topic vocabulary in the target text summary with transition text. Then set the answer to the corresponding question as a mask and use the Mencius-T5 model to predict the masked content to obtain an overview and / or summary text. The overview and / or summary text and the target text summary with transition text are combined to obtain a final target text summary.

Images