Abstract generation method and device

Abstract

This application provides a summary generation method and device, wherein the summary generation method comprises: obtaining a target text to be processed; extracting a summary from the target text to obtain an initial summary of the target text; inputting the target text and the initial summary into a text generation model, summarizing the target text with the initial summary as a reference, and outputting a target summary of the target text. This method achieves the generation of a target summary for the target text using an extractive summary as guidance, thereby improving the accuracy of the generated target summary.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method and apparatus for generating a summary, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of the internet, the amount of information stored by individuals, businesses, and information retrieval services is increasing. Within this vast amount of stored information, searching for information based on user query criteria can yield a significant amount of relevant information. To help users quickly filter out information of interest from this information, automatic summarization technology can be used to enable users to quickly understand the retrieved information, thereby reducing the amount of information they need to read and improving their reading efficiency.

[0003] Existing automatic summarization technologies mainly include extractive summarization technology and generative summarization technology. The advantage of extractive summarization technology is that the extracted summary has a high relevance to the original text in terms of grammar and syntax. The disadvantage is that the extracted summary has poor coherence and flexibility, and errors in the extraction of summary content may occur. Generative summarization technology allows the summary to contain new words or phrases, so the generated summary is highly flexible, but it is easy to deviate from the original text. Therefore, there is an urgent need to provide a summary generation solution to solve the above problems. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a summary generation method and apparatus, a computing device, and a computer-readable storage medium to address the technical deficiencies in the prior art.

[0005] According to a first aspect of an embodiment of the present application, a method for generating a summary is provided, comprising:

[0006] Get the target text to be processed;

[0007] Extracting a summary of the target text to obtain an initial summary of the target text;

[0008] The target text and the initial summary are input into a text generation model, a summary of the target text is generated with reference to the initial summary, and a target summary of the target text is output.

[0009] According to a second aspect of an embodiment of the present application, a summary generation device is provided, including:

[0010] An acquisition module is configured to acquire a target text to be processed;

[0011] an extraction module configured to extract a summary of the target text to obtain an initial summary of the target text;

[0012] The generation module is configured to input the target text and the initial summary into a text generation model, generate a summary of the target text with reference to the initial summary, and output a target summary of the target text.

[0013] According to a third aspect of an embodiment of the present application, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the summary generation method when executing the computer instructions.

[0014] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer instructions. When the computer instructions are executed by a processor, the steps of the summary generation method are implemented.

[0015] According to a fifth aspect of the embodiments of the present application, a chip is provided, which stores computer instructions. When the computer instructions are executed by the chip, the steps of the summary generation method are implemented.

[0016] In an embodiment of the present application, a target text to be processed is obtained, a summary is extracted from the target text, an initial summary of the target text is obtained, the target text and the initial summary are then input into a text generation model, a summary of the target text is generated using the initial summary as a reference, and a target summary of the target text is output. This allows the extracted summary content to be used as the control text to control the text generation model for summary generation, thereby not only leveraging the strong relevance of extractive summaries but also ensuring that the generated summary is fluent and logical. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a structural block diagram of a computing device provided by an embodiment of the present application;

[0018] Figure 2 This is a flowchart of a summary generation method provided in one embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of determining a target text in a summary generation method provided in one embodiment of the present application;

[0020] Figure 4 This is a schematic diagram of a summary generation method provided by an embodiment of the present application in a multi-text scenario;

[0021] Figure 5 This is a processing flow chart of a summary generation method applied in a retrieval scenario provided by an embodiment of the present application;

[0022] Figure 6 Schematic diagram of the structure of a summary generation device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0024] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0025] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, first may also be referred to as second, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determining."

[0026] First, the terms involved in one or more embodiments of the present invention are explained.

[0027] Extractive summarization: Generate a summary by extracting and concatenating key sentences from the source text.

[0028] Generative summarization: This method summarizes the source text by organizing the language based on the key elements expressed in the text. Generative summarization is based on Natural Language Generation (NLG). The model generates sentences independently. Generative summarization offers greater flexibility than extractive summarization, allowing the model to generate new words or phrases with a certain probability. It is typically implemented using a sequence-to-sequence architecture. This type of summarization is the method implemented in this article.

[0029] TF-IDF (term frequency–inverse document frequency) is a commonly used weighting technique for information retrieval and data mining. It can also be understood as a statistical method used to evaluate the importance of a word to a document set or a document in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but at the same time decreases in inverse proportion to the frequency of its appearance in the corpus. Specifically, term frequency (TF) can be obtained by dividing the number of times the word appears in a text by the total number of words contained in the text; inverse document frequency (IDF) can be obtained by dividing the total number of texts by the number of texts containing the word, and then taking the logarithm of the quotient; the value of TF-IDF is the product of the TF and IDF obtained above.

[0030] List-wise: This is a strategy or method in the ranking learning algorithm. This method regards the entire item sequence (the item sequence in this scheme is a list of multiple texts) as a sample and is implemented by directly optimizing the information retrieval evaluation method and defining a loss function.

[0031] TextRank: This algorithm is a graph-based ranking algorithm for text. It divides the text into several components (sentences), constructs a node connection graph, uses the similarity between sentences as the edge weight, calculates the TextRank value of the sentence through a loop iteration, and finally extracts the high-ranking sentences to form a text summary.

[0032] Normalized exponential function (softmax function): It can "compress" a k-dimensional vector containing any real number into another K-dimensional real number vector, so that the range of each element is between (0, 1) and the sum of all elements is 1. It is often used to solve multi-classification problems.

[0033] Feedforward neural network (FNN): This is the simplest type of neural network. Neurons are arranged in layers, and each neuron is connected only to neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer, with no feedback between layers.

[0034] In this application, a summary generation method and apparatus, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0035] Figure 11 shows a block diagram of a computing device 100 according to an embodiment of the present application. Components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and a database 150 is used to store data.

[0036] The computing device 100 also includes an access device 140 that enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0037] In one embodiment of the present application, the above components of the computing device 100 and Figure 1 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 1 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0038] The computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 100 may also be a mobile or stationary server.

[0039] The processor 120 may execute Figure 2 Steps in the summary generation method shown. Figure 2 A flowchart of a summary generation method according to an embodiment of the present application is shown, which specifically includes the following steps:

[0040] Step 202: Obtain the target text to be processed.

[0041] The target text specifically refers to an article or document for which a summary is to be generated. It should be noted that the target text can be a text of any content, such as a news-type text, a subject-type text, an entertainment-type text, etc., or a text of any format, such as a txt-type text, a doc-type text, or a wps-type text. In addition, the target text to be processed can also be one text or multiple texts. In the embodiment of the present application, by generating a summary for the target text, the user can quickly understand the main content of the target text by reading the summary.

[0042] In practical applications, there are various ways to obtain the target text to be processed. For example, in a retrieval scenario, when a user searches for information in a text library that stores a large amount of text, dozens or even hundreds of relevant texts may be retrieved. However, most of these relevant texts may not be highly relevant to the retrieval information. If summaries are generated for all of these texts, it will result in a waste of computing resources. In order to only generate summaries for highly relevant texts, an optional implementation method provided in this embodiment of the application is specifically implemented as follows: Steps 222 to 26:

[0043] Step 2022: Receive the information to be retrieved submitted by the user.

[0044] Specifically, the processor 120 receives information to be retrieved input by the user on the computing device 100. The information to be retrieved may be keywords, query conditions, query questions, etc. For example, the information to be retrieved may be information that the user wants to know, such as "recipes", "writing methods", or "how to prepare for exams".

[0045] Step 2024: Perform a search based on the information to be retrieved to obtain at least one initial text corresponding to the information to be retrieved.

[0046] On the basis of receiving the information to be retrieved submitted by the user, a search can be performed in a text collection such as a document library or folder based on the information to be retrieved. In specific implementation, the information to be retrieved can be retrieved by literal retrieval and / or deep semantic matching, among which literal retrieval can use an open source search engine to retrieve the information to be retrieved; or the information to be retrieved can be retrieved based on a question index established in a preset document library. Deep semantic matching can be understood as calculating the similarity between the information to be retrieved and each question in the preset document library to obtain the most similar questions, and then obtaining the text corresponding to these questions as the initial text. In actual applications, in different retrieval scenarios, different retrieval strategies or solutions can be selected according to different needs, and there is no limitation here.

[0047] Step 2026: Filter the target text from at least one initial text.

[0048] Specifically, since the number of initial texts retrieved based on the information to be retrieved may be large, in order to reduce the user's reading volume and improve the accuracy of the user's acquisition of relevant information of the information to be retrieved, it is necessary to further filter the target text from these initial texts and then generate a summary of the target text.

[0049] Since the number of the initial texts may be one or more, the number of the target texts filtered out from the initial texts is at least one.

[0050] In specific implementation, considering that there are many ways to filter the target text from at least one initial text, in order to avoid the target text filtered from at least one initial text lacking important information related to the information to be retrieved, it is necessary to compare the relevance of each initial text with the information to be retrieved, and then determine the initial text with high relevance to the information to be retrieved from the initial text as the target text. In this embodiment of the application, an optional implementation method is provided using the importance score as a measure of relevance, which is specifically implemented in the following way:

[0051] Calculate the importance score of each initial text;

[0052] Based on the importance score, a target text is selected from at least one initial text.

[0053] Specifically, the importance score can be understood as the relevance score between the initial text and the information to be retrieved, wherein the higher the relevance between the initial text and the information to be retrieved, the higher the importance score.

[0054] In specific implementation, the importance score of the initial text can be calculated based on the similarity or matching degree between the initial text and the information to be retrieved. The higher the similarity or matching degree between the initial text and the information to be retrieved, the higher the relevance, and accordingly, the greater the importance score of the initial text. Based on the importance score of the initial text, a preset number of target texts are selected from the initial text in descending order of importance score; or the importance score is compared with a preset score threshold, and target texts with an importance score greater than the preset score threshold are screened out from the initial text, so as to generate a summary of the initial text (i.e., the target text) that has a high relevance to the information to be retrieved.

[0055] Furthermore, since the relevance of the initial text to the information to be retrieved determines the importance score, in order to more accurately measure the relevance of the initial text to the information to be retrieved, the optional implementation method provided in the embodiment of the present application specifically uses the following two methods to calculate the importance score of the initial text:

[0056] (1) Segment the information to be retrieved to obtain target word units contained in the information to be retrieved; count the target word units contained in each initial text to obtain the statistical score of each target word unit in each initial text, and determine the statistical score of each initial text based on the statistical score of each target word unit, and use the statistical score of each initial text as the importance score of each initial text.

[0057] Among them, the target word unit refers to at least one word unit obtained after the word segmentation processing of the search information. On the basis of obtaining the target word unit, the number of occurrences of the target word unit contained in each initial text is counted, and the statistical score of each target word unit is determined based on the statistical results. In practical applications, the statistical score of each target word unit can be calculated using the TF-IDF statistical algorithm, or the corresponding relationship between the number of occurrences of the target word unit and the statistical score can be predefined. Then, based on the number of occurrences of the target word unit obtained by statistics, the statistical score of each target word unit can be determined; in addition, the statistical score of the target word unit can also be determined by other methods, which are not limited here.

[0058] Furthermore, based on the statistical score of each target word unit, at least one statistical score of at least one target word unit in each initial text can be calculated (e.g., summation, average, weighted average, etc.) to determine the statistical score of each initial text. Specifically, the larger the statistical score of the initial text, the greater the relevance of the initial text to the information to be retrieved, and the more important the initial text. The statistical score of each initial text is used as the importance score of each initial text.

[0059] For example, if a user submits search information for "writing methods," a search is performed on a large text library containing a large number of texts based on "writing methods." Three initial texts related to "writing methods" are obtained: initial text d1, initial text d2, and initial text d3. Word segmentation is performed on the search information for "writing methods," resulting in two target word units: "writing" and "method."

[0060] The number of times the target word unit "writing" appears in the initial text d1 is 5, and the number of times the target word unit "method" appears in the initial text d1 is 7. Based on the statistical results and the TF-IDF statistical algorithm, the statistical score of the target word unit "writing" in the initial text d1 is determined to be S. d1 _1, and determine the statistical score of the target word unit "method" in the initial text d1 as S d1_2; Similarly, the number of times the target word unit "writing" appears in the initial text d2 is 6 times, and the number of times the target word unit "method" appears in the initial text d2 is 5 times. Based on the statistical results and the TF-IDF statistical algorithm, the statistical score of the target word unit "writing" in the initial text d2 is determined to be S d2 _1, and determine the statistical score of the target word unit "method" in the initial text d2 as S d2 _2; Count the number of times the target word unit "writing" appears in the initial text d3, which is 5 times, and the number of times the target word unit "method" appears in the initial text d3, which is 3 times. Based on the statistical results, determine the statistical score of the target word unit "writing" in the initial text d3 as S d3 _1, and determine the statistical score of the target word unit "method" in the initial text d3 to be S d3 _2.

[0061] In an optional embodiment, the statistical score of the initial text is calculated by summing up. The statistical score S of the target word unit "writing" in the initial text d1 is d1 _1, and the statistical score S of the target word unit "method" in the initial text d1 d1 _2 Perform sum calculation to obtain the statistical score of the initial text d1 as S d1 , and the statistical score S d1 As the importance score of the initial text d1; similarly, the statistical score S of the target word unit "writing" in the initial text d2 d2 _1, and the statistical score S of the target word unit "method" in the initial text d2 d2 _2 performs sum calculation and obtains the statistical score of the initial text d2 as S d2 , and the statistical score S d2 As the importance score of the initial text d2; the statistical score S of the target word unit "writing" in the initial text d3 d3 _1, and the statistical score S of the target word unit "method" in the initial text d3 d3 _2 Perform sum calculation to obtain the statistical score of the initial text d3 as S d3 , and the statistical score S d3 As the importance score of the initial text d3. And according to the importance scores of the initial text d1, the initial text d2 and the initial text d3 from high to low, two target texts are selected from the three initial texts, and these two target texts are target text d1 and target text d2.

[0062] (2) Inputting the information to be retrieved and at least one initial text into the ranking model, obtaining the ranking score of each initial text output by the ranking model, and using the ranking score as the importance score of each initial text.

[0063] Specifically, the ranking model can be a language model, such as the Bert language model. The Bert language model can include list-wise ranking algorithms, point-wise ranking algorithms, or pair-wise ranking algorithms, without limitation. Furthermore, the ranking model outputs a ranking score for each initial text, where a higher ranking score indicates a greater relevance of the initial text to the information to be retrieved, and thus indicates a more important initial text.

[0064] Still taking the information to be retrieved as "writing method" and the initial texts as: initial text d1, initial text d2 and initial text d3 as an example, the information to be retrieved "writing method" and the three initial texts are input into the ranking model, and the ranking score of the initial text d1 output by the ranking model is 0.39, the ranking score of the initial text d2 is 0.5, and the ranking score of the initial text d3 is 0.11. The output ranking scores are used as importance scores, and two target texts are selected from the three initial texts according to the order of the importance scores of initial text d1, initial text d2 and initial text d3 from high to low: target text d1 and target text d2.

[0065] In summary, in the retrieval scenario, the texts in the text library are screened according to the user's information to be retrieved to obtain at least one initial text, and the importance score of each initial text is calculated through a statistical algorithm or a sorting algorithm, so as to further determine the target text with a higher relevance to the information to be retrieved, thereby improving the accuracy of determining the target text.

[0066] In addition to the two specific implementation methods for calculating the importance score of each initial text described above, in order to avoid inaccurate calculation results that may be calculated by any of the above calculation methods, the above two calculation methods can be combined to improve the accuracy of calculating the importance score. An optional implementation method provided in the embodiment of the present application is specifically implemented as follows:

[0067] Segment the information to be retrieved to obtain target word units contained in the information to be retrieved;

[0068] Counting the target word units contained in each initial text to obtain a statistical score for each target word unit in each initial text;

[0069] Determine the statistical score of each initial text based on the statistical score of each target word unit;

[0070] Input the information to be retrieved and at least one initial text into the ranking model, and obtain the ranking score of each initial text output by the ranking model;

[0071] The statistical score and ranking score of each initial text are weighted averaged to obtain the importance score of each initial text.

[0072] Specifically, the specific implementation method for determining the statistical score of each initial text in the embodiment of the present application is similar to the specific implementation method for determining the statistical score of each initial text described above. Reference can be made to the specific implementation method for determining the statistical score of each initial text described above, and no further details will be given here. Furthermore, the specific implementation method for obtaining the ranking score of each initial text output by the ranking model in the embodiment of the present application is similar to the specific implementation method for obtaining the ranking score of each initial text output by the ranking model described above. Reference can be made to the specific implementation method for obtaining the ranking score of each initial text output by the ranking model described above, and no further details will be given here.

[0073] Furthermore, on the basis of determining the statistical score of each initial text and obtaining the ranking score of each initial text output by the ranking model, the statistical score and ranking score of each initial text are weighted averaged to obtain the importance score of each initial text. In specific implementation, the weight of the statistical score and the weight of the ranking score can be pre-set according to experience or actual needs, such as the weight of the statistical score is 0.4, and the weight of the ranking score is 0.6; or the weight of the statistical score is 0.58, and the weight of the ranking score is 0.42, etc., and there is no restriction here.

[0074] Specifically, such as Figure 3 The schematic diagram of determining the target text in a summary generation method provided by an embodiment of the present application is shown. By matching the question information (information to be retrieved) with keywords in the document library, the documents in the document library are roughly sorted, and m documents (initial texts) related to the question information are screened out. The statistical score of each document is calculated separately by the statistical method TF-IDF, and the ranking score of each document is calculated separately by the language model method. The statistical score and ranking score of each document are weighted averaged to calculate the importance score, and then n documents (target texts) are determined from the m documents according to the order of the importance score from large to small.

[0075] In summary, by combining statistical algorithms and / or ranking algorithms to calculate the importance score of each initial text, the importance score can more accurately reflect the importance of each initial text relative to the information to be retrieved, and then the target text can be screened from the initial text based on the importance score, further improving the accuracy of determining the target text.

[0076] In specific implementation, in addition to the specific implementation method of obtaining the target text, the target text can also be obtained according to a preset text identifier, or according to a storage location, etc. In specific implementation, the corresponding acquisition method can be selected according to the actual needs of the user, and is not limited here.

[0077] Step 204: extracting a summary of the target text to obtain an initial summary of the target text.

[0078] On the basis of acquiring the target text as described above, the target text is subjected to summary extraction. An extractive summary algorithm can be used to extract a summary from the target text to obtain an extractive summary (i.e., an initial summary) of each target text. Specifically, the extractive summary algorithm includes: TextRank (text ranking) algorithm, SIF (smooth inverse frequency) algorithm, summary extraction algorithm based on sentence position, etc., which are not limited here. Among them, the SIF algorithm obtains the sentence vector of each sentence in the target text, and then calculates the similarity between each sentence vector and the text vector of the entire target text through formulas such as cosine similarity or Euclidean distance, and obtains the K sentences with the highest similarity as the summary extracted for the target text. The summary extraction algorithm based on sentence position can be to extract the first K sentences in the target text, or to extract the last K sentences in the target text, etc., which are not limited here.

[0079] In practical applications, the number of abstract sentences to be extracted can be pre-set (e.g., 5, 6, etc.), and then the pre-set number of sentences can be extracted from the target text as the initial abstract. Alternatively, the ratio of the number of sentences in the extracted abstract to the number of sentences in the target text can be pre-set (e.g., 10%-25% of the sentences in each target text can be extracted as the extractive abstract), and then the pre-set ratio of sentences can be extracted from the target text as the initial abstract. The specific number of abstract sentences to be extracted can be set based on actual needs or experience and is not limited here.

[0080] Continuing with the above example, the TextRank algorithm is used to extract summaries of the target text d1 and the target text d2, respectively. The extracted summary of the target text d1 is obtained as initial summary 1, and the extracted summary of the target text d2 is obtained as initial summary 2.

[0081] Step 206: Input the target text and the initial summary into a text generation model, generate a summary of the target text with reference to the initial summary, and output a target summary of the target text.

[0082] Based on the initial summary of the target text obtained above, we consider that the initial summary extracted from the target text may have problems such as incoherent sentences and poor flexibility. To further improve the accuracy of the generated summary, we can use the extracted summary (i.e., the initial summary) as a reference. This ensures that the target summary generated by the text generation model is not only highly relevant to the original text in terms of grammar and syntax, but also logical and can highly summarize the target text.

[0083] Specifically, a text generation model refers to a text-based, self-monitored text generation model. It can also be understood as a controllable text generation model based on text variables, such as the CoCon model. The initial summary serves as the text variable of the text generation model (i.e., the control text of the text generation model). The initial summary is used as a reference, with the goal of ensuring that the generated target summary contains the core content of the extractive summary (the initial summary).

[0084] In a specific implementation, a target summary is generated by a text generation model, which includes an embedding layer, an encoding layer, and a decoding layer. An optional implementation method provided in an embodiment of the present application is specifically implemented through the following steps 2062 to 2066:

[0085] Step 2062: Input the target text and the corresponding initial summary into the embedding layer to obtain the text vector corresponding to the target text and the control vector corresponding to the initial summary.

[0086] The embedding layer is a hierarchical structure in a text generation model that converts natural language into computer-readable vectors. A text vector is a vector representation of the target text, obtained by embedding the characters in the target text. A control vector is a vector representation of the characters in the initial summary, obtained by embedding them.

[0087] Step 2064: Input the text vector and the control vector into the encoding layer to obtain the encoding vector.

[0088] The encoding layer is also a hierarchical structure in the text generation model, which is used to extract semantic features from the vector for encoding. Correspondingly, the encoding vector specifically refers to the vector representation obtained after encoding the text vector and the control vector.

[0089] In a specific implementation, the encoding layer includes a self-attention layer and a feedforward neural network layer. An optional implementation provided in the embodiment of the present application inputs the text vector and the control vector into the encoding layer to obtain the encoding vector output by the encoding layer, which is specifically implemented as follows:

[0090] Input the text vector and control vector into the self-attention layer to obtain the self-attention vector;

[0091] The self-attention vector is input into the feedforward neural network layer for conversion processing to obtain the encoding vector.

[0092] Among them, the self-attention layer is a hierarchical structure used to perform self-attention processing on the text vector and the control vector in the text generation model. Based on the text vector and the control vector output by the embedding layer, the text vector and the control vector are processed by the self-attention layer to obtain the self-attention vector. The self-attention vector is then input into the feedforward neural network layer for conversion processing, and the conversion result is used as the encoding result of the encoding layer, i.e., the encoding vector. Specifically, the feedforward neural network layer contains the parameters that the model needs to learn, and the self-attention vector is converted (i.e., function fitting) based on the parameters to obtain the conversion result: In practical applications, a residual network can also be added after the feedforward neural network, and the residual network is used to perform residual calculation on the conversion result, and the calculation result is used as the encoding result of the encoding layer, i.e., the encoding vector; the residual network can solve the problems of gradient explosion and network degradation.

[0093] Furthermore, in order to enable the text generation model to generate a target summary of the target text with the initial summary as a reference, self-attention processing can be performed through the self-attention layer of the text generation model to enable the text vector and the control vector to interact. An optional implementation method provided in the embodiment of the present application is specifically implemented in the following manner:

[0094] splicing the key vector corresponding to the control vector and the key vector corresponding to the text vector to obtain a spliced key vector;

[0095] splicing the value vector corresponding to the control vector and the value vector corresponding to the text encoding vector to obtain a spliced value vector;

[0096] Self-attention processing is performed based on the splicing key vector, the splicing value vector, and the question vector corresponding to the text vector to obtain a self-attention vector.

[0097] Among them, the key vector corresponding to the control vector is calculated by calculating the control vector and the weight matrix W K Similarly, the key vector corresponding to the text vector is obtained by calculating the text vector and the weight matrix W K Based on the key vector corresponding to the control vector and the key vector corresponding to the text encoding vector, the key vectors of the two are concatenated to obtain a concatenated key vector.

[0098] Correspondingly, the value vector corresponding to the control vector is calculated by the control vector and the weight matrix W V Similarly, the value vector corresponding to the text vector is obtained by calculating the text vector and the weight matrix W VOn the basis of obtaining the value vector corresponding to the control vector and the value vector corresponding to the text vector, the value vectors of the two are spliced to obtain a spliced value vector.

[0099] Furthermore, the question vector corresponding to the text vector is calculated by calculating the text vector and the weight matrix W Q The question vector corresponding to the text vector is multiplied by the transpose of the splicing key vector to obtain the attention score corresponding to the question vector. The attention score is then normalized by the normalized exponential function and multiplied by the splicing value vector to obtain the self-attention vector. The normalized attention score can be understood as the attention weight. The larger the weight value, the greater the degree to which the corresponding splicing value vector participates in the calculation, and the more similar the self-attention vector is to the splicing value vector. The purpose of the self-attention layer is to capture the internal correlation between these vectors, so that the text generation model can make more accurate judgments without incurring greater computational and storage overhead for the text generation model.

[0100] Step 2066: Input the encoding vector into the decoding layer for decoding processing, and output the target summary based on the decoding processing result.

[0101] Specifically, the decoding layer refers to the hierarchical structure used to convert the encoding vector into the decoding vector in the text generation model. In actual applications, after the decoder outputs the decoding vector, the decoding vector is input to the output layer to obtain the target summary output by the output layer.

[0102] Based on the above-mentioned extractive summary of the target text d1 as the initial summary 1, the target text d1 and the initial summary 1 are input into the embedding layer of the CoCon model to obtain the text vector H corresponding to the target text d1 and the control vector H corresponding to the initial summary 1. c In order to realize the text vector H and the control vector H c The interaction between them, the weight matrix W corresponding to the text vector H is set in the CoCon model. K 、W v and W Q , and the control vector H c The corresponding weight matrix W Kc and W Vc . The text vector H is combined with the corresponding weight matrix W K Multiply to obtain the key vector K=W corresponding to the text encoding vector H K *H, the control vector H c And the corresponding weight matrix W Kc Multiply to obtain the control vector H c The corresponding key vector K c =WKc *H c ; Then control vector H c The corresponding K c The key vector K corresponding to the text vector H is spliced to obtain the spliced key vector K'=[K c , K];

[0103] And the text vector H and the corresponding weight matrix W v Multiply to obtain the value vector V=W corresponding to the text vector H V *H, the control vector H c And the corresponding weight matrix W Vc Multiply to obtain the control vector H c The corresponding value vector V c =W Vc *H c Then the control vector H c The corresponding value vector V c and the value vector V corresponding to the text vector H to obtain the spliced value vector V'=[V c , V];

[0104] Text vector H and weight matrix W Q Multiply to obtain the question vector Q=W corresponding to the text vector H Q *H. Then the question vector Q, the splicing key vector K' and the splicing value vector V' are processed by self-attention to obtain the self-attention vector A. Specifically, the self-attention layer calculates the question vector Q, the splicing key vector K' and the splicing value vector V' as follows: A = softmax(QK' T )V', and obtain the self-attention vector A. Then input the self-attention vector A into the feedforward neural network layer for conversion processing to obtain the encoding vector, input the encoding vector into the decoding layer for decoding processing, obtain the decoding vector corresponding to the encoding vector output by the decoding layer, and input the decoding vector into the output layer to obtain the target summary 1 of the target text d1 output by the output layer.

[0105] In summary, the interaction between the target text and the initial summary is achieved through the self-attention mechanism in the text generation model, so that the text generation model generates a summary of the target text with the initial summary as a reference, and the generated target summary has the main content of the initial summary.

[0106] In addition, considering that different target texts have different levels of importance relative to the information to be retrieved, target texts with different levels of importance will generate summaries of the same length. In order to avoid generating summaries of the same length for different target texts, the detail of the summary generated for each text can be controlled based on the importance score of the text and the length of the text, so that the more important the text is relative to the information to be retrieved, the longer the summary generated, and the longer the text is, the longer the summary generated. An optional implementation method provided in the embodiment of the present application generates a target summary of the target text, which is specifically implemented in the following way:

[0107] Input the target text and the initial summary into the text generation model, generate a summary of the target text based on the length factor and the initial summary in the text generation model, and output the i-th character;

[0108] Determine whether the i-th character is the end mark;

[0109] If not, determine the text length of the text to which the i-th character is added, and update the length factor based on the text length of the target text, the text length of the text, and the importance score of the target text to obtain a target length factor; use the target length factor as the length factor, and return to the above-mentioned summary generation process for the target text with reference to the length factor in the text generation model and the initial summary, and output the i+1-th character;

[0110] If so, add the i-th character to the text and extract the target summary from the text.

[0111] In practical applications, in order to control the length of the summary generated by the text generation model, a length factor setting is added based on the text generation model. The length factor can be understood as a variable parameter of the text generation model. This parameter will change dynamically with the length of the text output by the text generation model until the text generation model outputs an end tag, where the end tag can correspond to the start tag. Specifically, the end tag can be <s> Corresponding< / s> ,or <eos>etc., no restriction is made here.

[0112] In practical applications, the length factor can be dynamically changed according to the length of the text composed of characters generated by the text generation model. The law of this dynamic change can be determined by a preset formula (such as α = score * L d / L, where a is the length factor, L is the text length of the above text, L d The default formula can be set based on actual needs and experience and is not limited here.

[0113] As the text generation model outputs characters one by one, i starts at 1 and is a positive integer. When the text generation model outputs the first character, it is called character 1; when the text generation model outputs the second character, it is called character 2; and so on. Each time a character is output, the length factor is iteratively updated based on the length of the text composed of the characters generated by the text generation model, the length of the target text, and the importance score of the target text. Furthermore, the generated length factor influences the probability of the text generation model outputting an end tag.

[0114] Specifically, by multiplying the length factor by the probability of the text generation model outputting other characters (other characters except the end mark), the output probability of the text generation model outputting other characters can be obtained. The smaller the length factor, the smaller the output probability of the text generation model outputting other characters will be. Correspondingly, the greater the output probability of the text generation model outputting the end mark.

[0115] It should be noted that determining the text length of the text with the i-th character added is specifically to add the i-th character to the text and then determine the text length of the text after adding the i-th character, wherein the text length refers to the length of the characters in the text excluding the model-generated tags (start tags, end tags, etc.).

[0116] When the text generation model outputs an end tag, indicating that the text generation model has ended, the target summary can be extracted from the text composed of the output characters. In specific implementation, it is necessary to extract the target summary from the text except for the model-generated tags (such as the start tag and the end tag).

[0117] Furthermore, in order to optimize the update of the length factor, an optional implementation method provided in an embodiment of the present application updates the length factor based on the text length of the target text, the text length of the text, and the importance score of the target text to obtain a target length factor, including:

[0118] Determine the text length of the target text;

[0119] Determining a summary length threshold of the target text based on the text length of the target text and a preset summary length condition;

[0120] The length factor is updated according to the text length of the text, the summary length threshold, and the importance score of the target text.

[0121] The text length of the target text refers to the number of characters in the target text. Accordingly, the preset summary length condition refers to the pre-set condition that the summary length must meet; the summary length threshold refers to the maximum upper limit of the summary length generated by the text generation model. For example, if the preset summary length condition is: the maximum summary length is 0.1 times the text length of the target text, then the summary length threshold (i.e., L) can be calculated based on the text length of the target text and the preset summary length condition. lim =0.1*L d , where L lim is the summary length threshold, i.e. the maximum length limit of the target summary, L d is the text length of the target text), therefore, the length factor of the summary finally generated by the target text (that is, the length factor of the text generation model) can be calculated using the importance score of the target text and the text length L d Take control together.

[0122] In specific implementation, the length factor is calculated using the following formula:

[0123] α=f(score,γ), γ=exp(LL lim ), where L≤L lim

[0124] Among them, L lim =β*L d , where the coefficient β represents the ratio of the longest abstract length, and the coefficient β can be adjusted by itself, such as taking 0.1 or 0.15; L d is the text length of the target text; L represents the text length of the text generated by the text generation model; function f is used to control the size of the penalty factor α (i.e., length factor). Function f can be set by yourself. The basic principle is that the larger the score, the smaller the penalty, the larger the penalty factor α (length factor), and the larger the γ, the larger the penalty, and the smaller the penalty factor α (length factor). Among them, γ is a variable that increases with the increase of L, because L represents the text length of the currently generated text, and L is constantly increasing. When L is closer to L lim The closer γ is to 1, the more we want to stop generating summaries, so the smaller the penalty factor α (length factor) is.

[0125] Continuing with the above example, the target text d1 and the initial summary 1 are input into the generated text model. The target text d1 is summarized using the length factor α1 in the generated text model and the initial summary 1 as references, and the first character c1_1 of the target text d1 is output. The calculation formula of α1 is α1=f(score1, exp(L-βL lim 1), the coefficient β represents the ratio of the longest abstract length.

[0126] If the result of judging whether the first character c1_1 is an end mark is no, the first character c1_1 is added to the text, and the text length of the text is determined to be 1. d 1. When the preset summary length condition is that the longest summary length ratio is 0.1 times the text length of the target text (i.e. the longest summary length ratio is 0.1), the summary length threshold L of the target text d1 can be determined. lim 1=0.1*L d 1, and based on the above formula, update α to obtain the updated length factor (i.e., target length factor) α1 = f(score1, exp(1-0.1*L d 1)).

[0127] The execution continues with the updated length factor α1 in the generated text model and the initial summary 1 as a reference to generate a summary of the target text d1 and output the second character c1_2 of the target text d1. If the result of the judgment on whether the second character c1_2 is an end mark is no, the second character c1_2 is added to the text, the text length of the text is determined to be 2, and α1 is updated based on the above formula to obtain the updated length factor (i.e., the target length factor) α1 = f(score1, exp(2-0.1*L d 1)).

[0128] And continue to execute to generate the updated length factor α1 in the text model and the initial summary 1 as a reference to generate a summary of the target text d1 and output the third character c1_3 of the target text d1, and so on. Assuming that the judgment result of whether the fiftieth character c1_50 is the end mark is yes, the fiftieth character c1_50 is added to the text, and the characters other than the start mark and the end mark are extracted from the text to form the target summary 1 of the target text d1, that is, the target summary 1 is composed of the characters other than the start mark and the end mark from the first character c1_1 to the fiftieth character c1_50.

[0129] Similarly, the target text d2 and the initial summary 2 are input into the generated text model, and the target text d2 is summarized and outputted with reference to the length factor α2 and the initial summary 2 in the generated text model. The specific implementation method of the target summary 2 of the target text d2 is similar to the specific implementation of the target summary 1 of the target text d1. The specific implementation of the target summary 1 of the target text d1 can be referred to. It should be noted that the above-mentioned determination that the text length of the text is 1 indicates that the first character c1_1 is not a start mark; if the first character c1_1 is a start mark, then after the first character c1_1 is added to the text, the text length of the determined text is 0, then α1=f(score1,exp(0-0.1*L d 1)), and so on, to output and calculate subsequent characters.

[0130] In summary, the level of detail generated for each target text is dynamically adjusted according to the importance score of each target text and the text length of the target text, and then a corresponding target summary is generated for each target text, thereby achieving dynamic adjustment of the summary length of the target summary and making the generated target summary more in line with the user's reading needs.

[0131] Since each target text generates a target summary, when there are a large number of target texts, the user's reading volume is still large, and the generated target summaries may have the same or similar content. In order to further improve the user's reading efficiency and reduce the time and energy consumed by the user in reading the target texts, an optional implementation method provided by the embodiment of the present application, when there are multiple target texts, also includes:

[0132] Splicing the target summaries of each target text to obtain a spliced summary;

[0133] Perform text generation on the concatenated summary to obtain a target combined summary of the target text.

[0134] In practical applications, concatenating the target summaries of each target text can be understood as combining the text information in the target summaries of each target text to generate a new summary (i.e., a concatenated summary). In specific implementations, the order of concatenating the target summaries can be random or in a preset order, which is not limited here.

[0135] Based on the obtained concatenated summary, text generation is performed on the concatenated summary. This can be understood as treating the concatenated summary as a piece of text and generating a corresponding summary for that text. In practical applications, a generative summarization model can be used to generate summaries of the concatenated summary, generating a summary of the concatenated summary, and using the summary of the concatenated summary as the summary of multiple target texts (i.e., a target composite summary). Alternatively, an extractive summarization model can be used to extract summaries from the concatenated summary to generate a target composite summary of the concatenated summary. Specifically, the basic structure of a generative summarization model consists of an encoder and a decoder, both of which are implemented by neural networks. The encoder encodes the input concatenated summary into a vector C (context), while the decoder extracts important information from this vector C, processes and edits it, and generates a text summary. This architecture is called Sequence-to-Sequence (hereafter referred to as Seq2Seq) and is widely used in scenarios with input and output sequences. In specific implementations, the generative summarization model can be a PEGASUS model.

[0136] Based on the target summary of the target text obtained above, target summary 1 of target text d1 and target summary 2 of target text d2 are concatenated to obtain a concatenated summary. The concatenated summary is input into the generative summarization model for summary generation to obtain a target combined summary of target text d1 and target text d2.

[0137] In the prior art, the summary generation method for multiple texts is mostly to use multiple texts to directly generate a summary, which usually loses a lot of important information. The effect of the summary finally generated is much worse than that of a single text, and it cannot be adjusted accordingly according to the characteristics of each text. In view of this, in order to improve the accuracy and flexibility of multi-text summary generation, the embodiment of the present application calculates the importance score (i.e., importance score) of each target text, dynamically adjusts the level of detail of the summary generated for each text according to the importance score of each target text and other information (such as the length of the single text), and then generates a corresponding single text summary for each target text, and then uses the generated multiple single text summaries to generate a summary information corresponding to multiple texts (i.e., target combination summary).

[0138] For details, see Figure 4 , Figure 4 A schematic diagram of a summary generation method provided according to an embodiment of the present application in a multi-text scenario is shown.

[0139] In the summary generation scenario of multiple texts (multiple documents), on the basis of obtaining multiple documents, the importance score of each document is obtained by calculating the document importance of each document, specifically including document d1 and the importance score score1 corresponding to document d1, abbreviated as (d1, score1), document d2 and the importance score score2 corresponding to document d2, abbreviated as (d2, score2), ... and document dN and the importance score scoreN corresponding to document dN, abbreviated as (dN, scoreN), etc., and a summary is generated for each document through a single-document hybrid summary extraction algorithm, that is, a summary is generated for each document by a mixture of extractive summary and generative summary to obtain the target summary of each document, and then the target summaries of these documents are combined into a summary set, and summary generation is performed based on the summary set to obtain a multi-document summary.

[0140] In summary, the summary generation method provided in this application obtains a target text to be processed, extracts a summary of the target text, obtains an initial summary of the target text, then inputs the target text and the initial summary into a text generation model, generates a summary of the target text using the initial summary as a reference, and outputs a target summary of the target text. This method uses the extracted summary content as the control text to control the text generation model for summary generation, not only leveraging the strong correlation of extractive summaries but also ensuring that the generated summary is fluent and logical.

[0141] The following combined Figure 5 , taking the application of the abstract generation method provided by this application in the retrieval scenario as an example, the abstract generation method is further explained. Figure 5 A processing flow chart of a summary generation method applied to a search scenario provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0142] Step 502: Receive information to be retrieved input by the user.

[0143] In practical applications, when users perform information retrieval on a text library containing a large amount of text (also understood as documents), they may retrieve a large number of relevant texts. For example, if a user wishes to search for information related to "brake failure," searching the text library by entering "brake failure" as the search information may result in dozens or even hundreds of relevant texts. Users need to read all of the retrieved texts to obtain the most comprehensive information on "brake failure." This consumes considerable time and effort, and this reading method significantly impacts the efficiency with which users obtain relevant information. Prior art methods often assist users in reading large amounts of text by providing summaries of the retrieved texts. These summaries are typically generated using extractive or generative summarization techniques.

[0144] However, although the extractive summarization technology in the prior art makes the extracted summary highly relevant to the original text in terms of grammar and syntax, the extracted summary has poor coherence and flexibility, and errors in the extraction of summary content may occur; and although the generative summarization technology in the prior art allows the summary to include new words or phrases, making the generated summary highly flexible, it is easy to deviate from the original text.

[0145] In view of this, in order to improve user reading efficiency and ensure the accuracy of summary information, the embodiments of this application combine the advantages of extractive summarization and generative summarization to generate text summaries. By controlling the generation of generative summaries through extractive summarization, the generated summary is not only highly relevant to the original text in terms of grammar and syntax, but also generates a smooth and logical summary text that can highly summarize the original text.

[0146] Specifically, the information to be retrieved may be keywords, query conditions, query questions, etc., which are not limited here.

[0147] Step 504: Search the text library according to the information to be retrieved input by the user to obtain m initial texts corresponding to the information to be retrieved.

[0148] Specifically, you can search for texts in the text library through keyword matching. For example, if the user inputs the information to be searched as Q1, keyword matching is performed on the texts in the text library based on the keywords in Q1 to obtain three initial texts corresponding to Q1: initial text t1, initial text t2, and initial text t3.

[0149] Step 506: Segment the information to be retrieved to obtain the target word units contained in the information to be retrieved, and count the target word units contained in each initial text to obtain the statistical score of each target word unit in each initial text, and determine the statistical score of each initial text based on the statistical score of each target word unit.

[0150] Based on this, Q1 is segmented to obtain the target word units word1 and word2 contained in Q1. The TF-IDF statistical algorithm is used to calculate the statistical scores of the target word units word1 and word2 in each of the three initial texts. Among them, the statistical score of word1 in the initial text t1 is S Score 1 _ The statistical score of word 1 and word 2 is S Score 1_2, the statistical score of word1 in the initial text t2 is S Score The statistical scores of 2_1 and word2 are S Score 2_2, the statistical score of word1 in the initial text t3 is S Score The statistical scores of 3_1 and word2 are S Score 3_2. And further S Score 1_1 and S Score The sum of 1_2 S Score 1, as the statistical score of the initial text t1, and S Score_ 2_1 and S Score The sum of 2_2 S Score 2 as the statistical score of the initial text t2, and S Score 3_1 and S Score The sum of 3_2 S Score 3 is used as the statistical score of the initial text t3.

[0151] Step 508: Input the information to be retrieved and the m initial texts into the ranking model, and obtain the ranking score of each initial text output by the ranking model.

[0152] Based on this, the information to be retrieved Q1 and the three initial texts are input into the ranking model to obtain the ranking score R of the initial text t1. Score 1. Ranking score R of initial text t2 Score 2 and the ranking score R of the initial text t3 Score 3.

[0153] Step 510: Perform a weighted average of the statistical score and ranking score of each initial text to obtain the importance score of each initial text.

[0154] Specifically, by combining the statistical score and the ranking score to calculate the importance score of each text, the importance score can more accurately reflect the importance of each initial text relative to the information to be retrieved.

[0155] Based on this, the preset weight for the statistical score is determined to be w1, the preset weight for the ranking score is w2, and w1+w2=1, then the importance score of the initial text t1 is S Score 1*w1+R Score 1*w2=score1, the importance score of the initial text t2 is S Score 2*w1+R Score 2*w2=score2, the importance score of the initial text t3 is S Score 3*w1+R Score 3*w2=score3.

[0156] Step 512: Filter out n target texts from the m initial texts based on the importance score of each initial text; where m≥n and n is a positive integer.

[0157] On the basis of determining the importance score of each initial text, the importance score of initial text t1 is score1, the importance score of initial text t2 is score2, and the importance score of initial text t3 is score3, among which score1>score2>score3, then two initial texts with higher importance scores are selected from the three initial texts according to the importance scores. The two initial texts with higher importance scores are initial text t1 and initial text t2, and these two initial texts are used as target texts.

[0158] Step 514: extract a summary of each target text to obtain an initial summary of each target text.

[0159] Based on this, the TextRank algorithm is used to extract summaries of the target text t1 and the target text t2 respectively, and the extractive summary of the target text t1 is obtained as the initial summary ab1, and the extractive summary of the target text t2 is obtained as the initial summary ab2.

[0160] Step 516: Input each target text and the initial summary of each target text into the text generation model respectively, generate a summary for each target text with reference to the length factor and the initial summary in the text generation model, and output the target summary of each target text.

[0161] Based on this, the target text t1 and the initial summary ab1 are input into the generated text model. The target text t1 is summarized with reference to the length factor α1 in the generated text model and the initial summary ab1, and the first character pab1_1 of the target text t1 is output. The calculation formula of α1 is α1=f(score1, exp(L-βL d )), where the coefficient β represents the ratio of the longest summary length; L represents the length of the text currently generated by the text generation model; and the function f is used to control the size of the penalty factor α1 (i.e., the length factor). The function f can be set by yourself. The basic principle is that the larger the score, the smaller the penalty, and the larger the penalty factor α1, exp(L-βL d ) is larger, the greater the penalty is, and the smaller the penalty factor α1 is.

[0162] If the result of judging whether the first text pab1_1 contains an end mark is no, the first character pab1_1 is added to the text, and the text length of the text is determined to be 1, and the text length of the target text t1 is determined to be L. d , when the preset summary length condition is 0.15 times the text length of the target text, the summary length threshold L of the target text t1 can be determined lim 1=0.15*L d , and based on the above formula, update α1 to obtain the updated length factor α1=f(score1,exp(1-0.15*L d )).

[0163] And continue to execute to generate the updated length factor α1 in the text model and the initial summary ab1 as a reference to generate a summary of the target text t1 and output the second character pab1_2 of the target text t1, and judge whether the second character pab1_2 is an end mark. If the judgment result is no, the second character pab1-2 is added to the text, and the text length of the text is determined to be 2, and α1 is updated based on the above formula to obtain the updated length factor α1=f(score1, exp(2-0.15*L d )).

[0164] And continue to execute to generate the updated length factor α1 in the text model and the initial summary ab1 as a reference to generate a summary of the target text t1 and output the third character pab1_3 of the target text t1, and so on. Assuming that the judgment result of whether the fiftieth character pab1_50 is an end mark is yes, the fiftieth character pab1_50 is added to the text, and the characters other than the start mark and the end mark are extracted from the text to form the target summary pab1 of the target text t1, that is, the target summary pab1 is composed of the characters other than the start mark and the end mark from the first character pab1_1 to the fiftieth character pab1_50.

[0165] Similarly, the target text t2 and the initial summary ab2 are input into the generated text model, and the target text t2 is summarized and outputted with reference to the length factor α2 and the initial summary ab2 in the generated text model. The specific implementation method of outputting the target summary pab2 of the target text t2 is similar to the specific implementation of the target summary pad1 of the above-mentioned output target text t1, and the specific implementation of the target summary pad1 of the above-mentioned output target text t1 can be referred to. It should be noted that the above-mentioned determination that the text length of the text is 1 indicates that the first character pab1_1 is not a start mark; if the first character pab1_1 is a start mark, then after the first character pab1_1 is added to the text, the text length of the determined text is 0, then α1=f(score1,exp(0-0.15*L d )), and so on, to output and calculate subsequent characters.

[0166] Step 518: Concatenate the target summaries of each target text to obtain a concatenated summary.

[0167] On the basis of the target summary of the target text obtained above, the target summary pab1 of the target text t1 and the target summary pab2 of the target text t2 are concatenated to obtain a concatenated summary cpab.

[0168] Step 520: Generate a summary of the concatenated summary to obtain a target combined summary of the target text.

[0169] On the basis of the concatenated summary obtained above, a summary is generated for the concatenated summary cpab to obtain the target combined summary pcab of the target text t1 and the target text t2.

[0170] In summary, the summary generation method provided by the embodiment of the present application utilizes the extracted summary content, uses the extracted summary content as the control text, and controls the text generation model to generate the summary. It can not only utilize the strong correlation of the extractive summary, but also utilize the generation method to generate a smooth and logical summary text that can highly summarize the original text. In addition, in the summary generation scenario of multiple texts, the application embodiment can also calculate the importance score of each target text, dynamically adjust the level of detail generated for each text according to the importance score of each target text and other information (such as the length of a single text), and then generate a corresponding single text summary for each target text, and then use the generated multiple single text summaries to generate a multi-text summary information (i.e., a target combination summary), thereby improving the accuracy and flexibility of multi-text summary generation.

[0171] Corresponding to the above method embodiment, the present application also provides an embodiment of a summary generation device, Figure 6 FIG. 1 shows a schematic diagram of the structure of a summary generation device provided by an embodiment of the present application. Figure 6 As shown, the device includes:

[0172] An acquisition module 602 is configured to acquire a target text to be processed;

[0173] An extraction module 604 is configured to extract a summary of the target text to obtain an initial summary of the target text;

[0174] The generation module 606 is configured to input the target text and the initial summary into a text generation model, generate a summary of the target text with reference to the initial summary, and output a target summary of the target text.

[0175] Optionally, the acquisition module 602 includes:

[0176] A receiving submodule is configured to receive information to be retrieved submitted by a user;

[0177] A retrieval submodule is configured to perform a search based on the information to be retrieved, and obtain at least one initial text corresponding to the information to be retrieved;

[0178] The determination submodule is configured to filter the target text from the at least one initial text.

[0179] Optionally, the determining submodule includes:

[0180] a calculation unit configured to calculate an importance score of each initial text;

[0181] The text unit is determined and configured to filter a target text from the at least one initial text based on the importance score.

[0182] Optionally, the generating module 606 includes:

[0183] a character generation submodule configured to input the target text and the initial summary into a text generation model, generate a summary of the target text based on the length factor in the text generation model and the initial summary, and output an i-th character;

[0184] A judging submodule is configured to judge whether the i-th character is an end mark;

[0185] If not, running an update submodule, the update submodule is configured to determine the text length of the text to which the i-th character is added, and update the length factor based on the text length of the target text, the text length of the text, and the importance score of the target text to obtain a target length factor; i+1, using the target length factor as the length factor, and performing summary generation on the target text with reference to the length factor in the text generation model and the initial summary, and outputting the i+1-th character;

[0186] If so, the extraction submodule is executed, wherein the extraction submodule is configured to add the i-th character to the text and extract the target summary from the text.

[0187] Optionally, the update submodule is further configured to:

[0188] Determining the text length of the target text;

[0189] Determining a summary length threshold of the target text based on the text length of the target text and a preset summary length condition;

[0190] The length factor is updated according to the text length of the text, the summary length threshold, and the importance score of the target text.

[0191] Optionally, the computing unit is further configured to:

[0192] Segmenting the information to be retrieved to obtain target word units contained in the information to be retrieved; performing statistics on the target word units contained in each initial text to obtain a statistical score of each target word unit in each initial text, and determining a statistical score of each initial text based on the statistical score of each target word unit, and using the statistical importance score as the importance score; or

[0193] The information to be retrieved and the at least one initial text are input into a ranking model, a ranking score of each initial text output by the ranking model is obtained, and the ranking score is used as the importance score.

[0194] Optionally, the computing unit is further configured to:

[0195] Segmenting the information to be retrieved to obtain target word units contained in the information to be retrieved;

[0196] Counting the target word units contained in each original text to obtain a statistical score for each target word unit in each original text;

[0197] Determine the statistical score of each initial text based on the statistical score of each target word unit;

[0198] Inputting the information to be retrieved and the at least one initial text into a ranking model, and obtaining a ranking score of each initial text output by the ranking model;

[0199] The statistical score and ranking score of each initial text are weighted averaged to obtain the importance score of each initial text.

[0200] Optionally, when there are multiple target texts, the summary generating device further includes:

[0201] a splicing module configured to splice the target summaries of each target text to obtain a spliced summary;

[0202] The text generation module is configured to generate text for the concatenated summary to obtain a target combined summary of the target text.

[0203] Optionally, the text generation model includes an embedding layer, an encoding layer, and a decoding layer, and the generation module 606 includes:

[0204] an embedding submodule configured to input the target text and the corresponding initial summary into an embedding layer to obtain a text vector corresponding to the target text and a control vector corresponding to the initial summary;

[0205] an encoding submodule, configured to input the text vector and the control vector into the encoding layer to obtain an encoding vector;

[0206] The decoding submodule is configured to input the encoding vector and the text vector into the decoding layer for decoding processing, and output the target summary according to the decoding processing result.

[0207] Optionally, the encoding layer includes a self-attention layer and a feedforward neural network layer, and the encoding submodule is further configured as follows:

[0208] A self-attention unit is configured to input the text vector and the control vector into the self-attention layer to obtain a self-attention vector;

[0209] The conversion unit is configured to input the self-attention vector into the feedforward neural network layer for conversion processing to obtain an encoding vector.

[0210] Optionally, the self-attention unit is further configured to:

[0211] splicing the key vector corresponding to the control vector and the key vector corresponding to the text vector to obtain a spliced key vector;

[0212] splicing the value vector corresponding to the control vector and the value vector corresponding to the text vector to obtain a spliced value vector;

[0213] Self-attention processing is performed based on the splicing key vector, the splicing value vector, and the question vector corresponding to the text vector to obtain a self-attention vector.

[0214] Optionally, the self-attention submodule is further configured to:

[0215] splicing the key vector corresponding to the control vector and the key vector corresponding to the text vector to obtain a spliced key vector;

[0216] splicing the value vector corresponding to the control vector and the value vector corresponding to the text vector to obtain a spliced value vector;

[0217] Based on the question vector corresponding to the text vector, the splicing key vector, and the splicing value vector, a self-attention vector output by the self-attention layer is obtained.

[0218] In summary, the summary generation device provided by this application obtains a target text to be processed, extracts a summary of the target text, obtains an initial summary of the target text, then inputs the target text and the initial summary into a text generation model, generates a summary of the target text using the initial summary as a reference, and outputs a target summary of the target text. This method uses the extracted summary content as the control text to control the text generation model for summary generation, not only leveraging the strong correlation of extractive summaries but also ensuring that the generated summary is fluent and logical.

[0219] The above is a schematic diagram of a summary generation device according to this embodiment. It should be noted that the technical solution of this summary generation device and the technical solution of the summary generation method described above share the same concept. For details not described in detail in the technical solution of the summary generation device, please refer to the description of the technical solution of the summary generation method described above.

[0220] It should be noted that the components in a device claim should be understood as the functional modules necessary to implement each step of the program flow or method. The individual functional modules are not defined by actual functional division or separation. A device claim defined by such a set of functional modules should be understood as a functional module architecture that primarily implements the solution through the computer program described in the specification, rather than a physical device that primarily implements the solution through hardware.

[0221] In one embodiment of the present application, a computing device is further provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the computer instructions, the steps of the summary generation method are implemented.

[0222] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the summary generation method described above are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the summary generation method described above.

[0223] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the summary generation method as described above.

[0224] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the summary generation method described above are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the summary generation method described above.

[0225] An embodiment of the present application discloses a chip storing computer instructions, which, when executed by a processor, implement the steps of the summary generation method described above.

[0226] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0227] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0228] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0229] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0230] The preferred embodiments of the present application disclosed above are intended only to help illustrate the present application. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of this application. This application selects and describes these embodiments in detail in order to better explain the principles and practical applications of this application, so that those skilled in the art can better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.< / eos>

Claims

1. A method for generating a summary, characterized in that: include: Get the target text to be processed; Extracting a summary of the target text to obtain an initial summary of the target text; The target text and the initial summary are input into a text generation model, and a summary of the target text is generated with reference to the initial summary, and a target summary of the target text is output, wherein the inputting the target text and the initial summary into the text generation model includes: inputting the target text and the initial summary into the text generation model, and each time a character is output, iteratively updating a length factor based on the text length of the text composed of the characters generated by the text generation model, the text length of the target text, and the importance score of the target text until the text generation model outputs an end tag, and extracting a target summary from the text composed of the characters generated by the text generation model, and the length factor affects the probability of the text generation model outputting an end tag.

2. The method for generating a summary according to claim 1, wherein: The step of obtaining the target text to be processed includes: Receive information to be retrieved submitted by the user; Performing a search based on the information to be retrieved to obtain at least one initial text corresponding to the information to be retrieved; A target text is selected from the at least one initial text.

3. The method for generating a summary according to claim 2, wherein: The step of screening the target text from the at least one initial text includes: Calculate the importance score of each initial text; Based on the importance score, a target text is selected from the at least one initial text.

4. The method for generating a summary according to claim 3, wherein: The step of inputting the target text and the initial summary into a text generation model, generating a summary of the target text with reference to the initial summary, and outputting a target summary of the target text includes: Inputting the target text and the initial summary into a text generation model, generating a summary of the target text with reference to the length factor in the text generation model and the initial summary, and outputting the i-th character; Determine whether the i-th character is an end mark; If not, determining the text length of the text to which the i-th character is added, and updating the length factor based on the text length of the target text, the text length of the text, and the importance score of the target text to obtain a target length factor; using the target length factor as the length factor, and performing summary generation on the target text with reference to the length factor in the text generation model and the initial summary, and outputting the i+1-th character; If so, the i-th character is added to the text, and a target summary is extracted from the text.

5. The method for generating a summary according to claim 4, wherein: The updating of the length factor based on the text length of the target text, the text length of the text, and the importance score of the target text to obtain a target length factor includes: Determining the text length of the target text; Determining a summary length threshold of the target text based on the text length of the target text and a preset summary length condition; The length factor is updated according to the text length of the text, the summary length threshold, and the importance score of the target text to obtain a target length factor.

6. The method for generating a summary according to claim 3, wherein: The calculation of the importance score of each initial text includes: Segmenting the information to be retrieved to obtain target word units contained in the information to be retrieved; performing statistics on the target word units contained in each initial text to obtain a statistical score for each target word unit in each initial text, and determining a statistical score for each initial text based on the statistical score of each target word unit, and using the statistical score as the importance score; or The information to be retrieved and the at least one initial text are input into a ranking model, a ranking score of each initial text output by the ranking model is obtained, and the ranking score is used as the importance score.

7. The method for generating a summary according to claim 3, wherein: The calculation of the importance score of each initial text includes: Segmenting the information to be retrieved to obtain target word units contained in the information to be retrieved; Counting the target word units contained in each original text to obtain a statistical score for each target word unit in each original text; Determine the statistical score of each initial text based on the statistical score of each target word unit; Inputting the information to be retrieved and the at least one initial text into a ranking model, and obtaining a ranking score of each initial text output by the ranking model; The statistical score and ranking score of each initial text are weighted averaged to obtain the importance score of each initial text.

8. The method for generating a summary according to claim 1, wherein: In the case where there are multiple target texts, it also includes: Splicing the target summaries of each target text to obtain a spliced summary; Text generation is performed on the concatenated summary to obtain a target combined summary of the target text.

9. The method for generating a summary according to claim 1, wherein: The text generation model includes an embedding layer, an encoding layer, and a decoding layer. The step of inputting the target text and the initial summary into the text generation model, generating a summary of the target text with reference to the initial summary, and outputting a target summary of the target text includes: Inputting the target text and the corresponding initial summary into the embedding layer to obtain a text vector corresponding to the target text and a control vector corresponding to the initial summary; Inputting the text vector and the control vector into the encoding layer to obtain an encoding vector; The encoding vector is input into the decoding layer for decoding processing, and the target summary is output according to the decoding processing result.

10. The method for generating a summary according to claim 9, wherein: The encoding layer includes a self-attention layer and a feedforward neural network layer. Inputting the text vector and the control vector into the encoding layer to obtain the encoding vector output by the encoding layer includes: Inputting the text vector and the control vector into the self-attention layer to obtain a self-attention vector; The self-attention vector is input into the feedforward neural network layer for conversion processing to obtain an encoding vector.

11. The method for generating a summary according to claim 10, wherein: Inputting the text vector and the control vector into the self-attention layer to obtain the self-attention vector output by the self-attention layer includes: splicing the key vector corresponding to the control vector and the key vector corresponding to the text vector to obtain a spliced key vector; splicing the value vector corresponding to the control vector and the value vector corresponding to the text vector to obtain a spliced value vector; Self-attention processing is performed based on the splicing key vector, the splicing value vector, and the question vector corresponding to the text vector to obtain a self-attention vector.

12. A summary generation device, characterized in that: include: An acquisition module is configured to acquire a target text to be processed; an extraction module configured to extract a summary of the target text to obtain an initial summary of the target text; A generation module is configured to input the target text and the initial summary into a text generation model, generate a summary of the target text with reference to the initial summary, and output a target summary of the target text, wherein the inputting the target text and the initial summary into the text generation model includes: inputting the target text and the initial summary into the text generation model, and each time a character is output, iteratively updating a length factor based on the text length of the text composed of the characters generated by the text generation model, the text length of the target text, and the importance score of the target text until the text generation model outputs an end tag, extracting the target summary from the text composed of the characters generated by the text generation model, and the length factor affecting the probability of the text generation model outputting an end tag.

13. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein: When the processor executes the computer instructions, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

Images