Document abstract generation method and device, storage medium and electronic device

Abstract

The invention discloses a document abstract generation method and device, a storage medium and an electronic device.The method comprises the steps that key sentences in a first document are screened out from the first document through a pre-trained BERT model and a sentence embedding algorithm; document information of the first document and the key sentences are combined through a preset prompt word template, combined data are obtained, and the document information at least comprises a document title of the first document; and inputting the combined data into a large model to generate a document abstract of the first document. The problem that an extraction scheme of the document abstract of the long document is poor in extraction effect is solved, and the extraction effect of the document abstract is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method and device for generating a document summary, a storage medium, and an electronic device. Background Art

[0002] In the field of automatic text summarization, summarization generation for long documents faces a series of challenges, especially in extractive summarization methods. Extractive summarization technology, as a means of directly selecting key sentences based on the original content to construct a summary, ensures the reliability of information, but the limitations of traditional sentence embedding methods in processing long sentences cause the quality of its summary to be significantly affected. Specifically, when simple strategies such as mean pooling of word embeddings are used to represent sentences, redundant or non-critical information in long sentences will dilute the expressive power of sentence vectors, causing the model to tend to select shorter sentences, even if these sentences do not necessarily contain the most core information. This sentence length bias not only reduces the semantic coherence of the summary, but also may miss key details in the document, failing to fully reflect the main idea and importance of the original text, and thus making it difficult to meet users' needs for high-quality, comprehensive summaries.

[0003] On the other hand, generative summary algorithms, through deep learning models such as recurrent neural networks (RNN) or Transformer architectures, can produce more abstract and coherent summary content, thus having an advantage in the freedom of information expression. In particular, the introduction of the Transformer model overcomes the serial computing limitations of RNN and significantly improves the efficiency of summary generation. However, with the surge in the scale of model parameters, especially the emergence of pre-trained models such as the Bidirectional Encoder Representations from Transformers (BERT) model, although the generalization ability and summary quality of the model have been improved, the computing resource requirements and inference time when processing long documents have also increased accordingly, limiting its application in high-efficiency summary generation scenarios.

[0004] More importantly, since the rise of large-scale autoregressive language models such as GPT-3, although they can take into account the dual tasks of language understanding and generation, in order to achieve adaptability to multiple tasks, the model learns from massive data, resulting in the expansion of parameter scale, further exacerbating the consumption of computing resources. In an environment with limited resources, the generation of summaries of long documents becomes extremely slow, making it difficult to achieve fast and accurate information extraction and summarization in practical applications, especially in the summary tasks of information-intensive texts such as news, academic papers, and market reports. This problem is particularly prominent.

[0005] Therefore, how to overcome the limitations of extractive summarization in processing long documents while ensuring the quality of summaries, and how to optimize the resource consumption of generative summaries, has become an important topic in the current research of text summarization algorithms. In other words, in the related art, there is a problem that the extraction effect of document summarization for long documents is poor.

[0006] In view of the problem in the related art that the extraction effect of document summaries of long documents is poor, no effective solution has been found yet. Summary of the invention

[0007] The embodiments of the present application provide a method and device for generating a document summary, a storage medium, and an electronic device, so as to at least solve the problem of poor extraction effect of document summary extraction schemes for long documents in the prior art.

[0008] According to one embodiment of the embodiments of the present application, a method for generating a document summary is provided, comprising: filtering out key sentences in a first document from the first document through a pre-trained BERT model and a sentence embedding algorithm; combining document information of the first document with the key sentences through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document; and inputting the combined data into a large model to generate a document summary of the first document.

[0009] In an exemplary embodiment, key sentences in a first document are screened out from the first document using a pre-trained BERT model and a sentence embedding algorithm, including: determining a first sentence embedding vector corresponding to each sentence in the first document using the BERT model; inputting the first sentence embedding vector into the sentence embedding algorithm to obtain a second sentence embedding vector for each sentence; and determining the key sentence using the second sentence embedding vector.

[0010] In an exemplary embodiment, determining a first sentence embedding vector corresponding to each sentence in the first document by the BERT model includes: inserting separators at the beginning and the end of a first character sequence corresponding to each sentence to obtain a second character sequence corresponding to the first document, wherein the second character sequence can be directly encoded by the BERT model; and inputting the second character sequence into the BERT model to obtain the first sentence embedding vector.

[0011] In an exemplary embodiment, determining the key sentence by the second sentence embedding vector includes: inputting the second sentence embedding vector into a linear layer to obtain a key score of each sentence; comparing the key score with a key sentence score threshold; and determining that each sentence is a key sentence in the first document when the key score is greater than the key sentence score threshold.

[0012] In an exemplary embodiment, after comparing the criticality score with a critical sentence score threshold, the method further includes: determining that each sentence is not a critical sentence in the first document if the criticality score is less than the critical sentence score threshold; and updating each sentence by the next sentence of each sentence.

[0013] In an exemplary embodiment, before filtering out key sentences in the first document from the first document through a pre-trained BERT model and a sentence embedding algorithm, the method also includes: performing word segmentation processing on the acquired second document, and filtering stop words that are not related to the text topic of the first document through a stop word dictionary to obtain first text data; cleaning special characters in the first text data through regular expressions and pre-built special symbol representations to obtain second text data; splitting the second text data into N sentences through the regular expression to obtain the first document, where N is a positive integer.

[0014] According to another embodiment of the embodiments of the present application, a device for generating a document summary is also provided, including: a screening module, used to screen out key sentences in the first document from the first document through a pre-trained BERT model and a sentence embedding algorithm; a combining module, used to combine the document information of the first document with the key sentences through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document; a generating module, used to input the combined data into a large model to generate a document summary of the first document.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0016] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0018] In an embodiment of the present application, a key sentence in the first document is screened out from the first document through a pre-trained BERT model and a sentence embedding algorithm; the document information of the first document is combined with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: the document title of the first document; the combined data is input into a large model to generate a document summary of the first document. Through the above embodiment, the problem of poor extraction effect of the document summary extraction scheme for long documents in the prior art is solved, and the extraction effect of the document summary is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for generating a document summary according to an embodiment of the present application;

[0021] Figure 2 is a flowchart of a method for generating a document summary according to an embodiment of the present application;

[0022] Figure 3 It is a technical flow diagram of a method for generating a document summary according to an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the structure of each dimension-level attention module of the method for generating a document summary according to an embodiment of the present application;

[0024] Figure 5 It is a structural block diagram of a device for generating a document summary according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices; "plurality" means two or more.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device or cloud platform or independent physical server or software platform, wherein the above software platform is run through one or more servers. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for a method for generating a document summary according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the methods in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned methods are implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for generating a document summary is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flowchart of a method for generating a document summary according to an embodiment of the present application, the process comprising the following steps:

[0031] Step S202, filtering out key sentences in the first document from the first document using a pre-trained BERT model and a sentence embedding algorithm;

[0032] Step S204, combining the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: the document title of the first document;

[0033] Step S206: input the combined data into a large model to generate a document summary of the first document.

[0034] like Figure 3 As shown, the prompt word template can be: [key sentence 1, key sentence 2, ..., key sentence m, document type, document name, document related information]. Among them, the document title is the document name.

[0035] Through the above steps, the key sentences in the first document are screened out from the first document through the pre-trained BERT model and sentence embedding algorithm; the document information of the first document is combined with the key sentences through a preset prompt word template to obtain combined data, wherein the document information at least includes: the document title of the first document; the combined data is input into the large model to generate a document summary of the first document. Through the above embodiment, the problem of poor extraction effect of the document summary extraction scheme for long documents in the prior art is solved, and the extraction effect of the document summary is improved.

[0036] In an exemplary embodiment, key sentences in a first document are screened out from the first document using a pre-trained BERT model and a sentence embedding algorithm, including: determining a first sentence embedding vector corresponding to each sentence in the first document using the BERT model; inputting the first sentence embedding vector into the sentence embedding algorithm to obtain a second sentence embedding vector for each sentence; and determining the key sentence using the second sentence embedding vector.

[0037] It is understandable that the embodiment of the present application identifies and filters key sentences in a document by combining a pre-trained BERT model with a custom sentence embedding algorithm (such as a sentence embedding algorithm DSE based on dimension-level attention). This process is divided into three main steps:

[0038] 1) Determine the first sentence embedding vector through the BERT model:

[0039] First, the entire first document is decomposed into multiple sentences, and each sentence is regarded as an independent text unit.

[0040] Then, each sentence is encoded using the pre-trained BERT model, which can understand the semantics, word order, and contextual information in a sentence, and converts each sentence into a first sentence embedding vector, which contains the semantic features and structural information of the sentence.

[0041] The input of the BERT model usually includes the word embedding, position embedding, and segment embedding of the sentence. After being processed by a multi-layer encoder module, the vector representation of each word is output, and these vectors are then integrated into a sentence-level vector representation.

[0042] 2) The sentence embedding algorithm is used to obtain the second sentence embedding vector:

[0043] The first sentence embedding vector is the output of the BERT model, but in extractive summarization, it may need further optimization to reduce the impact of sentence length bias.

[0044] To this end, the first sentence embedding vector is taken as input, and a second sentence embedding vector is generated through a designed sentence embedding algorithm (such as the DSE algorithm). This vector pays more attention to the key information of the sentence rather than just the length of the sentence.

[0045] The DSE algorithm uses multiple dimension-level attention modules to deeply process sentences, eliminating non-critical information and producing a more condensed and representative sentence vector representation, namely the second sentence embedding vector.

[0046] 3) Finally, the key sentence is determined by the second sentence embedding vector:

[0047] Through the above steps, the embodiment of the present application can effectively process long documents, reduce the impact of sentence length on summary quality, screen out key sentences that truly carry the core information of the document, and provide high-quality input for subsequent summary generation, thereby improving the coherence and accuracy of the summary. This method combines the powerful representation capabilities of the pre-trained model and the optimization characteristics of the custom algorithm, aiming to address the shortcomings of existing extractive summarization methods in processing long documents and improve the overall performance of summary generation.

[0048] In an exemplary embodiment, determining a first sentence embedding vector corresponding to each sentence in the first document by the BERT model includes: inserting separators at the beginning and the end of a first character sequence corresponding to each sentence to obtain a second character sequence corresponding to the first document, wherein the second character sequence can be directly encoded by the BERT model; and inputting the second character sequence into the BERT model to obtain the first sentence embedding vector.

[0049] It is understandable that when processing each sentence, the sentence must first be preprocessed to adapt to the input requirements of the BERT model. The BERT model requires each input sequence to have special markers at the beginning and end, namely "[CLS]" and "[SEP]" as separators at the beginning and end of the sentence. This is done to tell the model where the input sequence starts and ends, which is crucial for sentence-level understanding and encoding. The second character sequence obtained after inserting the separators, that is, the sequence containing these special markers, can be directly input to the BERT model for encoding.

[0050] The processed second character sequence is input into the BERT model for encoding. The BERT model is able to understand and encode the complex semantic relationships of the input text through a deep bidirectional Transformer architecture. During the encoding process, the model generates a vector representation for each character (or vocabulary unit) in the sequence. These vectors contain rich information about word meaning, context, and sequence position. For each sentence, the BERT model will output a series of word-level embedding vectors.

[0051] After BERT generates an embedding vector for each word in a sentence, the next challenge is how to integrate these word vectors into a sentence-level first sentence embedding vector. This can usually be achieved through a variety of strategies, such as taking the mean or weighted sum of all word vectors, or using specific word vectors (such as the vector of the [CLS] tag) to represent the semantics of the entire sentence. The specific method may vary depending on the implementation scenario, but the goal is to generate a sentence embedding vector that can fully represent the content and context of the sentence.

[0052] Through the above steps, this embodiment effectively utilizes the capabilities of the BERT pre-trained model to generate the first sentence embedding vector for each sentence in the document. These vectors will be used as input for subsequent processing for further sentence importance evaluation, so that key information can be more accurately identified and utilized when generating document summaries. This method is particularly important in the field of text summarization because it can help the model understand the complex structure and semantics of long documents and is the basis for building high-quality, coherent summaries.

[0053] In an exemplary embodiment, determining the key sentence by the second sentence embedding vector includes: inputting the second sentence embedding vector into a linear layer to obtain a key score of each sentence; comparing the key score with a key sentence score threshold; and determining that each sentence is a key sentence in the first document when the key score is greater than the key sentence score threshold.

[0054] Furthermore, after comparing the criticality score with the critical sentence score threshold, the method further includes: when the criticality score is less than the critical sentence score threshold, determining that each sentence is not a critical sentence in the first document; and updating each sentence through the next sentence of each sentence.

[0055] Optionally, the second sentence embedding vector of each sentence is input into a linear layer. The linear layer here is actually a simple neural network layer, usually containing a weight matrix and a bias term. The role of the linear layer is to convert the input sentence embedding vector into a scalar value, which can be used as the criticality score of the sentence. Optionally, in order to ensure that the score value is between 0 and 1, indicating the importance of the sentence, a nonlinear activation function such as the sigmoid function can be applied after the linear layer.

[0056] Once the key score of each sentence is generated, it is then compared to a preset key sentence score threshold. This threshold is a value determined empirically or experimentally to distinguish whether a sentence is important enough to be included in the document summary. By comparison, sentences with scores above the threshold are considered key sentences in the document, while sentences below the threshold may be excluded from the summary.

[0057] The above embodiments ensure that only those sentences that are considered by the algorithm to carry the core information of the document are selected, thus improving the accuracy and relevance of the abstract.

[0058] In an exemplary embodiment, before screening out the key sentences in the first document through a pre-trained BERT model and sentence embedding algorithm, the method further includes: performing word segmentation on the obtained second document, and filtering out stop words unrelated to the text theme of the first document through a stop word dictionary to obtain first text data; cleaning special characters in the first text data through regular expressions and a pre-constructed special symbol representation to obtain second text data; splitting the second text data into N sentences through the regular expressions to obtain the first document, where N is a positive integer.

[0059] Optionally, perform word segmentation on the obtained second document, that is, decompose the document into a series of words or lexical units. Then filter out stop words unrelated to the text theme of the first document through a stop word dictionary. Stop words usually refer to those words that appear frequently in the text but contribute little to the semantics, such as "of", "is", "in", etc. This process helps to reduce noise and focus on the core vocabulary in the text, obtaining the first text data after preliminary cleaning.

[0060] After word segmentation and stop word filtering, clean special characters in the first text data through regular expressions and a pre-constructed special symbol representation. Special characters include but are not limited to punctuation marks, numbers, website addresses, email addresses, etc. These characters often do not carry semantic information in natural language understanding and text summarization, or need to be processed in a specific way to avoid interfering with the model. Through regular expressions, these special characters can be accurately located and removed, obtaining cleaner text data, that is, second text data.

[0061] Finally, use regular expressions to further split the second text data into N sentences. Regular expressions can identify punctuation marks at the end of sentences, such as full stops, question marks, exclamation marks, etc., thus accurately splitting the document into independent sentence units.

[0062] Through the above embodiments, it can be ensured that the text data input to the BERT model and sentence embedding algorithm is cleaner and more structured, thus helping the model to more accurately understand and screen the key information in the document.

[0063] To better understand the process of the above method for generating a document abstract, the following further describes the above method for generating a document abstract in combination with optional embodiments, but it is not used to limit the technical solutions of the embodiments of the present application.

[0064] The optional embodiment of the present application relates to the field of artificial intelligence technology, and proposes a document summary generation method based on sentence importance screening and large model semantic understanding (equivalent to the document summary generation method in the above embodiment). The method first combines the BERT pre-training model (i.e., BERT model) and the dimension-level attention-based sentence embedding (DSE) algorithm to filter out key sentences from long documents. Then design a prompt word template, combine key sentences, document titles and other information, and input them into the large model. Finally, a document summary with excellent readability is generated through the large model. The optional embodiment of the present application effectively solves the problem of resource overflow when long documents are input into a large model, and improves the efficiency and quality of summary generation.

[0065] Specifically, the technical process of the optional embodiment of the present application is as follows: Figure 3 As shown, the following steps are included:

[0066] Step 1: Text preprocessing. First, convert the document (equivalent to the second document in the above embodiment) from pdf, docx and other formats to txt format. Then, construct a stop word dictionary, and perform word segmentation on the text content extracted from each element block. Use the stop word dictionary to filter during the word segmentation process to remove stop words that are not related to the text topic. Use regular expressions to construct special symbol representations, match special characters through regular expressions and clean them up. The data format of the text data after this step of data cleaning (equivalent to the second text data in the above embodiment) is in GBK or Unicode format, and is changed to the commonly used utf-8 format after the format conversion tool. Finally, use regular expressions to split the original text D into N sentences (ie, the first document), that is, D = {S1, S2…, S N}.

[0067] Step 2: BERT-based precoding. The precoding part consists of a pre-trained encoder BERT model. In document D (i.e., the first document), each sentence consists of a sequence of characters, represented as m is the maximum sentence length. First, the input document is preprocessed by inserting separators "[CLS]" and "[SEP]" at the beginning and end of each sentence to obtain a character sequence that can be directly encoded by BERT (equivalent to the second character sequence in the above embodiment). For each sentence S i , which inputs BERT’s representation X i As shown in formula (1), C i is the word embedding, P i is the position embedding, E iis a segmented embedding. The BERT model is composed of multiple encoder modules. Each encoder module mainly includes a multi-head attention and a feedforward neural network module, and residual addition and regularization operations are performed after each module. The calculation of the multi-head attention module is shown in formula (2) (3), where head h is the result of the h-th attention head, W O is the weight matrix of the output layer, is the output of the k-th layer encoder. The output of the feedforward neural network is shown in formula (4), where FFN is the feedforward neural network and ReLU is the activation function. The final output embedding vector calculation formula is formula (5), where Z i (equivalent to the first sentence embedding vector in the above embodiment) is sentence S i is the embedding vector of , and T is the total number of encoder modules.

[0068] X i =[C i ;P i ; E i ] (1)

[0069] MultiHeadAttention (Q,K,V)= Concat ( head1,…, head h )W O (2)

[0070]

[0071] Step 3: Sentence embedding based on dimension-level attention. To address the problem of sentence-level extractive summarization being biased towards sentence length, an optional embodiment of the present application uses a sentence embedding algorithm based on dimension-level attention (DSE). This method uses a stack of multiple dimension-level attention modules (DimAttentionLayer) to form a neural network for text embedding. The structure of each dimension-level attention module is as follows: Figure 4 The sentence embedding calculation formula of each layer is shown in formula (6), where the jth layer DimAttentionLayer j The input is the j-1th layer DimAttentionLayer j-1 When j = 1, the input of the first layer DimAttentionLayer1 is the sentence embedding vector Z calculated in step 2 i The network still uses residual and layer normalization methods to accelerate network convergence, and the sentence S can be calculated. i The final sentence embedding vector is E i(equivalent to the second sentence embedding vector in the above embodiment).

[0072]

[0073] Step 4: Get the key sentence. Get the sentence S i The embedding vector E i After that, the key score Y of the sentence is calculated through the linear layer i , the calculation formula is shown in formula (7), where the linear layer weight matrix is ​​W l , the bias term is b l , the activation function uses the sigmoid function, which is defined as formula (8). Define the key sentence score threshold Y c , if Y i Greater than Y c , then define sentence S i is the key sentence in the document.

[0074] Y i =σ(W l ·E i +b l ) (7)

[0075]

[0076] Step 5: Build a large model and input prompt words (equivalent to the combined data in the above embodiment). Set a prompt word template, including the key sentences obtained in step 4, as well as the document title, subject and other information, combined with appropriate connecting words to form prompt words.

[0077] Step 6: Generate document summary. Finally, the prompt words constructed in step 5 are input into the large model to generate a document summary with good readability.

[0078] In summary, the optional embodiments of the present application significantly improve the efficiency and quality of document summary generation by combining sentence importance screening with large model semantic understanding. Not only does it reduce the amount of text input into the large model and speed up the reasoning speed, but it also uses the deep understanding ability of the large model to ensure the readability and accuracy of the summary content. At the same time, the optional embodiments of the present application effectively solve the problem of excess computing resources when processing long documents and optimize resource utilization efficiency. By constructing reasonable input prompt words, the comprehensiveness of the summary is also guaranteed, so that the generated summary can fully reflect the main purpose of the document. In addition, the present method has a wide range of applicability and can adapt to documents of different types and lengths to meet diverse user needs.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0080] The present application also provides a structural block diagram of a device for generating a document summary. Figure 5 is a structural block diagram of a device for generating a document summary according to an embodiment of the present application; Figure 5 As shown, including:

[0081] A screening module 52, configured to screen out key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm;

[0082] A combining module 54 is used to combine the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document;

[0083] The generating module 56 is used to input the combined data into a large model to generate a document summary of the first document.

[0084] Through the above device, the key sentences in the first document are screened out from the first document through the pre-trained BERT model and sentence embedding algorithm; the document information of the first document is combined with the key sentences through a preset prompt word template to obtain combined data, wherein the document information at least includes: the document title of the first document; the combined data is input into the large model to generate a document summary of the first document. Through the above embodiment, the problem of poor extraction effect of the document summary extraction scheme for long documents in the prior art is solved, and the extraction effect of the document summary is improved.

[0085] In an exemplary embodiment, the screening module 52 is further used to: determine the first sentence embedding vector corresponding to each sentence in the first document through the BERT model; input the first sentence embedding vector into the sentence embedding algorithm to obtain the second sentence embedding vector of each sentence; and determine the key sentence through the second sentence embedding vector.

[0086] In an exemplary embodiment, the screening module 52 is further used to: insert separators at the beginning and end of the first character sequence corresponding to each sentence to obtain a second character sequence corresponding to the first document, wherein the second character sequence can be directly encoded by the BERT model; and input the second character sequence into the BERT model to obtain the first sentence embedding vector.

[0087] In an exemplary embodiment, the screening module 52 is further used to: input the second sentence embedding vector into a linear layer to obtain a criticality score for each sentence; compare the criticality score with a critical sentence score threshold; and determine that each sentence is a critical sentence in the first document when the criticality score is greater than the critical sentence score threshold.

[0088] In an exemplary embodiment, the screening module 52 is further configured to: determine that each sentence is not a key sentence in the first document when the key score is less than the key sentence score threshold; and update each sentence by the next sentence of each sentence.

[0089] In an exemplary embodiment, the device also includes: a splitting module, which is used to filter out key sentences in the first document from the first document through a pre-trained BERT model and a sentence embedding algorithm: performing word segmentation on the acquired second document, and filtering stop words that are not related to the text topic of the first document through a stop word dictionary to obtain first text data; cleaning special characters in the first text data through regular expressions and pre-built special symbol representations to obtain second text data; splitting the second text data into N sentences through the regular expression to obtain the first document, where N is a positive integer.

[0090] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0091] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0092] S1, filtering out key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm;

[0093] S2, combining the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document;

[0094] S3, inputting the combined data into a large model to generate a document summary of the first document.

[0095] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0096] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0097] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0098] S1, filtering out key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm;

[0099] S2, combining the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document;

[0100] S3, inputting the combined data into a large model to generate a document summary of the first document.

[0101] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0102] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0103] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0104] S1, filtering out key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm;

[0105] S2, combining the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document;

[0106] S3, inputting the combined data into a large model to generate a document summary of the first document.

[0107] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0108] An embodiment of the present application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0109] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0111] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating a document summary, characterized in that: include: Filtering key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm; Combining the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document; The combined data is input into a large model to generate a document summary of the first document.

2. The method for generating a document summary according to claim 1, characterized in that: The key sentences in the first document are selected from the first document by using the pre-trained BERT model and sentence embedding algorithm, including: Determine a first sentence embedding vector corresponding to each sentence in the first document by using the BERT model; Inputting the first sentence embedding vector into the sentence embedding algorithm to obtain a second sentence embedding vector for each sentence; The key sentence is determined by using the second sentence embedding vector.

3. The method for generating a document summary according to claim 2, characterized in that: Determining a first sentence embedding vector corresponding to each sentence in the first document by using the BERT model includes: Inserting separators at the beginning and the end of the first character sequence corresponding to each sentence to obtain a second character sequence corresponding to the first document, wherein the second character sequence can be directly encoded by the BERT model; The second character sequence is input into the BERT model to obtain the first sentence embedding vector.

4. The method for generating a document summary according to claim 2, characterized in that: Determining the key sentence by using the second sentence embedding vector includes: Inputting the second sentence embedding vector into the linear layer to obtain the criticality score of each sentence; comparing the criticality score to a critical sentence score threshold; In a case where the keyness score is greater than the key sentence score threshold, each of the sentences is determined to be a key sentence in the first document.

5. The method for generating a document summary according to claim 4, characterized in that: After comparing the criticality score with a critical sentence score threshold, the method further includes: In a case where the keyness score is less than the key sentence score threshold, determining that each of the sentences is not a key sentence in the first document; Each of the sentences is updated by its next sentence.

6. The method for generating a document summary according to claim 1, characterized in that: Before filtering out key sentences in the first document from the first document using the pre-trained BERT model and sentence embedding algorithm, the method further includes: Performing word segmentation processing on the acquired second document, and filtering stop words irrelevant to the text subject of the first document through a stop word dictionary, to obtain first text data; Cleaning special characters in the first text data by using regular expressions and pre-built special symbol representations to obtain second text data; The second text data is split into N sentences using the regular expression to obtain the first document, where N is a positive integer.

7. A device for generating a document summary, characterized in that: include: A screening module, configured to screen out key sentences in the first document from the first document by using a pre-trained BERT model and a sentence embedding algorithm; A combining module, used to combine the document information of the first document with the key sentence through a preset prompt word template to obtain combined data, wherein the document information at least includes: a document title of the first document; A generating module is used for inputting the combined data into a large model to generate a document summary of the first document.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images