Abstract extraction method and apparatus

Abstract

The embodiment of the specification provides an abstract extraction method and device, wherein the abstract extraction method comprises: obtaining a plurality of document texts; splicing the plurality of document texts based on document identifiers to obtain spliced document texts; inputting the spliced document texts into an abstract extraction model to obtain an abstract extraction result, the abstract extraction model processes the spliced document texts based on attention distribution information, the attention distribution information is determined by a first document identifier and text characters in a first document text, a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the plurality of document texts. Based on the attention distribution information determined by the document identifiers and the text characters of the plurality of documents, the spliced document texts are processed to obtain the abstract extraction result, and the relationship between the characters of the plurality of document texts is fully considered in the processing process, thereby improving the accuracy of abstract extraction of the plurality of document texts.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a summary extraction method. BACKGROUND

[0002] Document summarization technology has many applications in business analysis, public opinion monitoring, etc. With more and more public data sets, single-document summarization technology is becoming more and more powerful, but the research on multi-document summarization technology is still very few.

[0003] The current multi-document summarization technology is to select several important sentences from multiple documents respectively, and to sort and reorganize them to form a summary. However, the connection between multiple documents is not considered in the process of generating the summary, which leads to poor accuracy of the summary obtained from multiple documents. Therefore, there is an urgent need for a method to improve the accuracy of multi-document summary extraction. SUMMARY

[0004] Therefore, the embodiments of the present specification provide a summary extraction method. One or more embodiments of the present specification also relate to a summary extraction device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0005] According to a first aspect of the embodiments of the present specification, a summary extraction method is provided, comprising:

[0006] obtaining multiple document texts;

[0007] splicing the multiple document texts based on document identifiers to obtain spliced document texts;

[0008] inputting the spliced document texts into a summary extraction model to obtain a summary extraction result, wherein the summary extraction model processes the spliced document texts based on attention distribution information, the attention distribution information is determined by a first document identifier and text characters in a first document text, a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the multiple document texts.

[0009] According to a second aspect of the embodiments of the present specification, a summary extraction device is provided, comprising:

[0010] an acquisition module configured to obtain multiple document texts;

[0011] a splicing module configured to splice the multiple document texts based on document identifiers to obtain spliced document texts;

[0012] The obtaining module is configured to input the spliced document text into an abstract extraction model to obtain an abstract extraction result, wherein the abstract extraction model processes the spliced document text based on attention distribution information determined by a first document identifier and text characters in a first document text, and a second document identifier in a second document text, the first document text and the second document text being any two of the plurality of document texts.

[0013] According to a third aspect of an embodiment of the present specification, a computing device is provided, comprising:

[0014] a memory and a processor;

[0015] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, which, when executed by the processor, implement the steps of the abstract extraction method.

[0016] According to a fourth aspect of an embodiment of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, which, when executed by a processor, implement the steps of the abstract extraction method.

[0017] According to a fifth aspect of an embodiment of the present specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the abstract extraction method.

[0018] One embodiment of the present specification obtains a plurality of document texts; splices the plurality of document texts based on document identifiers to obtain spliced document texts; inputs the spliced document texts into an abstract extraction model to obtain an abstract extraction result, wherein the abstract extraction model processes the spliced document texts based on attention distribution information determined by a first document identifier and text characters in a first document text, and a second document identifier in a second document text, the first document text and the second document text being any two of the plurality of document texts. Based on the attention distribution information determined by the first document identifier and the text characters in the first document text, and the second document identifier in the second document text, the spliced document texts obtained by splicing the plurality of document texts are processed to obtain the abstract extraction result, so that the relationship between the characters in the plurality of document texts is fully considered in the processing process, and the accuracy of abstract extraction of the plurality of document texts is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is an interactive flow diagram under the abstract extraction system architecture provided by one embodiment of the present specification;

[0020] Figure 2is a framework diagram of an abstract extraction system provided by an embodiment of the present specification;

[0021] Figure 3 is a flow chart of an abstract extraction method provided by an embodiment of the present specification;

[0022] Figure 4 is a process flow chart of an abstract extraction method provided by an embodiment of the present specification;

[0023] Figure 5 is a structural schematic diagram of an abstract extraction method provided by an embodiment of the present specification;

[0024] Figure 6 is a structural schematic diagram of an abstract extraction apparatus provided by an embodiment of the present specification;

[0025] Figure 7 is a structural block diagram of a computing device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0026] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples, set forth in this description. Those skilled in the art, in light of the description, can implement the present specification without limiting to the specific details disclosed in this description.

[0027] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments of the present specification, first can be termed second, and similarly, second can be termed first. The term "if' as used herein, can be interpreted as meaning "when" or "upon" or "in response to determining," depending on the context.

[0029] In addition, it should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or rejection.

[0030] Firstly, the nomenclature involved in one or more embodiments of the present specification is explained.

[0031] Abstractive summarization: Unlike extractive summarization, which directly selects original words and sentences from the input as a summary, abstractive summarization understands and selects words (including words not appearing in the input) to more concisely convey the information of the summarized document.

[0032] Single document summarization (SDS): Taking a document as input, it extracts an output as a summary. It is the most common form of text summarization.

[0033] Multi-document summarization (MDS): Unlike single document summarization, multi-document summarization takes multiple documents as input and extracts an output as a summary.

[0034] Pre-trained language model (PLM): Pre-training of language models using a large amount of unlabeled text allows the machine to find a more general natural language representation. Such models can be fine-tuned using downstream data to better complete downstream tasks.

[0035] Pre-training: Using an unlabeled dataset to train the language model for general natural language capabilities.

[0036] Fine-tuning: Usually using a specific task dataset to adjust the model parameters on a pre-trained model so that the model can better complete the specified task.

[0037] Encoder: A common model encodes the input to better represent the input content with vectors.

[0038] Decoder: A common model decodes the output of the encoder again to finally get the text output.

[0039] Document summarization techniques have many applications in business analysis, public opinion monitoring, etc. With more and more public data sets, single document summarization techniques are becoming more and more powerful, but the research on multi-document summarization techniques is still very few.

[0040] Multi-document summarization is much more difficult than single-document summarization. Such a task has more stringent requirements for the model, but also has more extensive application potential. The difficulty of multi-document summarization lies in the fact that the model needs to take into account the information differences or even conflicts that may occur in multiple documents, handle repeated content in the documents, and extract key points from more verbose information. For example, the same event will have different emphases and wording when reported by newspapers with different opinion tendencies. How to fairly and accurately extract the event process and ignore unimportant details often has very high challenges. For example, how to objectively summarize the pros and cons of a movie according to multiple different reviews also requires artificial intelligence to well handle and analyze the connections in multiple reviews. However, it is precisely because of the complexity of the multi-document summarization task that the application of artificial intelligence to multi-document summarization can greatly save human resources. Multi-document summarization can also be used to extract the views of both sides on the same event, summarize the overall evaluation of multiple users on the same product, etc.

[0041] Current multi-document summarization techniques are methods of selecting several important sentences from multiple documents, sorting and reorganizing them to form a summary, and generating a multi-document summary. However, the connection between multiple documents is not considered in the process of generating the summary, resulting in poor accuracy of the summary obtained from multiple documents.

[0042] Specifically, when performing multi-document summarization extraction, a pre-trained model is used. However, since the training process of the current model does not incorporate the training text and structured information of multiple documents, the pre-trained language model cannot handle multiple document inputs well. However, if the model is redesigned according to the characteristics of multiple documents and trained from scratch, due to the limited amount of data for multi-document summarization (much smaller than the hundreds of GB or even several TB of data for large-scale pre-training models), the new model often does not have as good language generalization ability as the pre-trained model. Therefore, models with special structural design can only achieve good results on tasks with large-scale annotations, and perform poorly when the data volume is insufficient. Our solution can better handle the relationship between multiple documents based on pre-trained language models.

[0043] In the process of multi-document summarization extraction, if a pre-trained model is not used, but a training data set is prepared by oneself, the model is trained, so that the model can perform summarization extraction on multi-documents. In real applications, preparing training data for multi-document summarization is very labor-intensive, and the task of multi-document summarization may not have enough data in some specific fields. Therefore, such a model has not been pre-trained with a large amount of data, and does not have the language generalization ability obtained after the pre-training of the pre-trained language model. Therefore, it can only have good results in a single data field, such as pre-training a model with a news data set to perform summarization extraction on multi-documents. However, the data set is still far from the pre-training data volume of the pre-trained language model (compared with hundreds of GB or several TB), and because the data comes from the news field, the model's performance in other fields (such as the academic literature field) becomes worse. Compared with directly fine-tuning an existing pre-trained model, re-pre-training a multi-document summarization model is also time-consuming and labor-intensive.

[0044] However, in the method of fine-tuning using pre-trained model parameters, many models tend to focus only on how to handle the long information of multi-document text, but cannot handle the connection between each document well, resulting in poor accuracy of the summary obtained from the multi-document,

[0045] For example, one implementation can be that when performing summarization extraction on multi-document text, the text vocabulary in each document text can focus on the text vocabulary in each document text, which can cause the attention mechanism to be unable to distinguish the boundaries between document texts, and increase the difficulty for the model to understand the connection between different document texts. Another implementation can be that the vocabulary in the document text is divided into global attention vocabulary and local attention vocabulary, where the local attention vocabulary pays attention to the n words (for example, n = 512, then each word pays attention to 256 words on the left and 256 words on the right) adjacent to the left and right, and the global attention vocabulary pays attention to all the words. This method limits each local attention vocabulary to learn only local information, while the global vocabulary can grasp global information. However, it still does not clearly construct the boundaries between different document texts.

[0046] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation entrances for users to choose authorization or refusal.

[0047] Therefore, in the present specification, an abstract extraction method is provided, and the present specification also relates to an abstract extraction device, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.

[0048] Referring to Figure 1 , Figure 1 An interaction flow diagram under an abstract extraction system architecture provided by an embodiment of the present specification is shown, as shown in Figure 1 The system includes a server 100 and a client 200.

[0049] The server 100 is configured to obtain a plurality of document texts, splice the plurality of document texts based on document identifiers to obtain spliced document texts, and input the spliced document texts into an abstract extraction model to obtain an abstract extraction result, wherein the abstract extraction model processes the spliced document texts based on attention distribution information determined by a first document identifier and text characters in a first document text and a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the plurality of document texts.

[0050] The client 200 is configured to receive the abstract extraction result.

[0051] According to the scheme of the embodiments of the present specification, a plurality of document texts are obtained, the plurality of document texts are spliced based on document identifiers to obtain spliced document texts, and the spliced document texts are input into an abstract extraction model to obtain an abstract extraction result, wherein the abstract extraction model processes the spliced document texts based on attention distribution information determined by a first document identifier and text characters in a first document text and a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the plurality of document texts. Based on the attention distribution information determined by the first document identifier and the text characters in the first document text and the second document identifier in the second document text, the spliced document texts obtained by splicing the plurality of document texts are processed to obtain the abstract extraction result, so that the relationship between characters in the plurality of document texts is fully considered in the processing process, and the accuracy of abstract extraction of the plurality of document texts is improved.

[0052] Referring to Figure 2 , Figure 2 A framework diagram of an abstract extraction system provided by an embodiment of the present specification is shown, and the system can include a server 100 and a plurality of clients 200. The plurality of clients 200 can establish a communication connection through the server 100, and in an abstract extraction scenario, the server 100 is used to provide an abstract extraction service between the plurality of clients 200, and the plurality of clients 200 can be used as a sending end or a receiving end to realize communication through the server 100.

[0053] The user can interact with the server 100 through the client 200 to receive data sent by other clients 200, or send data to other clients 200, etc. In the abstract extraction scenario, the user can issue an abstract extraction request to the server 100 through the client 200, the server 100 generates an abstract extraction result according to the abstract extraction request, and pushes the abstract extraction result to other clients 200 that establish communication.

[0054] The client 200 and the server 100 establish a connection through a network. The network provides a medium for the communication link between the client 200 and the server 100. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The data transmitted by the client 200 can need to be encoded, transcoded, compressed, etc. before being published to the server 100.

[0055] The client 200 can be a browser, an application (APP), or a web application such as a HyperText Markup Language 5 (H5) application, or a light application (also known as a small program, a lightweight application), or a cloud application, etc. The client 200 can be developed based on the software development kit (SDK) provided by the server for the corresponding service, such as based on the real-time communication (RTC) SDK, etc. The client 200 can be deployed in an electronic device, and needs to rely on the device or some APP in the device, etc. The electronic device can have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0056] The service end 100 can include a server providing various services, for example, a server providing a communication service for multiple clients, for example, a server for background training supporting a model used on a client, for example, a server processing data sent by a client, and the like. It should be noted that the service end 100 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server of a cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, and the like. Basic cloud computing services of artificial intelligence technology, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0057] It should be noted that the abstract extraction method provided in the embodiments of the present specification is generally executed by the service end 100, but in other embodiments of the present specification, the client 200 can also have similar functions as the service end, so as to execute the abstract extraction method provided in the embodiments of the present specification. In other embodiments, the abstract extraction method provided in the embodiments of the present specification can also be executed by the client 200 and the service end 100 together.

[0058] Referring to Figure 3 , Figure 3 A flowchart of an abstract extraction method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0059] Step 302: Obtain multiple document texts.

[0060] The embodiments of the present specification are applied to a client or a service end belonging to an application with abstract extraction, and the following is described by taking the service end as an example.

[0061] When there is a demand for multi-document text abstract extraction, the service end will obtain multiple document texts, which can be input by a user on the front end, or obtained by the service end from a database storing document texts.

[0062] Specifically, the document text refers to a document composed of multiple text characters, wherein the text characters can be Chinese characters, English characters, and the like, and accordingly, the document text can be a Chinese document, an English document, and the like.

[0063] In a possible implementation manner of the present specification, the service end can start a channel for uploading multiple document texts (for example, through Bluetooth, network uploading, and the like) through the user's click, upload multiple document texts by the user, and perform abstract extraction based on the multiple document texts uploaded by the user.

[0064] In another possible implementation manner of the present specification, there are many document texts in the server side, and the user clicks the document texts to be uploaded, and the selected document texts are used as multiple document texts, and the server extracts the summaries based on the multiple document texts.

[0065] Based on the embodiment of the present specification, the summaries of the multiple document texts are extracted, and therefore the multiple document texts are obtained, which are used for subsequent summary extraction based on the multiple document texts.

[0066] Step 304: based on the document identifier, splicing the multiple document texts to obtain a spliced document text.

[0067] For the multiple document text summary extraction, the multiple document texts need to be input into the summary extraction model for processing. In order to facilitate information fusion of the multiple document texts and then perform summary extraction, the multiple document texts need to be spliced to obtain a document text, that is, a spliced document text.

[0068] Specifically, the document identifier refers to a character used to identify each document text, for example, the document identifier can be a <s>indicates.

[0069] Based on the document identifier, the plurality of document texts are spliced to obtain a spliced document text. The implementation manner of splicing the plurality of document texts to obtain the spliced document text is that a document identifier is assigned to each document text as an identifier, so that each document text is updated from an original document text to a document text with the document identifier as the identifier, and the plurality of updated document texts are spliced to obtain the spliced document text. The splicing refers to sequentially connecting the plurality of document texts. For example, the sequential connection can be end-to-end connection.

[0070] Optionally, when the document identifier is one, the document identifier can be a start symbol or a termination symbol. When the document identifier is two, the document identifier can be any combination of a start symbol and a termination symbol.

[0071] After assigning the document identifier to each document text, the document text has a corresponding relationship with the document identifier. For example, the document identifier x is assigned to the document text 1 to form an updated document text. The document identifier x is the document identifier in the document text 1.

[0072] Before splicing the plurality of document texts, the plurality of document texts have no order and can be spliced in any order. The splicing order does not affect the summary extraction result. That is, the summary extraction result is not affected by the splicing order when the plurality of documents are spliced to obtain the spliced document text. For example, the document text A, the document text B, and the document text C are spliced. The splicing order can be the document text A, the document text B, and the document text C. The splicing order can also be the document text B, the document text C, and the document text A. The splicing order can also be the document text B, the document text A, and the document text C, and so on. The summary extraction results obtained by splicing the document texts in the above three examples to obtain the spliced document text and performing subsequent processing are the same.

[0073] Based on the plurality of document texts, the plurality of document texts are spliced to obtain a spliced document text. After that, the summary of the plurality of document texts in the spliced document text can be extracted based on the spliced document text and the summary extraction model.

[0074] Step 306: input the spliced document text into the summary extraction model to obtain a summary extraction result. The summary extraction model processes the spliced document text based on attention distribution information determined by the first document identifier and the text characters in the first document text and the second document identifier in the second document text. The first document text and the second document text are any two document texts in the plurality of document texts.

[0075] After obtaining the spliced document text composed of multiple document texts, a summary extraction model is used to process the spliced document text to obtain the summary extraction model. In the processing process, the summary extraction model fully considers the structural information between the multiple document texts to improve the accuracy of the summary extraction result.

[0076] Specifically, the summary extraction model refers to a model for extracting the summary of the document text, wherein the summary extraction model is obtained by changing the structure of the model based on a pre-trained model. The change process can be fine-tuning the pre-trained model using a pre-prepared data set, and the fine-tuned summary extraction model is generated. The attention distribution information refers to the information that defines the attention mechanism of the attention layer of the encoder in the summary extraction model, and is specifically determined by the first document identifier, the text characters in the first document text, and the document identifier in the second document text. The first document text and the second document text are any two document texts in the multiple document texts. The first document text corresponds to the first document identifier, and the second document text corresponds to the second document identifier. The text character is a character in the document text, such as the document text being this specification, and any one character in the document text is a text character.

[0077] The pre-trained model has strong language ability and generalization ability. The structure of the pre-trained model can be a Transformer (BART, Bidirectional and Auto-Regressive Transformers) that combines context information and self-recurrent characteristics, a text generation deep learning model (GPT, Generative Pre-training Transformer), etc. The Transformer is a natural language processing library.

[0078] The implementation of inputting the spliced document text into the summary extraction model to obtain the summary extraction result can be inputting the spliced document text into the summary extraction model, processing through the network layer in the summary extraction model, and generating the summary extraction model, wherein the network layer at least includes an embedding layer, an encoder, a decoder, and an output layer, and the attention distribution information exists in the encoder.

[0079] In the processing process, the document identifier learns attention based on the text characters in the document text corresponding to the document identifier and the document identifiers corresponding to the other document texts.

[0080] Optionally, the attention distribution information includes first identifier attention distribution information, and the first identifier attention distribution information is determined by the first document identifier, the text characters in the first document text, and the second document identifier in the second document text.

[0081] Optionally, the attention distribution information further comprises first text character attention distribution information, the first text character attention distribution information being determined by the target text character in the first document text and the characters in the first document text.

[0082] The embedding layer can include a processing module for position embedding of words in the document text, which resets the position processing result for each document text in the spliced document text, so that the encoder can learn each document text as an individually learned chapter when processing, rather than a subsequent chapter of the previous document text, i.e., hierarchical learning of each document text.

[0083] In practical applications, there are many ways to input the spliced document text into the summary extraction model to obtain the summary extraction result, which is determined according to the actual situation, and the present specification does not limit it.

[0084] In a possible implementation of the present specification, the spliced document text can pass through the embedding layer, the encoder, and the output layer to obtain the summary extraction result, wherein the encoder processes the spliced document text based on the attention distribution information, and the attention distribution information is determined by the first document identifier and the text characters in the first document text and the second document identifier in the second document text, and the first document text and the second document text are any two document texts in the plurality of document texts.

[0085] In another possible implementation of the present specification, the spliced document text can sequentially pass through each network layer in the summary extraction model to obtain the summary extraction result, i.e., the summary extraction model includes an embedding layer, an encoder, a decoder, and an output layer; the step of inputting the spliced document text into the summary extraction model to obtain the summary extraction result includes the following steps:

[0086] The spliced document text is input into the embedding layer to obtain text features corresponding to each character in the spliced document text, wherein the characters include text characters and document identifiers;

[0087] The text features corresponding to each character in the spliced document text are input into the encoder to obtain encoding features corresponding to each character, wherein the encoder processes the text features based on the attention distribution information;

[0088] The encoding features corresponding to each character are input into the decoder to obtain decoding features corresponding to each character;

[0089] The decoding features corresponding to each character are input into the output layer to obtain the summary extraction result.

[0090] In particular, the embedding layer refers to a network layer for feature extraction of input data, such as mapping input characters into a vector space to obtain text features corresponding to the input characters. The encoder refers to a network layer for encoding and converting input features into a form suitable for transmission, such as inputting text features into the encoder to obtain encoded features. The decoder refers to a network layer for processing input features to obtain output features, such as inputting encoded features into the decoder, processing through the decoder, and obtaining decoded features. The text feature refers to a vector representing the characteristics of the text, such as image features corresponding to an image, and text features corresponding to a document text. The text feature can include position information, which refers to the sequential position of each character in the concatenated document text. The sequential position can be a relative position or an actual position, such as a character position identifier.

[0091] The implementation of inputting the concatenated document text into the embedding layer to obtain text features corresponding to each character in the concatenated document text can be that the embedding layer includes a position embedding unit and a word embedding unit. The concatenated document text is input into the embedding layer, processed by the position embedding unit and the word embedding unit, and the text features corresponding to each character in the concatenated document text are obtained.

[0092] The implementation of inputting the text features corresponding to each character in the concatenated document text into the encoder to obtain the encoded features corresponding to each character is as follows. Based on the attention distribution information in the encoder, each document text in the concatenated document text is processed to obtain document features corresponding to each document text. Based on the document features, the encoded features corresponding to each character in the concatenated document text are obtained. The document feature is a feature corresponding to each document text.

[0093] Optionally, the attention distribution information can be pre-constructed or generated and used in the encoder.

[0094] The implementation of processing the concatenated document text by the encoder to obtain the encoded features corresponding to each character is to process the document text in layers according to the encoding layers included in the encoder. For example, the first encoding layer in the encoder processes the text features corresponding to each character in the concatenated document text to obtain the first encoding sub-feature corresponding to each character. The second encoding layer processes the first encoding sub-feature corresponding to each character in the concatenated document text to obtain the second encoding sub-feature corresponding to each character. The last encoding layer n in the encoder processes the (n-1)th encoding sub-feature corresponding to each character in the concatenated document text to obtain the nth encoding sub-feature corresponding to each character. The nth encoding sub-feature is taken as the encoded feature corresponding to each character.

[0095] In the formula, the processing manner of the encoding layer 1 for obtaining the encoding sub-feature 1 according to the text features corresponding to the characters in the spliced document text is that the document text 1, the document text 2, the document text 3, …, and the document text m in the spliced document text are processed to obtain the encoding sub-feature 1 corresponding to the characters in each document text, the encoding sub-feature 1 corresponding to the characters in each document text is spliced to obtain the encoding sub-feature 1 corresponding to the characters in the spliced document text.

[0096] In the formula, the manner of processing the document text 1 to obtain the encoding sub-feature 1 corresponding to the characters in the document text 1 is that the document identifier in the document text 1 is used to obtain the encoding sub-feature 1 corresponding to the characters in the document text 1. <s1>with the text characters in document text 1 and the document identifiers in other document texts <s2> 、 <s3> 、…、 <sm>, obtaining a document identifier <s1>The corresponding encoding sub-feature 1 is obtained by using the text characters in the document text 1 and the text characters in the document text 1. The corresponding encoding sub-feature 1 of each text character in the document text is determined, and the determination of the encoding sub-feature 1 of the document text 2, the document text 3, and the document text m is the same or similar.

[0097] The implementation of inputting the encoding features corresponding to each character into the decoder to obtain the decoding features corresponding to each character is that the encoding features corresponding to each character are input into the decoder by using the decoding capability in the decoder. In the decoder, attention processing is performed on each character based on the encoding features corresponding to each document identifier in the concatenated document text. Based on the attention processing result, the decoding features corresponding to each character are obtained. For example, the attention processing is performed on each character in the first document text based on the encoding features corresponding to the first document identifier in the first document text. Based on the attention processing result, the decoding features corresponding to each character in the first document text are obtained, wherein the characters in the first document text include the first document identifier and each text character.

[0098] The decoder includes multiple decoding layers, and the decoding layer includes a self-attention layer, a coding-decoding attention layer, and a feedforward layer. Exemplarily, the encoding features corresponding to each character are input into the coding-decoding attention layer. The self-attention processing is performed on the initial reference decoding features by using the self-attention layer to obtain the reference decoding features, and the coding-decoding attention layer processes the encoding features and the reference decoding features.

[0099] The implementation of inputting the decoding features corresponding to each character into the output layer to obtain the summary extraction result is that the decoding features corresponding to each character in the concatenated document text are input into the output layer. The output layer processes the decoding features to obtain the summary extraction result.

[0100] After the concatenated document text is obtained, the concatenated document text is input into the summary extraction model, and each network layer in the summary extraction model is processed to obtain the summary extraction model corresponding to the multiple document texts. The encoder processes the text features based on the attention distribution information, so that the subsequent obtained summary extraction result is determined based on the attention distribution information, and the accuracy of the summary extraction result is ensured.

[0101] Optionally, when the encoder in the summary extraction model processes, the attention unit in the encoder is used to construct the attention distribution information corresponding to the concatenated document text, and the input text features are processed based on the attention distribution information. That is, the encoder includes the attention unit. The above step of inputting the text features corresponding to each character in the concatenated document text into the encoder to obtain the encoding features corresponding to each character includes the following steps:

[0102] constructing attention distribution information corresponding to the spliced document text based on the attention unit, wherein the attention distribution information is determined by the first document identifier and the text characters in the first document text and the second document identifier in the second document text;

[0103] processing the text features corresponding to each character in the spliced document text based on the attention distribution information to obtain the encoding features corresponding to each character.

[0104] Specifically, the attention unit refers to a module in the encoder for attention learning.

[0105] Optionally, reference information of the attention distribution information is constructed in the attention unit, so that the attention unit constructs the attention distribution information.

[0106] The implementation mode of constructing the attention distribution information corresponding to the spliced document text based on the attention unit can be that the attention unit generates the attention distribution information corresponding to the spliced document text based on the spliced document text; or the reference information preset in the attention unit is obtained, and the attention distribution information corresponding to the spliced document text is constructed based on the reference information and the spliced document text, wherein the attention mechanism defined by the constructed attention distribution information is the same as the attention mechanism in the reference information.

[0107] The implementation mode of processing the text features corresponding to each character in the spliced document text based on the attention distribution information is that the attention mechanism included in the attention distribution information is used to make the characters in each document text learn attention respectively, and the encoding features corresponding to each character are obtained.

[0108] Optionally, the attention distribution information includes first identifier attention distribution information; and also includes first text character attention distribution information.

[0109] By applying the scheme of the embodiments of the present specification, the attention unit is included in the encoder, when the text features are input into the encoder, the attention distribution information is constructed by the attention unit, the text features are processed based on the attention distribution information, and the encoding features corresponding to each character are obtained, which are used for subsequent corresponding processing based on the encoding features.

[0110] Optionally, the attention unit of the encoder constructs the attention distribution information by constructing the attention distribution information corresponding to the spliced document text based on the character information of the characters in the spliced document text, so that subsequent processing is based on the attention distribution information, that is, the above step of constructing the attention distribution information corresponding to the spliced document text based on the attention unit includes the following steps:

[0111] obtaining characters corresponding to each document text in the spliced document text, wherein the characters include text characters and document identifiers;

[0112] constructing attention distribution information based on the characters corresponding to each document text and a preset reference attention distribution rule, wherein the preset reference attention distribution rule includes a first identifier attention distribution rule corresponding to a first document identifier, and the first identifier attention distribution rule includes attention calculation of the first document identifier with text characters in the first document text and a second document identifier in a second document text.

[0113] Specifically, the characters refer to characters in the document text, and the characters include document identifiers and text characters. Each document text includes document identifiers and text characters, and the text characters are usually multiple. The preset reference attention distribution rule refers to a rule for limiting the attention mechanism in the encoder, which is set in advance. The preset can be a rule added in the model when the pre-trained model is fine-tuned.

[0114] Optionally, the preset reference attention distribution rule further includes a first text character attention distribution rule corresponding to text characters in the first document text, and the first text character attention distribution rule includes attention calculation of the first text character attention distribution information by a target text character in the first document text with characters in the first document text.

[0115] The implementation of obtaining characters corresponding to each document text in the spliced document text can be to identify text features corresponding to the spliced document text, obtain feature lengths and character types of text features corresponding to each character, and the character types include document identifiers and text characters.

[0116] The implementation of constructing attention distribution information based on the characters corresponding to each document text and the preset reference attention distribution rule is specifically based on the feature lengths and the document identifiers of the characters in each document text, and the attention distribution information corresponding to the spliced document text is constructed according to the preset reference attention distribution rule.

[0117] The implementation of constructing the attention distribution information corresponding to the spliced document text based on the feature lengths and the document identifiers of the characters in each document text according to the preset reference attention distribution rule can be to set the attention distribution rule of each document identifier in the spliced document text according to the first identifier attention distribution rule corresponding to the first document identifier in the preset reference attention distribution rule, and to construct the attention distribution information corresponding to the spliced document text based on the feature lengths of the characters in each document text and the attention distribution rule of each document identifier.

[0118] According to the scheme of the embodiment of the present specification, the characters corresponding to each document text in the spliced document text are obtained, the attention distribution information is constructed based on the characters and the preset reference attention distribution rule, so that the constructed attention distribution information is constructed based on the preset reference attention distribution rule, and the attention distribution information corresponds to the spliced document text. In the encoder, the spliced document text can be processed based on the attention distribution information, and the accuracy of the multi-document text summary extraction is improved.

[0119] Optionally, obtaining the encoding features corresponding to each character requires corresponding processing of each character in the spliced document text, and then splicing the processing results to obtain the encoding features corresponding to the spliced document text, that is, the above step processes the text features corresponding to each character in the spliced document text based on the attention distribution information to obtain the encoding features corresponding to each character, including the following steps:

[0120] processing the text features corresponding to each character in each document text based on the attention distribution information to obtain the encoding features corresponding to each character in each document text;

[0121] splicing the encoding features corresponding to each character in each document text.

[0122] The implementation manner of processing the text features corresponding to each character in each document text based on the attention distribution information to obtain the encoding features corresponding to each character in each document text can be determining the identifier attention distribution information and the text character attention distribution information corresponding to the text character and the document identifier based on the attention distribution information; processing the text features corresponding to each text character in each document text based on the text character attention distribution information to obtain the encoding features corresponding to each text character in each document text; processing the text features corresponding to each document identifier in each document text based on the identifier attention distribution information to obtain the encoding features corresponding to each document identifier in each document text, wherein the identifier attention distribution information includes attention calculation of the first document identifier and the text character in the first document text, and the second document identifier, wherein the first document text and the second document text are any two document texts in the plurality of document texts.

[0123] The implementation manner of splicing the encoding features corresponding to each character in each document text is to splice the encoding features corresponding to each character in each document text according to the positions or sequences of the characters in the spliced document text to obtain the encoding features corresponding to each character in the spliced document text.

[0124] By applying the scheme of the embodiments of the present specification, the text features corresponding to each character in each document text are obtained based on the attention distribution information, the encoding features corresponding to each character in the spliced document text are obtained by splicing, the integrity of the processing result is ensured by the character-by-character processing manner, and further, the accuracy of the abstract extraction is improved by extracting the abstract based on the processing result.

[0125] Optionally, the attention distribution information is also determined by the text characters in the first document text and the characters in the first document text. Based on the attention distribution information, the text features corresponding to each character in each document text are processed based on the attention mechanism specified in the attention distribution information, and the manner of attention learning of the document identifier is limited, that is, the above step of processing the text features corresponding to each character in each document text based on the attention distribution information to obtain the encoding features corresponding to each character in each document text includes the following steps:

[0126] Based on the attention distribution information, the text features of the first document identifier are processed based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text to obtain the encoding features corresponding to the first document identifier.

[0127] Based on the attention distribution information, the text features of the target text character in the first document text are processed based on the text features of the text characters in the first document text to obtain the encoding features corresponding to the target text character, wherein the target text character is any one of the text characters in the first document text.

[0128] Based on the attention distribution information, the text features of the first document identifier are processed based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text to obtain the encoding features corresponding to the first document identifier. The implementation manner is that the first document identifier respectively performs attention calculation based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text, and obtains the encoding features corresponding to the first document identifier based on the attention calculation result.

[0129] Based on the attention distribution information, the first document identifier is processed based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text to perform attention calculation respectively, and based on the attention calculation result, the implementation manner of obtaining the encoding features corresponding to the first document identifier is obtained, that is, the text features of the first document identifier are respectively subjected to attention calculation with the text features of the text characters in the first document text and the text features of the second document identifier, to obtain the first document calculation result and the second document calculation result, and based on the first document calculation result and the second document calculation result, the encoding features corresponding to the first document identifier are obtained.

[0130] Optionally, in addition to obtaining the first document calculation result and the second document calculation result, the first document identifier can also perform self-attention calculation to obtain a third document calculation result, and based on the first document calculation result, the second document calculation result and the third document calculation result, the encoding features corresponding to the first document identifier are obtained.

[0131] The implementation manner of obtaining the encoding features corresponding to the target text character in the first document text based on the text features of the text characters in the first document text is that the text features corresponding to the target text character in the first document text are subjected to attention calculation with the text features of the text characters in the first document text to obtain a plurality of text calculation results, and based on the plurality of text calculation results, the encoding features corresponding to the target text character are obtained.

[0132] Optionally, the target text character can also be subjected to attention calculation with the document identifiers in the first document text to obtain an nth text calculation result, and based on the above plurality of text calculation results and the nth text calculation result, the encoding features corresponding to the target text character are obtained.

[0133] There is no sequence difference between obtaining the encoding features corresponding to each character in the first document text and obtaining the encoding features corresponding to each character in the second document text, and there is also no sequence difference between obtaining the encoding features corresponding to each character in the first document text and obtaining the encoding features corresponding to each character in the second document text.

[0134] According to the scheme of the embodiment of the present specification, the encoding features corresponding to the document identifiers and the text characters are obtained according to the attention distribution information, the encoding features of the first document identifier are determined based on the text features of the characters in the first document text and the text features of the second document identifiers in the second document text, and the encoding features of the target text character are determined based on the text features of the characters in the first document text, so that the document identifiers are determined based on the characters in the document text and the document identifier characters in other document texts, the learning of the structural information in other document texts is improved, and the accuracy of the summary extraction is improved.

[0135] Optionally, the decoder in the abstract extraction model processes the encoding features corresponding to each character in the spliced document text in two dimensions, the first being a document level dimension and the second being a text level dimension. The document level dimension processes the document identifiers, and the text level dimension processes the text characters. That is, the above step inputs the encoding features corresponding to each character into the decoder to obtain decoding features corresponding to each character, including the following steps:

[0136] Based on the encoding features corresponding to each character, initial attention features corresponding to each character are obtained.

[0137] Based on the initial attention features corresponding to the document identifiers of each document text, document level attention features corresponding to each document identifier are obtained. The document level attention features refer to the attention features of the document identifier relative to the plurality of document texts.

[0138] Based on the document level attention features corresponding to the first document identifier, decoding features corresponding to the text characters of the first document text and the first document identifier are obtained.

[0139] In one or more embodiments of the present specification, the document identifiers and text characters of the spliced document text are processed differently in the encoder to obtain respective encoding features. In the decoder, the processing of different characters is also different based on the different processing in the encoder.

[0140] Specifically, the document level attention features refer to the attention features of the document identifier relative to other document identifiers. The document identifier can learn the attention of the vocabulary in other document texts.

[0141] The implementation of obtaining initial attention features corresponding to each character based on the encoding features corresponding to each character can be key feature extraction on the encoding features corresponding to each character to obtain the initial attention features corresponding to each character.

[0142] The implementation of obtaining document level attention features corresponding to each document identifier based on the initial attention features corresponding to the document identifiers of each document text is that, for the initial attention features corresponding to the document identifiers of each document text, a preset normalization method is used for normalization to obtain the document level attention features corresponding to each document identifier.

[0143] Optionally, there are many kinds of preset normalization methods, such as decimal normalization, clipping normalization, maximum and minimum value normalization, standard deviation normalization, etc. The specific selection of the preset normalization method is determined according to the actual situation, which is not limited in the present specification.

[0144] The implementation manner of obtaining the decoding feature corresponding to each text character based on the document-level attention feature corresponding to the first document identifier is specifically: performing attention processing on the initial attention feature corresponding to each text character in the first document text and the first document identifier based on the document-level attention feature corresponding to the first document identifier, and obtaining the decoding feature corresponding to each text character based on the processing result.

[0145] The implementation manner of obtaining the decoding feature corresponding to each text character based on the document-level attention feature corresponding to the first document identifier is specifically: performing attention processing on the initial attention feature corresponding to each text character in the first document text and the first document identifier based on the document-level attention feature corresponding to the first document identifier, and obtaining the decoding feature corresponding to each text character based on the processing result.

[0146] Optionally, there are many ways of attention processing, which can be processing the initial attention feature corresponding to each character in the first document text based on the document-level attention feature corresponding to the first document identifier to obtain the decoding feature corresponding to each character respectively; or normalizing the initial attention feature corresponding to each character in the first document text to obtain the normalization result corresponding to each character, and processing the normalization result corresponding to each character in the first document text based on the document-level attention feature corresponding to the first document identifier to obtain the decoding feature corresponding to each character respectively.

[0147] The scheme of the embodiment of the present specification obtains the initial attention feature corresponding to each character based on the encoding feature corresponding to each character, and obtains the document-level attention feature corresponding to each document identifier based on the document identifier corresponding to each document text; obtains the decoding feature corresponding to each text character in the first document text and the first document identifier based on the document-level attention feature corresponding to the first document identifier, processes the initial attention feature corresponding to each text character and the document identifier in the document text based on the obtained document-level attention feature, and obtains the corresponding decoding feature, so that the obtained decoding feature includes the document-level information, which is helpful for extracting the document summary and further improves the accuracy of extracting the summary of the document.

[0148] Optionally, after obtaining the document-level attention feature, the text-level attention feature corresponding to the document text can also be obtained, and then the decoding feature corresponding to each character is obtained by processing the document-level attention feature and the text-level attention feature. That is, the above step obtains the decoding feature corresponding to the text character of the first document text and the first document identifier based on the document-level attention feature corresponding to the first document identifier, and includes the following steps:

[0149] Based on the encoding feature corresponding to each character in the first document text, obtain the text-level attention feature corresponding to each character in the first document text, wherein the text-level attention feature refers to the attention feature of the character relative to the document text to which it belongs;

[0150] Based on the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character, obtain the decoding feature corresponding to each character.

[0151] Specifically, the text-level attention feature refers to the attention feature of the text character relative to the characters in the document text, and the text character is the attention of the learned vocabulary in the document text.

[0152] The implementation of obtaining the text-level attention feature corresponding to each character in the first document text based on the initial attention feature corresponding to each character in the first document text can be to normalize the initial attention feature corresponding to each character in the first document text according to a predetermined normalization manner to obtain the text-level attention feature corresponding to each character.

[0153] Optionally, the predetermined normalization manner used in the process of obtaining the document-level attention feature and the text-level attention feature can be the same or different, which is determined according to the actual situation, and this specification does not limit it.

[0154] The implementation of obtaining the decoding feature corresponding to each character based on the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character can be to obtain the weight attention feature corresponding to each character based on the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character; and obtain the decoding feature corresponding to each character based on the weight attention feature corresponding to each character.

[0155] Among them, the implementation of obtaining the weight attention feature corresponding to each character based on the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character is to multiply the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character to obtain the weight attention feature corresponding to each character.

[0156] Exemplarily, the document-level attention feature corresponding to the document text 0 and the document identifier 0 is S0, the document-level attention feature corresponding to the document text 1 and the document identifier 1 is S1, and the document-level attention feature corresponding to the document text 2 and the document identifier 2 is S2; wherein the initial attention features corresponding to the document text 0, the document identifier 0 and the text characters are a(0, 0), a(0, 1), a(0, 2), a(0, 3), a(0, 4), a(0, 5), a(0, 6), and the weight attention features corresponding to the characters in the document text 0 are obtained by multiplying a(0, 0), a(0, 1), a(0, 2), a(0, 3), a(0, 4), a(0, 5), a(0, 6) and S0 respectively.

[0157] The implementation manner of obtaining the decoding feature corresponding to each character based on the weight attention feature corresponding to each character is specifically that the decoding feature corresponding to each character is obtained by processing the weight attention feature corresponding to each character and the encoding feature corresponding to each character, for example, the decoding feature corresponding to each character can be obtained by performing attention calculation on the weight attention feature corresponding to each character, the encoding feature corresponding to each character and the reference decoding feature in the decoder, wherein the reference decoding feature is obtained by performing self-attention calculation on the initial reference decoding feature.

[0158] By applying the scheme of the embodiments of the present specification, the text-level attention feature corresponding to each character in the first document text is obtained based on the initial attention feature corresponding to each character in the first document text; and the decoding feature corresponding to each character is obtained based on the document-level attention feature corresponding to the first document identifier and the text-level attention feature corresponding to each character. By obtaining the document-level attention feature in advance, the text-level attention feature corresponding to each character in the document text is processed, so that each character also obtains content related to the document-level information, which lays a foundation for subsequent abstract extraction and improves the accuracy of abstract extraction.

[0159] Optionally, the embedding layer can process the position and vocabulary of the spliced document text to obtain the text feature, and the embedding layer includes a word embedding unit and a position embedding unit; that is, the above step inputs the spliced document text into the embedding layer to obtain the text feature corresponding to each character in the spliced document text, including the following steps:

[0160] The spliced document text is input into the word embedding unit to obtain the word feature corresponding to each character in the spliced document text.

[0161] The word feature corresponding to each character in the spliced document text is input into the position embedding unit to obtain the text feature corresponding to each character in the spliced document text.

[0162] Specifically, the word embedding unit refers to a processing unit for embedding processing of words in the document text, such as inputting the words into the word embedding unit to obtain the word features corresponding to the words. The position embedding unit refers to a processing unit for position embedding processing of word features in the document text, such as inputting the word features into the position embedding unit to obtain the text features corresponding to the words.

[0163] The implementation manner of inputting the spliced document text into the word embedding unit to obtain the word features corresponding to each character in the spliced document text can be that the word embedding unit maps each character in the spliced document text to a vector space to obtain the word features corresponding to each character.

[0164] The implementation manner of inputting the word features corresponding to each character in the spliced document text into the position embedding unit to obtain the text features corresponding to each character in the spliced document text can be that the position embedding unit performs position embedding on the word features corresponding to each character to obtain the text features with position sequences.

[0165] When the position embedding unit performs processing, each document identifier in the spliced document text is recognized, and the position embedding unit performs position embedding on the word features corresponding to the spliced document text according to the positions of the document identifiers to obtain the text features corresponding to each character.

[0166] By applying the scheme of the embodiments of the present specification, the spliced document text is input into the word embedding unit to obtain the word features corresponding to each character in the spliced document text, and the word features corresponding to each character in the spliced document text are input into the position embedding unit to obtain the text features corresponding to each character in the spliced document text. Through the word embedding unit, the word features corresponding to each character in the spliced document text are obtained, and then based on the position embedding unit, the text features corresponding to each character in the spliced document text are obtained. Through the processing of word embedding and position embedding, the subsequent encoder and decoder can process the spliced document text through the position information corresponding to each character when performing processing, thereby improving the efficiency of processing the spliced document text.

[0167] Optionally, the position embedding unit of the embedding layer in the abstract extraction model can reset the position information of each document text in the spliced document text according to the document identifiers, and then obtain the text features corresponding to the reset position information, that is, the above step of inputting the word features corresponding to each character in the spliced document text into the position embedding unit to obtain the text features corresponding to each character in the spliced document text includes the following steps:

[0168] Obtain the position information corresponding to each character in the spliced document text.

[0169] updating position information of each character in the document text corresponding to each document identifier based on the document identifiers in the spliced document text, to obtain updated position information;

[0170] obtaining text features corresponding to each character in the spliced document text based on the updated position information and word features corresponding to each character in the spliced document text.

[0171] Specifically, the position information refers to the sequential position of each character in the spliced document text, which can be a relative position or an actual position. For example, the actual position can be represented by a position identifier. The updated position information refers to the updated position information obtained based on the document identifiers on the basis of the position information.

[0172] The implementation of obtaining the position information corresponding to each character in the spliced document text can be identifying the position of each character in the spliced document text, obtaining the position identifier to which the position belongs, and taking the position identifier as the position information.

[0173] The implementation of updating the position information of each character in the document text corresponding to each document identifier based on the document identifiers in the spliced document text, to obtain updated position information can be updating the position information of each character in the document text corresponding to each document identifier based on the document identifiers in the spliced document text, splicing the updated position information corresponding to each document text, and obtaining the updated position information corresponding to the spliced document text.

[0174] Updating the position information of each character in the document text corresponding to each document identifier based on the document identifiers in the spliced document text, for example, for the document text 1 including the document identifier 1 and n text characters, based on the document identifier 1, the position information corresponding to the document text 1 is obtained, if the document identifier 1 is the starting symbol, the position information of the document identifier is updated to 0, and the position information of the n text characters corresponding to the document text 1 is sequentially updated to 1 to n, and the processing process of other document texts is similar to that of the document text 1.

[0175] The implementation of obtaining text features corresponding to each character in the spliced document text based on the updated position information and word features corresponding to each character in the spliced document text is that the word features corresponding to each character in the spliced document text are added to the updated position information corresponding to each character, to obtain the text features corresponding to each character.

[0176] According to the scheme of the embodiment of the present specification, the position information corresponding to each character in the spliced document text is obtained; the position information of the characters in the document text corresponding to each document identifier is updated based on each document identifier in the spliced document text, to obtain updated position information; and the text features corresponding to each character in the spliced document text are obtained based on the updated position information and the word features corresponding to each character in the spliced document text. The position information of the spliced document text is updated based on the document identifiers in the spliced document text, to be processed based on the updated position information, so that the encoder can learn the information of the document identifiers corresponding to different document texts when performing attention processing, the learning of the structural information in the spliced document text is improved, and the accuracy of multi-document summarization extraction based on the structural information is further improved.

[0177] In one or more embodiments of the present specification, the already trained model parameters in the pre-trained model are used to better emphasize the connection between multiple documents by structuring the encoding and decoding processes. By quickly fine-tuning the parameters of the existing pre-trained language model on each specific task, the ability of the model to process multiple document text inputs is strengthened to achieve abstract summarization extraction, and good multi-document summarization extraction results are achieved in multiple different data fields, including news, film reviews, and paper reviews. In addition, since the summarization extraction model inherits the language ability learned by the pre-trained model, it can also achieve good results on smaller task data.

[0178] In the embodiment of the present specification, both the relative attention proportion of different characters in a single document text close to the method of the pre-trained model and the different attention mechanisms for different document texts are saved, which can further strengthen the ability of the model to learn the relevance of different document texts.

[0179] Further, based on the embodiment of the present specification, experiments are performed on 9 multi-document summarization data sets, which cover multiple fields such as news, film reviews, technical literature, and paper peer review. The smallest amount of task data is only 2.5K training examples. The addition of the structured decoder based on the structured encoder can further improve the results.

[0180] The following describes the embodiment of the present specification in conjunction with the accompanying Figure 4 The summary extraction method provided in the present specification is further described by taking the application of the summary extraction method in multi-document text summarization extraction as an example. Among them, Figure 4 FIG. 1 shows a process flow diagram of a summary extraction method according to an embodiment of the present specification, which specifically includes the following steps.

[0181] Step 402: Obtain a plurality of document texts.

[0182] Step 404: splice the plurality of document texts based on the document identifiers to obtain a spliced document text.

[0183] Step 406: input the spliced document text into a word embedding unit to obtain word features corresponding to each character in the spliced document text.

[0184] Step 408: input the word features corresponding to each character in the spliced document text into a position embedding unit to obtain text features corresponding to each character in the spliced document text.

[0185] Step 410: construct attention distribution information corresponding to the spliced document text based on an attention unit.

[0186] Step 412: process the text features of the first document identifier based on the attention distribution information, obtain the encoding features corresponding to the first document identifier based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text.

[0187] Step 414: process the text features of the target text character in the first document text based on the attention distribution information, obtain the encoding features corresponding to the target text character based on the text features of the text characters in the first document text, wherein the target text character is any one of the text characters in the first document text.

[0188] Step 416: splice the encoding features corresponding to each character in each document text.

[0189] Step 418: obtain document-level attention features corresponding to each document identifier based on the encoding features corresponding to the document identifiers of each document text.

[0190] Step 420: obtain text-level attention features corresponding to each character in the first document text based on the encoding features corresponding to each character in the first document text.

[0191] Step 422: obtain decoding features corresponding to each character based on the document-level attention features corresponding to the first document identifier and the text-level attention features corresponding to each character.

[0192] It should be noted that the specific implementation modes of steps 402 to 422 are the same as the implementation modes of the summary extraction method provided above, and the embodiments of the present application will not be described again. Figure 3

[0193] ​According to the scheme of the embodiment of the present specification, a plurality of document texts are obtained; based on the document identifiers, the plurality of document texts are spliced to obtain spliced document texts; the spliced document texts are input into an abstract extraction model to obtain an abstract extraction result, the abstract extraction model processes the spliced document texts based on attention distribution information, the attention distribution information is determined by a first document identifier and text characters in a first document text, a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the plurality of document texts. Based on the attention distribution information determined by the first document identifier and the text characters in the first document text, the second document identifier in the second document text, the spliced document texts obtained by splicing the plurality of document texts are processed to obtain the abstract extraction result, so that the relationship between the characters in the plurality of document texts is fully considered in the processing process, and the accuracy of abstract extraction of the plurality of document texts is improved.

[0194] Referring to Figure 5 , Figure 5 A structural schematic diagram of an abstract extraction method provided by an embodiment of the present specification is shown.

[0195] Embedding layer: obtain document text 0, document text 1 and document text 2, based on document identifiers <s>, splice the document text 0, the document text 1 and the document text 2 to obtain a spliced document text, input the spliced document text into the word embedding unit to obtain the word features corresponding to each character, for example, the word features corresponding to the document text 0 include <s>, X(0, 1), X(0, 2), X(0, 3),...; the word features corresponding to the document text 1 are <s>, X(1,1), X(1,2), X(1,3),... ; the word features corresponding to the document text 2 are <s>X(2,1), X(2,2), X(2,3), …; the position embedding unit obtains the position information corresponding to the document text 0, the document text 1 and the document text 2 as 0, 1, 2, 3, …, and obtains the text features corresponding to each character based on the processing of the position embedding unit and the word embedding unit, for example, adding the processing results corresponding to the word embedding unit and the position embedding unit, wherein the position information of the document identifier is 0, resetting the position information of the document text corresponding to the document identifier from the document identifier, and obtaining the updated position information corresponding to each document text;

[0196] The encoder: input the text features corresponding to each character in the spliced document text into the encoder, and obtain the encoding features corresponding to each character after the corresponding processing of the encoder, as follows:

[0197] Wherein, the gray block corresponds to the text features corresponding to the document identifier, and the black block corresponds to the text features corresponding to the text character. The document identifier in the document text 0 is <s>corresponding text features are associated with the document identifiers in document text 1, document text 2, respectively <s>corresponding text features and document identifier in document text 0 <s>corresponding text features and text characters corresponding text features are calculated for attention, generating a document identifier in document text 0 <s>encoding features; text characters 1 in document text 0 with text characters in document text 0 and document identifier <s>The attention calculation is performed to obtain the encoding features corresponding to the text character 1; based on the calculation mode of the text character 1, the encoding features of any text character in the document text 0 are obtained; based on the calculation mode of the document text 0, the text features corresponding to each character in the document text 1 and the document text 2 are calculated to obtain the encoding features corresponding to each character;

[0198] The decoder: input the encoding features corresponding to each character in the spliced document text into the decoder, and perform corresponding processing in the decoder to obtain the decoding features corresponding to each character in the spliced document text, wherein the decoder includes n decoding layers, and the specific process is as follows:

[0199] 1. Based on the encoding features corresponding to each character in each document text in the spliced document text, the initial attention features corresponding to each character are obtained;

[0200] 2. Based on the initial attention features a(0, 0) corresponding to the document identifier of the document text 0, the initial attention features a(1, 0) corresponding to the document identifier of the document text 1, and the initial attention features a(2, 0) corresponding to the document identifier of the document text 2, normalization processing is performed to obtain the document-level attention features S(0) corresponding to the document text 0, the document-level attention features S(1) corresponding to the document text 1, and the document-level attention features S(2) corresponding to the document text 2, respectively;

[0201] 3. Based on the initial attention features a(0, 0), a(0, 1), a(0, 2), a(0, 3), … of the characters in the document text 0, the initial attention features a(1, 0), a(1, 1), a(1, 2), a(1, 3), … of the characters in the document text 1, and the initial attention features a(2, 0), a(2, 1), a(2, 2), a(2, 3), … of the characters in the document text 2, normalization processing is performed to obtain the text-level attention features corresponding to each character in the document text 0, the document text 1 and the document text 2;

[0202] 4. The document-level attention features of the document text 0 are multiplied (*) with the text-level attention features of the document text 0 respectively to obtain the weight attention features corresponding to each character in the document text 0, such as w(0, 0), w(0, 1), w(0, 2), w(0, 3), …; the document-level attention features of the document text 1 are multiplied with the text-level attention features respectively to obtain the decoding features corresponding to each character in the document text 1, such as w(1, 0), w(1, 1), w(1, 2), w(1, 3), …; the document-level attention features of the document text 2 are multiplied with the text-level attention features respectively to obtain the decoding features corresponding to each character in the document text 2, such as w(2, 0), w(2, 1), w(2, 2), w(2, 3), …;

[0203] 5. Attention calculation is performed based on the weight attention features corresponding to each character and the reference decoding features to obtain the output result of the decoding layer 1, i.e., the decoding sub-feature 1 corresponding to each character, wherein the reference decoding features are obtained by self-attention calculation on the initial reference decoding features;

[0204] Step 5 is repeated until the output of the decoding layer n, i.e., the decoding sub-feature n corresponding to each character, is obtained, and the decoding sub-feature n corresponding to each character is taken as the decoding feature corresponding to each character.

[0205] Output layer: the decoding features corresponding to each character in the spliced document text are input into the output layer to obtain the summary extraction result output by the output layer: y(0), y(1), y(2), ….

[0206] According to the scheme of the embodiment of the present specification, a plurality of document texts are obtained; based on the document identifiers, the plurality of document texts are spliced to obtain a spliced document text; the spliced document text is input into a summary extraction model to obtain a summary extraction result, the summary extraction model processes the spliced document text based on attention distribution information, the attention distribution information is determined by the first document identifier and the text characters in the first document text, the second document identifier in the second document text, and the first document text and the second document text are any two document texts in the plurality of document texts. Based on the attention distribution information determined by the first document identifier and the text characters in the first document text, the second document identifier in the second document text, the spliced document text obtained by splicing the plurality of document texts is processed to obtain the summary extraction result, so that the relationship between the characters in the plurality of document texts is fully considered in the processing process, and the accuracy of the summary extraction of the multi-document text is improved.

[0207] Corresponding to the method embodiments described above, the present specification also provides summary extraction device embodiments, Figure 6 A structural schematic diagram of a summary extraction device provided by an embodiment of the present specification is shown. As shown in the figure, Figure 6 The device comprises:

[0208] The obtaining module 602 is configured to obtain a plurality of document texts;

[0209] The splicing module 604 is configured to splice the plurality of document texts based on the document identifiers to obtain a spliced document text;

[0210] The obtaining module 606 is configured to input the spliced document text into an abstract extraction model to obtain an abstract extraction result, wherein the abstract extraction model processes the spliced document text based on attention distribution information determined by a first document identifier and text characters in the first document text and a second document identifier in the second document text, and the first document text and the second document text are any two document texts in the plurality of document texts.

[0211] Optionally, the abstract extraction model includes an embedding layer, an encoder, a decoder, and an output layer; and the obtaining module 606 is further configured to input the spliced document text into the embedding layer to obtain text features corresponding to each character in the spliced document text, wherein the characters include text characters and document identifiers; input the text features corresponding to each character in the spliced document text into the encoder to obtain encoding features corresponding to each character, wherein the encoder processes the text features based on attention distribution information; input the encoding features corresponding to each character into the decoder to obtain decoding features corresponding to each character; and input the decoding features corresponding to each character into the output layer to obtain the abstract extraction result.

[0212] Optionally, the encoder includes an attention unit; and the obtaining module 606 is further configured to construct attention distribution information corresponding to the spliced document text based on the attention unit, wherein the attention distribution information is determined by a first document identifier and text characters in the first document text and a second document identifier in the second document text; and process the text features corresponding to each character in the spliced document text based on the attention distribution information to obtain encoding features corresponding to each character.

[0213] Optionally, the obtaining module 606 is further configured to obtain characters corresponding to each document text in the spliced document text, wherein the characters include text characters and document identifiers; and construct the attention distribution information based on the characters corresponding to each document text and a preset reference attention distribution rule, wherein the preset reference attention distribution rule includes a first identifier attention distribution rule corresponding to the first document identifier, and the first identifier attention distribution rule includes attention calculation of the first document identifier and text characters in the first document text and a second document identifier in the second document text.

[0214] Optionally, the obtaining module 606 is further configured to process text features corresponding to each character in each document text based on the attention distribution information to obtain encoding features corresponding to each character in each document text; and splice the encoding features corresponding to each character in each document text.

[0215] Optionally, the attention distribution information is further determined by the text characters in the first document text and the characters in the first document text; the obtaining module 606 is further configured to process the text features of the first document identifier based on the attention distribution information, obtain the encoding features corresponding to the first document identifier based on the text features of the text characters in the first document text and the text features of the second document identifier in the second document text; and process the text features of the target text character in the first document text based on the attention distribution information, obtain the encoding features corresponding to the target text character based on the text features of the text characters in the first document text, wherein the target text character is any one of the text characters in the first document text.

[0216] Optionally, the obtaining module 606 is further configured to obtain initial attention features corresponding to each character based on the encoding features corresponding to each character; obtain document-level attention features corresponding to each document identifier based on the initial attention features corresponding to the document identifiers in the document texts; and obtain decoding features corresponding to each character in the first document text and the first document identifier based on the document-level attention features corresponding to the first document identifier.

[0217] Optionally, the obtaining module 606 is further configured to obtain text-level attention features corresponding to each character in the first document text based on the initial attention features corresponding to each character in the first document text, wherein the text-level attention features refer to the attention features of the character with respect to the document text to which the character belongs; and obtain decoding features corresponding to each character based on the document-level attention features corresponding to the first document identifier and the text-level attention features corresponding to each character.

[0218] Optionally, the embedding layer includes a word embedding unit and a position embedding unit; the obtaining module 606 is further configured to input the spliced document text to the word embedding unit to obtain word features corresponding to each character in the spliced document text; and input the word features corresponding to each character in the spliced document text to the position embedding unit to obtain text features corresponding to each character in the spliced document text.

[0219] Optionally, the obtaining module 606 is further configured to obtain position information corresponding to each character in the spliced document text; update the position information of the characters in the document text corresponding to each document identifier based on each document identifier in the spliced document text to obtain updated position information; and obtain text features corresponding to each character in the spliced document text based on the updated position information and the word features corresponding to each character in the spliced document text.

[0220] According to the scheme of the embodiment of the present specification, a plurality of document texts are obtained; the plurality of document texts are spliced based on document identifiers to obtain spliced document texts; and the spliced document texts are input into an abstract extraction model to obtain an abstract extraction result. The abstract extraction model processes the spliced document texts based on attention distribution information determined by a first document identifier and text characters in a first document text and a second document identifier in a second document text, the first document text and the second document text being any two of the plurality of document texts. The spliced document texts obtained by splicing the plurality of document texts are processed based on the attention distribution information determined by the first document identifier and the text characters in the first document text and the second document identifier in the second document text to obtain the abstract extraction result, so that the relationship between the characters in the plurality of document texts is fully considered in the processing process, and the accuracy of abstract extraction of the plurality of document texts is improved.

[0221] The above is a schematic scheme of the abstract extraction device of the embodiment. It should be noted that the technical scheme of the abstract extraction device belongs to the same concept as the technical scheme of the abstract extraction method described above, and the details of the technical scheme of the abstract extraction device that are not described in detail can be referred to the description of the technical scheme of the abstract extraction method.

[0222] Figure 7 A structural block diagram of a computing device provided by an embodiment of the present specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 through a bus 730, and a database 750 is used to save data.

[0223] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or combinations of such networks, such as the Internet. The access device 740 can include one or more of any type of network interface (for example, a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC).

[0224] In one embodiment of the present specification, the above-mentioned components of the computing device 700 and other components not shown in the Figure 7 may be connected to each other, such as through a bus. It should be understood that Figure 7 The computing device structure diagram shown is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced by those skilled in the art as needed.

[0225] The computing device 700 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (for example, a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, and the like), a mobile phone (for example, a smartphone), a wearable computing device (for example, a smart watch, smart glasses, and the like), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 can also be a mobile or stationary server.

[0226] The processor 720 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned abstract extraction method.

[0227] The above is a schematic solution of the computing device of the embodiment. It should be noted that the technical solution of the computing device and the technical solution of the abstract extraction method belong to the same concept, and the details of the technical solution of the computing device that are not described in detail can be referred to the description of the technical solution of the abstract extraction method.

[0228] An embodiment of the present specification further provides a computer readable storage medium, which stores computer executable instructions. The computer executable instructions are executed by a processor to implement the steps of the abstract extraction method.

[0229] The above is a schematic solution of the computer readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the abstract extraction method belong to the same concept, and the details of the technical solution of the storage medium that are not described in detail can be referred to the description of the technical solution of the abstract extraction method.

[0230] An embodiment of the present specification further provides a computer program, which causes a computer to execute the steps of the abstract extraction method when the computer program is executed in the computer.

[0231] The above is a schematic solution of the computer program of the embodiment. It should be noted that the technical solution of the computer program and the technical solution of the abstract extraction method belong to the same concept, and the details of the technical solution of the computer program that are not described in detail can be referred to the description of the technical solution of the abstract extraction method.

[0232] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps in a claim can be performed in an order other than the order in which the acts or steps are recited, and still accomplish the desired result. In addition, the process depicted in the accompanying figures does not necessarily require the particular order shown, or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0233] The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or deletions according to the requirements of patent practice, for example, according to the patent practice in some regions, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0234] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the present specification are not limited by the order of the described actions, because according to the embodiments of the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of the present specification.

[0235] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0236] The preferred embodiments of the present specification disclosed above are only used to help explain the present specification. The alternative embodiments do not describe all the details and limit the invention to the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and use the present specification. The present specification is limited by the claims and their entire scope and equivalents.< / s> < / s> < / s> < / s> < / s> < / s> < / s> < / s> < / s> < / sm> < / s3> < / s2> < / s>

Claims

1. An abstract extraction method, comprising: obtaining a plurality of document texts; splicing the plurality of document texts based on document identifiers to obtain spliced document texts; inputting the spliced document texts into an embedding layer of an abstract extraction model to obtain text features corresponding to each character in the spliced document texts, wherein the abstract extraction model comprises the embedding layer, an encoder, a decoder, and an output layer, and the characters include text characters and document identifiers; inputting the text features corresponding to each character in the spliced document texts into the encoder, and constructing attention distribution information corresponding to the spliced document texts based on an attention unit of the encoder, wherein the attention distribution information is determined by a first document identifier and text characters in a first document text, and a second document identifier in a second document text, and the first document text and the second document text are any two document texts in the plurality of document texts; processing the text features corresponding to each character in the spliced document texts based on the attention distribution information to obtain encoding features corresponding to each character; inputting the encoding features corresponding to each character into the decoder to obtain decoding features corresponding to each character; inputting the decoding features corresponding to each character into the output layer to obtain an abstract extraction result.

2. The method of claim 1, wherein constructing the attention distribution information corresponding to the spliced document texts based on the attention unit comprises: obtaining characters corresponding to each document text in the spliced document texts, wherein the characters include text characters and document identifiers; constructing attention distribution information based on the characters corresponding to each document text and a preset reference attention distribution rule, wherein the preset reference attention distribution rule comprises a first identifier attention distribution rule corresponding to a first document identifier, and the first identifier attention distribution rule comprises attention calculation of the first document identifier and text characters in a first document text, and a second document identifier in a second document text.

3. The method of claim 1, wherein processing the text features corresponding to each character in the spliced document texts based on the attention distribution information to obtain encoding features corresponding to each character comprises: processing the text features corresponding to each character in each document text based on the attention distribution information to obtain encoding features corresponding to each character in each document text; splicing the encoding features corresponding to each character in each document text.

4. The method of claim 3, wherein the attention distribution information is further determined by text characters in the first document text and characters in the first document text; and wherein processing the text features corresponding to each character in each document text based on the attention distribution information to obtain encoding features corresponding to each character in each document text comprises: processing text features of a first document identifier based on text features of text characters in a first document text and text features of a second document identifier in a second document text to obtain encoding features corresponding to the first document identifier based on the attention distribution information. ​ Based on the attention distribution information, the text features of the target text character in the first document text are processed. Based on the text features of the text characters in the first document text, the encoding features corresponding to the target text character are obtained, wherein the target text character is any text character in the first document text.

5. The method as described in claim 1, wherein the encoded features corresponding to each character are input to the decoder to obtain the decoded features corresponding to each character, comprising: Based on the encoding features corresponding to each character, the initial attention features corresponding to each character are obtained; Based on the initial attention features corresponding to the document identifiers in each document text, document-level attention features corresponding to each document identifier are obtained, wherein the document-level attention features refer to the attention features of the document identifiers relative to multiple document texts. Based on the document-level attention features corresponding to the first document identifier, the decoding features of the text characters in the first document text and the first document identifier are obtained.

6. The method as described in claim 5, wherein the decoding features corresponding to the text characters in the first document text and the first document identifier are obtained based on the document-level attention features corresponding to the first document identifier, comprising: Based on the initial attention features corresponding to each character in the first document text, text-level attention features corresponding to each character in the first document text are obtained, wherein the text-level attention features refer to the attention features of the character relative to the document text to which it belongs; Based on the document-level attention features corresponding to the first document identifier and the text-level attention features corresponding to each character, the decoding features corresponding to each character are obtained.

7. The method of claim 1, wherein the embedding layer comprises a word embedding unit and a position embedding unit; The concatenated document text is input into the embedding layer to obtain the text features corresponding to each character in the concatenated document text, including: The concatenated document text is input into the word embedding unit to obtain the word features corresponding to each character in the concatenated document text; The word features corresponding to each character in the concatenated document text are input into the position embedding unit to obtain the text features corresponding to each character in the concatenated document text.

8. The method as described in claim 7, wherein the word features corresponding to each character in the concatenated document text are input to the position embedding unit to obtain the text features corresponding to each character in the concatenated document text, comprising: Obtain the position information corresponding to each character in the concatenated document text; Based on each document identifier in the concatenated document text, update the position information of the characters in the document text corresponding to each document identifier to obtain the updated position information; Based on the updated position information and the word features corresponding to each character in the concatenated document text, the text features corresponding to each character in the concatenated document text are obtained.

9. A summary extraction device, comprising: The acquisition module is configured to acquire text from multiple documents. The splicing module is configured to splice the multiple document texts based on document identifiers to obtain spliced ​​document text. The module is configured as follows: The concatenated document text is input into the embedding layer of the summary extraction model to obtain the text features corresponding to each character in the concatenated document text. The summary extraction model includes the embedding layer, encoder, decoder and output layer. The characters include text characters and document identifiers. The text features corresponding to each character in the concatenated document text are input into the encoder. Based on the attention unit of the encoder, attention distribution information corresponding to the concatenated document text is constructed. The attention distribution information is determined by the first document identifier and the text characters in the first document text, and the second document identifier in the second document text. The first document text and the second document text are any two document texts among the plurality of document texts. Based on the attention distribution information, the text features corresponding to each character in the concatenated document text are processed to obtain the encoding features corresponding to each character; The encoded features corresponding to each character are input into the decoder to obtain the decoded features corresponding to each character; The decoding features corresponding to each character are input into the output layer to obtain the summary extraction result.

10. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images