Method, device, equipment and storage medium for generating abstract

Abstract

The embodiments of the present application belong to the field of artificial intelligence and are applied to generating text summaries of supervision documents. They relate to a method, apparatus, device, and storage medium for generating summaries. The method comprises: obtaining text data of a text; encoding paragraph data and paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; extracting backbone information and entity information from the paragraph data to obtain a backbone vector and an entity vector; fusing the paragraph vector and the backbone vector to obtain a first fused vector; calculating a cross-attention vector based on the first fused vector and the paragraph summary vector; fusing the cross-attention vector and the entity vector to obtain a second fused vector; and decoding the second fused vector to obtain a summary of the text. This method can improve the quality of generated text summaries.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for generating an abstract. Background Art

[0002] Currently, the main methods for generating summaries include extractive summarization and generative summaries. Extractive summaries usually sort all the sentences in a text by importance, extract the sentences ranked before a threshold, and reassemble them into a paragraph as a summary of the original text. However, extractive summaries often lack a higher-level semantic summary of the entire document content, resulting in semantic incoherence in the generated text summary. Generative summaries use computers combined with neural network models to imitate human thinking and extract text summaries as a summary of the original text. However, in actual application, generative summaries suffer from significant information loss and even tampering of key information. Therefore, the quality of text summaries generated by current text summary generation methods is poor. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method, apparatus, device and storage medium for generating a summary to solve the problem of poor quality of the generated text summary.

[0004] To solve the above technical problems, the present invention provides a method for generating a summary, which adopts the following technical solution:

[0005] Acquire text data of a text; the text data includes paragraph data and paragraph summary data, one paragraph data corresponds to one paragraph summary data; encode the paragraph data and the paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; extract backbone information and entity information from the paragraph data to obtain a backbone vector and an entity vector; the backbone information at least includes data corresponding to a subject-predicate-object structure or a subject-predicate-object-attributive-adverb-complement structure in the paragraph corresponding to the paragraph data; the entity information at least includes quantifiers and nouns in the paragraph corresponding to the paragraph data; fuse the paragraph vector and the backbone vector to obtain a first fused vector; calculate a cross-attention vector based on the first fused vector and the paragraph summary vector; fuse the cross-attention vector and the entity vector to obtain a second fused vector; decode the second fused vector to obtain a summary of the text.

[0006] Furthermore, the fusing of the paragraph vector and the backbone vector to obtain a first fused vector specifically includes: performing dimensionality conversion processing on the paragraph vector and the backbone vector respectively to obtain a paragraph conversion vector and a backbone conversion vector; the paragraph conversion vector is a conversion vector of the paragraph data; the backbone conversion vector is a conversion vector of the backbone information; based on the paragraph conversion vector, a self-attention vector of the paragraph data is calculated; and the self-attention vector and the backbone conversion vector are fused to obtain a first fused vector.

[0007] Furthermore, the fusion of the self-attention vector and the backbone transformation vector to obtain a first fusion vector specifically includes: according to the formula The self-attention vector and the backbone transformation vector are fused to obtain a first fused vector; wherein V′ is used to represent the first fused vector, is used to represent the self-attention vector, Z′ is used to represent the backbone transformation vector, A parameter matrix for representing the first fused vector, A bias term used to represent the first fused vector.

[0008] Furthermore, the cross-attention vector is calculated based on the first fusion vector and the paragraph summary vector, specifically including: performing dimension conversion processing on the first fusion vector and the paragraph summary vector through a transformer decoder to obtain a cross-transformation vector; and calculating the cross-attention vector based on the cross-transformation vector.

[0009] Furthermore, the fusion of the cross attention vector and the entity vector to obtain a second fusion vector specifically includes: according to the formula The entity vector is dimensionally transformed to obtain an entity transformation vector, where C′ is used to represent the entity transformation vector, i represents the i-th word in the entity information, and when i is an even number, pos(i)=sin(i / 10000 2 ), when i is an odd number, pos(i)=cos(i / 10000 2 ), Used to represent vector product, c i Used to represent the i-th element in the entity vector; fusing the cross attention vector and the entity transformation vector to obtain a second fused vector.

[0010] Furthermore, the fusion of the cross attention vector and the entity transformation vector to obtain a second fusion vector specifically includes: according to the formula Fusing the cross attention vector and the entity transformation vector to obtain a second fused vector; wherein V″′ is used to represent the second fused vector, V″ is used to represent the cross attention vector, and C′ is used to represent the entity transformation vector. A parameter matrix for representing the second fused vector, A bias term used to represent the second fused vector.

[0011] Furthermore, before obtaining the text data of the text, the generation method also includes: obtaining sample data of the historical text; wherein the sample data includes historical paragraph data and historical summary data, and one historical paragraph data corresponds to one historical summary data; inputting the sample data into a preset model for training to obtain a summary generation model; determining the parameter matrix and bias term of the target vector based on the summary generation model; wherein the target vector includes at least the first fusion vector and the second fusion vector.

[0012] In order to solve the above technical problems, the embodiment of the present application further provides a device for generating a summary, which adopts the following technical solution:

[0013] An acquisition module is used to acquire text data of a text; the text data includes paragraph data and paragraph summary data, and one paragraph data corresponds to one paragraph summary data; an encoding module is used to encode the paragraph data and the paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; an extraction module is used to extract the backbone information and entity information in the paragraph data to obtain a backbone vector and an entity vector; the backbone information at least includes data corresponding to a subject-predicate-object structure or a subject-predicate-object-attributive-adverb-complement structure in the paragraph corresponding to the paragraph data; the entity information at least includes quantifiers and nouns in the paragraph corresponding to the paragraph data; a first fusion module is used to fuse the paragraph vector and the backbone vector to obtain a first fusion vector; a calculation module is used to calculate a cross-attention vector based on the first fusion vector and the paragraph summary vector; a second fusion module is used to fuse the cross-attention vector and the entity vector to obtain a second fusion vector; a decoding module is used to decode the second fusion vector to obtain a summary of the text.

[0014] Furthermore, the first fusion module includes a conversion submodule, a calculation submodule and a fusion submodule; the conversion submodule is used to perform dimension conversion processing on the paragraph vector and the trunk vector respectively to obtain a paragraph conversion vector and a trunk conversion vector; the paragraph conversion vector is the conversion vector of the paragraph data; the trunk conversion vector is the conversion vector of the trunk information; the calculation submodule is used to calculate the self-attention vector of the paragraph data based on the paragraph conversion vector; the fusion submodule is used to fuse the self-attention vector and the trunk conversion vector to obtain a first fusion vector.

[0015] Furthermore, the fusion submodule is specifically used to: The self-attention vector and the backbone transformation vector are fused to obtain a first fused vector; wherein V′ is used to represent the first fused vector, is used to represent the self-attention vector, Z′ is used to represent the backbone transformation vector, A parameter matrix for representing the first fused vector, A bias term used to represent the first fused vector.

[0016] Furthermore, the calculation module includes a first sub-module and a second sub-module; the first sub-module is used to perform dimension conversion processing on the first fusion vector and the paragraph summary vector through the transformer decoder to obtain a cross-conversion vector; the second sub-module is used to calculate the cross-attention vector based on the cross-conversion vector.

[0017] Furthermore, the second fusion module includes a third submodule and a fourth submodule; the third submodule is used to calculate the value of the fusion module according to the formula The entity vector is dimensionally transformed to obtain an entity transformation vector, where C′ is used to represent the entity transformation vector, i represents the i-th word in the entity information, and when i is an even number, pos(i)=sin(i / 10000 2 ), when i is an odd number, pos(i)=cos(i / 10000 2 ), Used to represent vector product, c i Used to represent the i-th element in the entity vector; the fourth submodule is used to fuse the cross-attention vector and the entity transformation vector to obtain a second fused vector.

[0018] Furthermore, the fourth submodule is specifically used to: according to the formula Fusing the cross attention vector and the entity transformation vector to obtain a second fused vector; wherein V″′ is used to represent the second fused vector, V″ is used to represent the cross attention vector, and C′ is used to represent the entity transformation vector. A parameter matrix for representing the second fused vector, A bias term used to represent the second fused vector.

[0019] Furthermore, the generation device also includes a sample acquisition module, a training module and a determination module; the sample acquisition module is used to obtain sample data of historical text; wherein, the sample data includes historical paragraph data and historical summary data, and one historical paragraph data corresponds to one historical summary data; the training module is used to input the sample data into a preset model for training to obtain a summary generation model; the determination module is used to determine the parameter matrix and bias term of the target vector according to the summary generation model; wherein, the target vector includes at least the first fusion vector and the second fusion vector.

[0020] In order to solve the above technical problems, an embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for generating a summary are implemented.

[0021] In order to solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for generating a summary are implemented.

[0022] Compared with the prior art, the embodiments of the present application have the following beneficial effects: encoding the text data to obtain a paragraph vector and a paragraph summary vector; extracting the main information and entity information in the paragraph data to obtain a main vector and an entity vector; fusing the paragraph vector and the main vector to obtain a first fusion vector; calculating the cross-attention vector based on the first fusion vector and the paragraph summary vector; fusing the cross-attention vector and the entity vector to obtain a second fusion vector; decoding the second fusion vector to obtain a summary of the text. That is, the present application fuses the main information and entity information of the paragraph data in the process of generating the text summary, and the main information includes the subject-predicate-object structure or the subject-predicate-object-attributive-adverbial-complement structure in the paragraph, and the entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by the present application must include the main structure (subject-predicate-object structure or subject-predicate-object-attributive-adverbial-complement structure), quantifiers and nouns in the text, avoiding the problem of excessive loss of generated text summary information or information tampering, and improving the quality of the generated text summary. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0025] Figure 2 is a flowchart of an embodiment of a method for generating an abstract according to the present application;

[0026] Figure 3 yes Figure 2 Flowchart of an embodiment of step S24;

[0027] Figure 4 yes Figure 2 Flowchart of an embodiment of step S25;

[0028] Figure 5 yes Figure 2 Flowchart of an embodiment of step S26;

[0029] Figure 6 is a schematic structural diagram of an embodiment of a device for generating an abstract according to the present application;

[0030] Figure 7 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0032] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, system architecture 100 may include terminal devices, a network 104, and a server 105. The terminal devices may include a first terminal device 101, a second terminal device 102, and a third terminal device 103. The network 104 is used to provide a medium for communication links between the terminal devices and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0035] Users can use terminal devices to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0036] The terminal device can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptop computers and desktop computers, etc.

[0037] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on a terminal device.

[0038] It should be noted that the summary generation method provided in the embodiment of the present application is generally executed by a server, and accordingly, the summary generation device is generally set in the server.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0040] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating an abstract according to the present application. The method for generating an abstract comprises the following steps:

[0041] Step S21: Acquire text data of the text.

[0042] In this embodiment, the method for generating a summary is executed on the electronic device (eg Figure 1 The server shown in the figure) can communicate with the terminal through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0043] The text data includes paragraph data and paragraph summary data, and one paragraph data corresponds to one paragraph summary data.

[0044] Optionally, the text in this application may be the text corresponding to the supervision document, and the text summary generated based on the text of the supervision document may be the task summary corresponding to each supervision document. The supervision document refers to the cooperation agreement signed between Company A and Company B. The supervision document may involve many subsidiaries or business units, and the specific tasks corresponding to each subsidiary or business unit are the supervision tasks.

[0045] Specifically, the method of obtaining the text data of the text can be to use a pre-trained paragraph division model to divide the paragraphs corresponding to the text to obtain the paragraph data of the text; use a pre-trained paragraph summary extraction model to extract the summary of the paragraph corresponding to the text to obtain the paragraph summary data of the text.

[0046] Step S22: Encode the paragraph data and paragraph summary data to obtain a paragraph vector and a paragraph summary vector.

[0047] Specifically, an encoder is used to encode the paragraph data to obtain a paragraph vector. For example, a paragraph encoder trained with a word2vec model (or bert model) is used to encode the paragraph data to obtain a paragraph vector. For another example, the paragraph vector is Among them, X is used to represent the paragraph vector, R is a real number, k x Used to indicate the length of a paragraph (i.e. the number of words in the paragraph), d m Used to indicate the length of the paragraph vector. m The length of is usually preset. For example, d m The length of is the default value or the value written by the relevant personnel. For example, d m The length is 256, 300, 512, 768 (the paragraph encoder trained by the BERT model corresponds to d m length) etc.

[0048] Similarly, use the encoder to encode the paragraph summary data to obtain the paragraph summary vector. For example, use the paragraph summary encoder trained by the word2vec model (or bert model) to encode the paragraph summary data to obtain the paragraph summary vector. For another example, the paragraph summary vector is Among them, Y is used to represent the paragraph summary vector, R is a real number, k y Used to indicate the length of the paragraph summary (i.e. the number of words in the paragraph summary), d m Used to represent the length of the paragraph summary vector.

[0049] Step S23: extract the main information and entity information from the paragraph data to obtain the main vector and entity vector.

[0050] The main information includes at least the data corresponding to the subject-verb-object structure or subject-verb-object-attributive-adverbial-complement structure in the paragraph data corresponding to the paragraph data; the entity information includes at least the quantifiers and nouns in the paragraph data corresponding to the paragraph data. Optionally, the nouns in the entity information may include organization names (ORG), person names (PER), location names (LOC), common nouns, and other special nouns.

[0051] Specifically, we can first use language processing tools such as Jieba, Han Language Processing (HanLP), Baidu Lexical Analysis of Chinese (LAC), and Language Technology Platform (LTP) to segment and analyze the paragraph to obtain the subject-verb-object structure, subject-verb-object-attributive-adverbial-complement structure, quantifiers, and nouns in the paragraph. For example, using language processing tools to segment and analyze the sentence "Company A actively participates in the construction of City B, introducing advanced concepts and implementation directions for comprehensive urban investment, development, and operation" will yield the subject-verb-object structure of the paragraph as "Company A participates in the construction of City B."

[0052] Afterwards, an encoder is used to encode the extracted backbone information and entity information to obtain a backbone vector and an entity vector. The specific encoding method is the same as the step of encoding the text data in step S22 above, and will not be described in detail here.

[0053] For example, the backbone vector is Among them, Z is used to represent the backbone vector, R is a real number, k z Used to indicate the length of the main information (i.e. the number of words in the main information), d m It is used to represent the length of the backbone vector. For example, the entity vector is C = {c1, c2, ..., c n}, where C is used to represent the entity vector, c n Used to represent the vector after encoding the nth word of entity information.

[0054] Step S24: fuse the paragraph vector and the backbone vector to obtain a first fused vector.

[0055] Specifically, Figure 3 This is a flowchart of fusing paragraph vectors and backbone vectors to obtain the first fusion vector. Figure 3 As shown, the process includes the following steps S241 to S243.

[0056] Step S241 , performing dimension conversion processing on the paragraph vector and the backbone vector respectively to obtain a paragraph conversion vector and a backbone conversion vector.

[0057] The paragraph conversion vector is a conversion vector of paragraph data, the trunk conversion vector is a conversion vector of trunk information, and the paragraph conversion vector includes a first paragraph sub-vector, a second paragraph sub-vector, and a third paragraph sub-vector.

[0058] Specifically, according to the formula Perform dimension conversion on the paragraph vector to obtain the paragraph conversion vector; where Q e Used to represent the first paragraph vector, K e Used to represent the second paragraph vector, V e is used to represent the third paragraph vector, X is used to represent the paragraph vector, The parameter matrix used to represent the first paragraph vector, The parameter matrix used to represent the second paragraph vector, The parameter matrix used to represent the third paragraph vector, Used to represent the bias term of the first paragraph vector, Used to represent the bias term of the second paragraph vector, Used to represent the bias term of the third paragraph vector.

[0059] Optionally, the encoder module of the transformer can be used to perform dimension conversion on the paragraph vector to obtain a paragraph conversion vector.

[0060] According to the formula Perform dimension conversion on the backbone vector to obtain the backbone conversion vector; where Z′ is used to represent the backbone conversion vector, and Z is used to represent the backbone vector. The parameter matrix used to represent the backbone transformation vector, Bias term used to represent the backbone transformation vector.

[0061] Step S242: Calculate the self-attention vector of the paragraph data based on the paragraph conversion vector.

[0062] Specifically, according to the formula Calculate the self-attention vector of the paragraph data; where, The self-attention vector Q used to represent the paragraph data e Used to represent the first paragraph sub-vector in the paragraph transformation vector, K e Used to represent the second paragraph vector in the paragraph transformation vector, V e Used to represent the third paragraph sub-vector in the paragraph transformation vector, d k The dimension used to represent the first paragraph vector.

[0063] Step S243: fuse the self-attention vector and the backbone transformation vector to obtain a first fused vector.

[0064] Specifically, according to the formula The self-attention vector and the backbone transformation vector are fused to obtain the first fused vector; where V′ is used to represent the first fused vector, is used to represent the self-attention vector, Z′ is used to represent the backbone transformation vector, The parameter matrix used to represent the first fused vector, The bias term used to represent the first fused vector.

[0065] In this embodiment, the paragraph vector and the backbone vector are fused. This means that the backbone information of the paragraph data is incorporated into the text summary during the text summary generation process. The backbone information includes the subject-verb-object structure or the subject-verb-object-attributive-adverbial-complement structure within the paragraph. This ensures that the text summary generated by this application includes the main structure of the text, avoiding issues such as excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0066] Step S25: Calculate a cross-attention vector based on the first fusion vector and the paragraph summary vector.

[0067] Specifically, Figure 4 This is a flowchart of calculating the cross attention vector based on the first fusion vector and the paragraph summary vector. Figure 4 As shown, the process includes the following steps S251 to S252.

[0068] In step S251 , the first fusion vector and the paragraph summary vector are dimensionally transformed by the transformer decoder to obtain a cross-transformation vector.

[0069] Specifically, according to the formula Perform dimension conversion on the first fusion vector and the paragraph summary vector to obtain a cross-conversion vector; wherein the cross-conversion vector includes a first cross sub-vector, a second cross sub-vector, and a third cross sub-vector; Qd Used to represent the first cross sub-vector, K d Used to represent the second cross vector, V d is used to represent the third cross sub-vector, Y is used to represent the paragraph summary vector, and V′ is used to represent the first fusion vector. The parameter matrix used to represent the first cross-subvector, The parameter matrix used to represent the second cross-subvector, The parameter matrix used to represent the third crossover vector, The bias term used to represent the first cross sub-vector, The bias term used to represent the second cross sub-vector, A bias term used to represent the third cross vector.

[0070] Step S252: Calculate the cross attention vector based on the cross conversion vector.

[0071] Specifically, according to the formula The cross attention vector is calculated; where V″ is used to represent the cross attention vector, Q d Used to represent the first cross sub-vector, K d Used to represent the second cross vector, V d Used to represent the third cross sub-vector, d k Dimension used to represent the first cross subvector.

[0072] Step S26: fuse the cross-attention vector and the entity vector to obtain a second fused vector.

[0073] Specifically, Figure 5 This is a flowchart of fusing the cross attention vector and the entity vector to obtain the second fusion vector, refer to Figure 5 As shown, it includes the following steps S261-S262.

[0074] Step S261 , performing dimension conversion processing on the entity vector to obtain an entity conversion vector.

[0075] Specifically, according to the formula Perform dimension conversion on the entity vector to obtain the entity conversion vector; where C′ is used to represent the entity conversion vector, i represents the i-th word in the entity information, and when i is an even number, pos(i)=sin(i / 10000 2 ), when i is an odd number, pos(i)=cos(i / 10000 2 ), Used to represent vector product, c i Used to represent the i-th element in the entity vector (i.e., the vector after encoding the i-th word of the entity information).

[0076] Step S262: Fuse the cross-attention vector and the entity transformation vector to obtain a second fused vector.

[0077] Specifically, according to the formula Fuse the cross attention vector and the entity transformation vector to obtain a second fusion vector; where V″′ is used to represent the second fusion vector, V″ is used to represent the cross attention vector, and C′ is used to represent the entity transformation vector. The parameter matrix used to represent the second fused vector, Used to represent the bias term of the second fused vector.

[0078] Step S27: Decode the second fused vector to obtain a summary of the text.

[0079] Specifically, a decoder is used to decode the second fused vector to obtain a summary of the text. For example, a vector decoder trained using a word2vec model (or a BERT model) is used to decode the second fused vector to obtain a summary of the text.

[0080] Optionally, before step S21, the summary generation method further includes the following steps S201 to S203.

[0081] Step S201 , obtaining sample data of historical text; wherein the sample data includes historical paragraph data and historical summary data, and one historical paragraph data corresponds to one historical summary data.

[0082] Specifically, the sample data can be obtained by manually marking the historical paragraphs of the historical text to obtain the historical paragraph data, and extracting the historical summary data of each historical paragraph. Thereafter, the historical paragraphs and the corresponding historical summary data are combined into a set of sample data.

[0083] Step S202: input the sample data into a preset model for training to obtain a summary generation model.

[0084] Specifically, the method of generating a model based on sample data and a preset model training summary is the same as the existing model training method and will not be described in detail here.

[0085] Step S203 : determining the parameter matrix and bias term of the target vector according to the summary generation model.

[0086] The target vector includes at least a first fusion vector and a second fusion vector.

[0087] Specifically, the target vector also includes a first paragraph sub-vector, a second paragraph sub-vector, a third paragraph sub-vector, a trunk conversion vector, a first cross sub-vector, a second cross sub-vector, and a third cross sub-vector.

[0088] In this embodiment, the text data is encoded to obtain a paragraph vector and a paragraph summary vector; the main information and entity information in the paragraph data are extracted to obtain a main vector and an entity vector; the paragraph vector and the main vector are fused to obtain a first fusion vector; the cross-attention vector is calculated based on the first fusion vector and the paragraph summary vector; the cross-attention vector and the entity vector are fused to obtain a second fusion vector; the second fusion vector is decoded to obtain a summary of the text. That is, the present application fuses the main information and entity information of the paragraph data in the process of generating a text summary, and the main information includes the subject-predicate-object structure or the subject-predicate-object-attributive-adverbial-complement structure in the paragraph, and the entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by the present application must include the main structure (subject-predicate-object structure or subject-predicate-object-attributive-adverbial-complement structure), quantifiers and nouns in the text, avoiding the problem of excessive information loss in the generated text summary or information tampering, thereby improving the quality of the generated text summary.

[0089] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0090] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0091] Further references Figure 6 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a device for generating a summary. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0092] like Figure 6As shown, the digest generation device 600 of this embodiment includes: an acquisition module 601, an encoding module 602, an extraction module 603, a first fusion module 604, a calculation module 605, a second fusion module 606, and a decoding module 607, wherein:

[0093] An acquisition module 601 is used to acquire text data of a text; the text data includes paragraph data and paragraph summary data, and one paragraph data corresponds to one paragraph summary data; an encoding module 602 is used to encode the paragraph data and the paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; an extraction module 603 is used to extract the main information and entity information in the paragraph data to obtain a main vector and an entity vector; the main information at least includes the data corresponding to the subject-predicate-object structure or the subject-predicate-object-attributive-adverb-complement structure in the paragraph corresponding to the paragraph data; the entity information at least includes the quantifiers and nouns in the paragraph corresponding to the paragraph data; a first fusion module 604 is used to fuse the paragraph vector and the main vector to obtain a first fusion vector; a calculation module 605 is used to calculate a cross-attention vector based on the first fusion vector and the paragraph summary vector; a second fusion module 606 is used to fuse the cross-attention vector and the entity vector to obtain a second fusion vector; a decoding module 607 is used to decode the second fusion vector to obtain a summary of the text.

[0094] In this embodiment, the process of generating a text summary integrates the main information and entity information of the paragraph data. The main information includes the subject-verb-object structure or subject-verb-object-attributive-adverbial-complement structure in the paragraph, and the entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by this application necessarily includes the main structure, quantifiers, and nouns in the text, avoiding the problems of excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0095] In some possible implementations of this embodiment, the first fusion module 604 includes a conversion submodule, a calculation submodule and a fusion submodule; the conversion submodule is used to perform dimensional conversion processing on the paragraph vector and the trunk vector respectively to obtain a paragraph conversion vector and a trunk conversion vector; the paragraph conversion vector is a conversion vector of the paragraph data; the trunk conversion vector is a conversion vector of the trunk information; the calculation submodule is used to calculate the self-attention vector of the paragraph data based on the paragraph conversion vector; the fusion submodule is used to fuse the self-attention vector and the trunk conversion vector to obtain a first fusion vector.

[0096] In some optional implementations of this embodiment, the fusion submodule is specifically configured to: The self-attention vector and the backbone transformation vector are fused to obtain a first fused vector; wherein V′ is used to represent the first fused vector, is used to represent the self-attention vector, Z′ is used to represent the backbone transformation vector, A parameter matrix for representing the first fused vector, A bias term used to represent the first fused vector.

[0097] In this embodiment, the paragraph vector and the backbone vector are fused. This means that the backbone information of the paragraph data is incorporated into the text summary during the text summary generation process. The backbone information includes the subject-verb-object structure or the subject-verb-object-attributive-adverbial-complement structure within the paragraph. This ensures that the text summary generated by this application includes the main structure of the text, avoiding issues such as excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0098] In some possible implementations of this embodiment, the calculation module 605 includes a first sub-module and a second sub-module; the first sub-module is used to perform dimension conversion processing on the first fusion vector and the paragraph summary vector through the transformer decoder to obtain a cross-conversion vector; the second sub-module is used to calculate the cross-attention vector based on the cross-conversion vector.

[0099] In this embodiment, the fused paragraph vector and backbone vector are used to calculate the cross-attention vector with the paragraph summary vector. In this way, the generated text summary must include the main structure of the text, avoiding the problem of excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0100] In some possible implementations of this embodiment, the second fusion module 606 includes a third submodule and a fourth submodule; the third submodule is configured to: The entity vector is dimensionally transformed to obtain an entity transformation vector, where C′ is used to represent the entity transformation vector, i represents the i-th word in the entity information, and when i is an even number, pos(i)=sin(i / 10000 2 ), when i is an odd number, pos(i)=cos(i / 10000 2 ), Used to represent vector product, c i Used to represent the i-th element in the entity vector; the fourth submodule is used to fuse the cross-attention vector and the entity transformation vector to obtain a second fused vector.

[0101] In some possible implementations of this embodiment, the fourth submodule is specifically configured to: Fusing the cross attention vector and the entity transformation vector to obtain a second fused vector; wherein V″′ is used to represent the second fused vector, V″ is used to represent the cross attention vector, and C′ is used to represent the entity transformation vector. A parameter matrix for representing the second fused vector, A bias term used to represent the second fused vector.

[0102] In this embodiment, the entity information of the paragraph data is integrated into the text summary generation process. The entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by this application necessarily includes the quantifiers and nouns in the text, avoiding the problem of excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0103] In some possible implementations of this embodiment, the generating device further includes a sample acquisition module, a training module, and a determination module; the sample acquisition module is used to acquire sample data of historical text; wherein the sample data includes historical paragraph data and historical summary data, and one historical paragraph data corresponds to one historical summary data; the training module is used to input the sample data into a preset model for training to obtain a summary generation model; the determination module is used to determine the parameter matrix and bias term of the target vector according to the summary generation model; wherein the target vector includes at least the first fusion vector and the second fusion vector.

[0104] In this embodiment, a summary generation model is obtained by training sample data of historical texts, and then the parameter matrix and bias term of the first fusion vector and the second fusion vector are determined based on the summary generation model, which can improve the quality of the generated text summary.

[0105] To solve the above technical problems, the present application also provides a computer device. Figure 7 , Figure 7 This is a basic structural block diagram of the computer device in this embodiment.

[0106] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 7 with components 71-73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0107] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0108] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as the hard disk or memory of the computer device 7. In other embodiments, the memory 71 may also be an external storage device of the computer device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 7. Of course, the memory 71 may also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions for the digest generation method. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0109] In some embodiments, the processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to execute computer-readable instructions stored in the memory 71 or process data, such as computer-readable instructions for executing the digest generation method.

[0110] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0111] The computer device provided in this embodiment can execute the steps of the above-mentioned method for generating a summary. The steps of the method for generating a summary here can be the steps of the method for generating a summary in each of the above-mentioned embodiments.

[0112] In this embodiment, the process of generating a text summary integrates the main information and entity information of the paragraph data. The main information includes the subject-verb-object structure or subject-verb-object-attributive-adverbial-complement structure in the paragraph, and the entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by this application necessarily includes the main structure, quantifiers, and nouns in the text, avoiding the problems of excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0113] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the abstract generation method as described above.

[0114] In this embodiment, the process of generating a text summary integrates the main information and entity information of the paragraph data. The main information includes the subject-verb-object structure or subject-verb-object-attributive-adverbial-complement structure in the paragraph, and the entity information includes the quantifiers and nouns in the paragraph. In this way, the text summary generated by this application necessarily includes the main structure, quantifiers, and nouns in the text, avoiding the problems of excessive information loss or information tampering in the generated text summary, and improving the quality of the generated text summary.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0116] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for generating a summary, characterized in that: The steps include: Acquire text data of the text; the text data includes paragraph data and paragraph summary data, and one paragraph data corresponds to one paragraph summary data; Encoding the paragraph data and the paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; Extracting the main information and entity information from the paragraph data to obtain a main vector and an entity vector; the main information at least includes data corresponding to a subject-predicate-object structure or a subject-predicate-object-attributive-adverbial-complement structure in the paragraph corresponding to the paragraph data; and the entity information at least includes quantifiers and nouns in the paragraph corresponding to the paragraph data; Fusing the paragraph vector and the backbone vector to obtain a first fused vector; Calculate a cross attention vector based on the first fusion vector and the paragraph summary vector; Fusing the cross-attention vector and the entity vector to obtain a second fused vector; Decoding the second fused vector to obtain a summary of the text; The text data of the acquired text includes: Using a pre-trained paragraph segmentation model to divide the paragraphs corresponding to the text to obtain paragraph data of the text; A pre-trained paragraph summary extraction model is used to extract a summary of the paragraph corresponding to the text to obtain paragraph summary data of the text.

2. The generation method according to claim 1, characterized in that The fusing the paragraph vector and the backbone vector to obtain a first fused vector includes: Performing dimension conversion processing on the paragraph vector and the backbone vector respectively to obtain a paragraph conversion vector and a backbone conversion vector; the paragraph conversion vector is a conversion vector of the paragraph data; the backbone conversion vector is a conversion vector of the backbone information; Calculating a self-attention vector of the paragraph data according to the paragraph transformation vector; The self-attention vector and the backbone transformation vector are fused to obtain a first fused vector.

3. The generation method according to claim 2, characterized in that The fusing the self-attention vector and the backbone transformation vector to obtain a first fused vector includes: According to the formula The self-attention vector and the backbone transformation vector are fused to obtain a first fused vector; wherein V′ is used to represent the first fused vector, is used to represent the self-attention vector, Z′ is used to represent the backbone transformation vector, A parameter matrix for representing the first fused vector, A bias term used to represent the first fused vector.

4. The generation method according to claim 1, characterized in that The calculating of the cross attention vector according to the first fusion vector and the paragraph summary vector includes: Performing dimension conversion processing on the first fusion vector and the paragraph summary vector through a transformer decoder to obtain a cross-conversion vector; The cross attention vector is calculated based on the cross conversion vector.

5. The generation method according to claim 1, characterized in that The fusing the cross attention vector and the entity vector to obtain a second fused vector includes: According to the formula The entity vector is dimensionally transformed to obtain an entity transformation vector, where C′ is used to represent the entity transformation vector, i represents the i-th word in the entity information, and when i is an even number, pos(i)=sin(i / 10000 2 ), when i is an odd number, pos(i)=cos(i / 10000 2 ), Used to represent vector product, c i Used to represent the i-th element in the entity vector; The cross-attention vector and the entity transformation vector are fused to obtain a second fused vector.

6. The generation method according to claim 5, characterized in that The fusing the cross attention vector and the entity transformation vector to obtain a second fused vector includes: According to the formula Fusing the cross attention vector and the entity transformation vector to obtain a second fused vector; wherein V″′ is used to represent the second fused vector, V″ is used to represent the cross attention vector, and C′ is used to represent the entity transformation vector. A parameter matrix for representing the second fused vector, A bias term used to represent the second fused vector.

7. The generation method according to claim 3 or 6, characterized in that: Before obtaining the text data of the text, the generation method further includes: Acquire sample data of historical text; wherein the sample data includes historical paragraph data and historical summary data, and one historical paragraph data corresponds to one historical summary data; Inputting the sample data into a preset model for training to obtain a summary generation model; A parameter matrix and a bias term of a target vector are determined according to the summary generation model; wherein the target vector includes at least the first fused vector and the second fused vector.

8. A device for generating a summary, characterized in that: include: An acquisition module, configured to acquire text data of a text; the text data includes paragraph data and paragraph summary data, where one paragraph data corresponds to one paragraph summary data; an encoding module, configured to encode the paragraph data and the paragraph summary data respectively to obtain a paragraph vector and a paragraph summary vector; an extraction module, configured to extract the main information and entity information from the paragraph data to obtain a main vector and an entity vector; the main information at least includes data corresponding to a subject-predicate-object structure or a subject-predicate-object-attributive-adverb-complement structure in the paragraph corresponding to the paragraph data; and the entity information at least includes quantifiers and nouns in the paragraph corresponding to the paragraph data; A first fusion module, configured to fuse the paragraph vector and the backbone vector to obtain a first fusion vector; A calculation module, configured to calculate a cross-attention vector based on the first fusion vector and the paragraph summary vector; A second fusion module, configured to fuse the cross attention vector and the entity vector to obtain a second fusion vector; a decoding module, configured to decode the second fused vector to obtain a summary of the text; The acquisition module is further used to use a pre-trained paragraph division model to divide the paragraphs corresponding to the text to obtain paragraph data of the text; and use a pre-trained paragraph summary extraction model to extract summaries of the paragraphs corresponding to the text to obtain paragraph summary data of the text.

9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the method for generating an abstract according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for generating an abstract according to any one of claims 1 to 7.

Images