Abstract generation method and device applied to dialogue, equipment and storage medium

Abstract

The invention belongs to the technical field of natural language processing, and relates to an abstract generation method applied to dialogues and related equipment, and the method comprises the steps: obtaining to-be-processed original dialogue data; labeling the original dialogue data to obtain labeled dialogue data; according to the Levi graph, performing graph coding operation and first feature weighting operation on the labeled dialogue data to obtain a first weighted feature vector; according to an encoder of the BART model, text encoding operation and second feature weighting operation are carried out on the annotated dialogue data, and a second weighted feature vector is obtained; performing splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced feature vector; and decoding the spliced feature vector according to a decoder of the BART model to obtain a conference summary generation text. According to the invention, Levi graph coding is carried out on speakers at different visual angles, and visual angle coding features are used as input to control the generation of abstracts of the same conference content seen by different roles.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a summary generation method, device, equipment and storage medium for conversation. Background Art

[0002] At present, with the popularity of electronic devices recording audio or video to record events, the ways of recording meetings have become more diverse. In addition to text recording, audio and video are also increasingly used. For example: shorthand text recording of large content to facilitate text live broadcast or press release; or audio and video recording of weekly meetings to facilitate future file review, etc.

[0003] There is an existing method for generating conference summaries, which uses an artificial intelligence model to generate conversation summaries, such as conference summaries, which will summarize the key parts of the overall conference content.

[0004] However, the applicant found that the traditional conference summary generation method does not distinguish between different speakers in the conference. Because this type of dialogue data has different speakers, each speaker's focus on the dialogue content is inconsistent. Especially in multi-person dialogues, speaker A will focus more on the context of his own speech and pay less attention to the interaction between others.

[0005] It can be seen that the traditional meeting summary generation method cannot generate meeting summaries corresponding to different roles. Summary of the invention

[0006] The purpose of the embodiments of the present application is to propose a summary generation method, device, equipment and storage medium for dialogue, and the problem that the traditional conference summary generation method cannot generate conference summaries corresponding to different roles.

[0007] In order to solve the above technical problems, the embodiment of the present application provides a summary generation method applied to a conversation, which adopts the following technical solution:

[0008] Get the raw conversation data to be processed;

[0009] Performing a labeling operation on the original conversation data to obtain labeled conversation data;

[0010] Constructing a Levi graph according to the annotated conversation data, and performing a graph encoding operation on the annotated conversation data according to the Levi graph to obtain graph encoded data;

[0011] Performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector;

[0012] Performing text encoding operation on the annotated conversation data according to the encoder of the BART model to obtain text encoded data;

[0013] Performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector;

[0014] Performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a spliced ​​feature vector;

[0015] The concatenated feature vector is decoded according to the decoder of the BART model to obtain the generated text of the conference summary.

[0016] Furthermore, the step of performing a first feature weighting operation on the graph coded data to obtain a first weighted feature vector specifically includes the following steps:

[0017] Input the graph encoding data into the RGCN layer for maximum pooling operation to obtain maximum pooling data;

[0018] Inputting the maximum pooling data into a first density layer to perform a first linear transformation operation to obtain first linear change data;

[0019] A first feature weighting operation is performed on the first linear change data according to a self-attention mechanism to obtain the first weighted feature vector.

[0020] Furthermore, the step of performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector specifically includes the following steps:

[0021] Inputting the text encoding data into a second density layer to perform a second linear change operation to obtain second linear change data;

[0022] A second feature weighting operation is performed on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

[0023] Furthermore, after the step of performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a spliced ​​feature vector, the following step is also included:

[0024] Input the concatenated feature vectors into two multi-layer perceptron layers respectively to obtain two multi-layer perceptron features;

[0025] Respectively mapping the two multi-layer perceptual features to general features of feature vector dimensions to obtain enhanced concatenated feature vectors;

[0026] The decoder according to the BART model decodes the concatenated feature vector to obtain the step of generating the conference summary text, which specifically includes the following steps:

[0027] The enhanced concatenated feature vector is decoded according to the decoder of the BART model to obtain the generated text of the conference summary.

[0028] Furthermore, the decoder according to the BART model decodes the concatenated feature vector to obtain the step of generating the conference summary text, which specifically includes the following steps:

[0029] Inputting the concatenated feature vector into a third density layer to perform a third linear change operation to obtain third linear change data;

[0030] Performing a third feature weighting operation on the third linear change data according to a self-attention mechanism to obtain a third weighted feature vector;

[0031] The third weighted feature vector is input into the decoder of the BART model for decoding to obtain a conference summary generation result.

[0032] In order to solve the above technical problems, the embodiment of the present application further provides a summary generation device applied to a conversation, which adopts the following technical solution:

[0033] The original data acquisition module is used to acquire the original conversation data to be processed;

[0034] A labeling module, used for labeling the original conversation data to obtain labeled conversation data;

[0035] A graph encoding module, used for constructing a Levi graph according to the annotated conversation data, and performing a graph encoding operation on the annotated conversation data according to the Levigraph to obtain graph encoded data;

[0036] A first feature weighting module, used for performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector;

[0037] A text encoding module, used for performing a text encoding operation on the annotated dialogue data according to an encoder of a BART model to obtain text encoding data;

[0038] A second feature weighting module, used for performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector;

[0039] A feature splicing module, used for performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced ​​feature vector;

[0040] The decoding module is used to perform a decoding operation on the concatenated feature vector according to a decoder of the BART model to obtain a generated text of the conference summary.

[0041] Furthermore, the first feature weighting module includes:

[0042] The maximum pooling submodule is used to input the graph encoding data into the RGCN layer for performing a maximum pooling operation to obtain maximum pooling data;

[0043] A first linear transformation submodule, used for inputting the maximum pooling data into a first density layer to perform a first linear transformation operation to obtain first linear change data;

[0044] The first feature weighting submodule is used to perform a first feature weighting operation on the first linear change data according to a self-attention mechanism to obtain the first weighted feature vector.

[0045] Furthermore, the second feature weighting module includes:

[0046] A second linear change submodule, used for inputting the text encoding data into a second density layer to perform a second linear change operation to obtain second linear change data;

[0047] The second feature weighting submodule is used to perform a second feature weighting operation on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0049] The invention comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the summary generation method applied to a conversation as described above when executing the computer-readable instructions.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0051] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the summary generation method applied to a conversation as described above are implemented.

[0052] The present application provides a summary generation method for dialogue, including: obtaining original dialogue data to be processed; annotating the original dialogue data to obtain annotated dialogue data; constructing a Levi graph according to the annotated dialogue data, and performing a graph encoding operation on the annotated dialogue data according to the Levi graph to obtain graph encoding data; performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector; performing a text encoding operation on the annotated dialogue data according to an encoder of a BART model to obtain text encoding data; performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector; performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced ​​feature vector; performing a decoding operation on the spliced ​​feature vector according to a decoder of a BART model to obtain a conference summary generation text. Compared with the prior art, the present application performs Levi graph encoding on speakers from different perspectives, and uses the perspective encoding feature as input to control the decoder to generate summaries of the same conference content viewed by different roles. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0055] Figure 2 is a flowchart of the implementation of the summary generation method applied to the conversation provided in the first embodiment of the present application;

[0056] Figure 3 is a schematic diagram of the structure of a summary generation device applied to a conversation provided in Embodiment 2 of the present application;

[0057] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0059] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0061] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0062] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0063] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0064] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0065] It should be noted that the summary generation method applied to the conversation provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the summary generation device applied to the conversation is generally set in the server / terminal device.

[0066] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0067] Embodiment 1

[0068] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating a summary of a conversation according to the present application. The method for generating a summary of a conversation includes: step S201, step S202, step S203, step S204, step S205, step S206, step S207 and step S208.

[0069] In step S201, original conversation data to be processed is obtained.

[0070] In step S202, the original conversation data is annotated to obtain annotated conversation data.

[0071] In an embodiment of the present application, it is necessary to manually annotate the conference data and manually annotate the conversation data in the conference according to the speaker role. For example, the same conference content is annotated with text summaries from the perspectives of speakers S and T respectively. At the same time, non-participants annotate the text summaries of the conference content. In this way, there are 2 types of participants in a conference conversation content, but there are 3 summaries, namely S, T and the overall Gen, where S and T are the conference text summaries annotated from the perspectives of different types of participants, and Gen is the summary of non-participants. The summary annotation of Gen will be used as objective overall summary annotation data. The format of the annotated training set is as follows:

[0072] S0: XXXXX, XX.

[0073] T0: XXXX.

[0074] T1: XXXXX.

[0075] S1: XXX.

[0076] S2: XXX.

[0077] T2: XXX, XXX.

[0078] …

[0079] Tn: XXXXXXX.

[0080] Sn: XXXXXXX.

[0081] SA:XXXXXXXX. TA:XXXXXX. GenA:XXXXXXXX.

[0082] Among them, content S and T are the conference speeches of participants S and T respectively, S0 is the first sentence spoken by participant S, T0 is the first sentence spoken by participant T, and n represents the nth sentence of each. SA is the conference summary from the perspective of S, TA is the conference summary from the perspective of participant T, and GenA is the conference summary after non-participants read the conference content of S and T.

[0083] In the embodiment of the present application, the conversation graph G = (V, E), where V is the content of the current speaker, and E is the transition of the current content (E represents the relationship between two sentences, which is strongly related to the current speaker role. E from the i-th node to the j-th node is defined as e{ij}, and the node i is the i-th sentence in the conference conversation. For each e{ij}, a label category l is manually marked. ij , this category includes three types: intent (In), action (Ac) and context (context):

[0084]

[0085] Among them, Ac, In and context are three types of labels of lij. Ac represents action, In represents intention, and context represents context (if there is a contextual relationship between the current two sentences). utt.i represents the current content node i, T and S represent the speaker role T or S of the conference content respectively. P represents the current role perspective.

[0086] Assume that the current speaker is T, utt.i∈T, and the character perspective is also on T or Gen, P∈'T','Gen'. Then when the current speech content node i is connected to other speech content j, the current label lij is marked as Ac.

[0087] Assume that the current speaker is S, utt.i∈S, and the character perspective is also on S or Gen, P∈'S','Gen'. Then when the current speech content node i is connected to other speech content j, the current label lij is marked as In.

[0088] Assume that the current speaker is S, utt.i∈S, the role perspective is on T, P∈'S', then when the current speech content node i is connected to other speech content j, the current label lij is marked as context.

[0089] Assume that the current speaker is T, utt.i∈T, the character perspective is also on S, P∈'S', then when the current speech content node i is connected to other speech content j, the current label lij is marked as context.

[0090] In step S203, a Levi graph is constructed according to the annotated conversation data, and a graph encoding operation is performed on the annotated conversation data according to the Levi graph to obtain graph encoded data.

[0091] In the embodiment of the present application, after the data is annotated, the conference conversation data is modeled as a Levigraph. Since the same sentence has different character perspectives, the annotation labels of E will be different, so when encoding the Levi graph, different perspectives will have different learning objectives.

[0092] In step S204, a first feature weighting operation is performed on the graph encoding data to obtain a first weighted feature vector.

[0093] In step S205, a text encoding operation is performed on the annotated dialogue data according to the encoder of the BART model to obtain text encoding data.

[0094] In step S206, a second feature weighting operation is performed on the text encoding data to obtain a second weighted feature vector.

[0095] In step S207, a concatenation operation is performed on the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a concatenated feature vector.

[0096] Although the two parts of the input are transformed by dense layers with different parameters, the two sides are symmetrical structures, which ensures the consistency of transformation of the same input and the same dimension of the input vector.

[0097] In the embodiment of the present application, two vectors H with the same dimension are D and H I To splice:

[0098] Z G =OTKE([H ′ D :H′ I ])

[0099] In step S208, the concatenated feature vector is decoded according to the decoder of the BART model to obtain the generated text of the conference summary.

[0100] In an embodiment of the present application, a summary generation method applied to a conversation is provided, including: obtaining original conversation data to be processed; annotating the original conversation data to obtain annotated conversation data; constructing a Levi graph according to the annotated conversation data, and performing a graph encoding operation on the annotated conversation data according to the Levi graph to obtain graph encoding data; performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector; performing a text encoding operation on the annotated conversation data according to an encoder of a BART model to obtain text encoding data; performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector; performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced ​​feature vector; performing a decoding operation on the spliced ​​feature vector according to a decoder of a BART model to obtain a conference summary generation text. Compared with the prior art, the present application performs Levi graph encoding on speakers from different perspectives, and uses the perspective encoding feature as an input to control the decoder to generate summaries of the same conference content viewed by different roles.

[0101] In some optional implementations of the embodiments of the present application, the above step S204 specifically includes the following steps:

[0102] Input the graph encoding data into the RGCN layer for maximum pooling operation to obtain the maximum pooling data;

[0103] Inputting the maximum pooling data into the first density layer to perform a first linear transformation operation to obtain first linear change data;

[0104] A first feature weighting operation is performed on the first linear change data according to the self-attention mechanism to obtain a first weighted feature vector.

[0105] In the embodiment of the present application, the same conversation data is divided into two parts. One part is input into the Levi graph for encoding, and then input into the RGCN layer for pooling (maximum pooling operation). The feature vector after pooling is input into a dense layer, which is composed of a fully connected layer. Then self-attention is performed, and then the self-attention weight is multiplied by the feature vector itself to obtain the vector H. D As one of the inputs of the OTK fusion model, the self attention calculation process is as follows:

[0106]

[0107] In some optional implementations of the embodiments of the present application, the above step S206 specifically includes the following steps:

[0108] Inputting the text encoding data into the second density layer to perform a second linear change operation to obtain second linear change data;

[0109] A second feature weighting operation is performed on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

[0110] In the embodiment of the present application, another part of the dialogue is input into a BART model for text encoding, and the obtained encoding vector is input into another fully connected layer (dense layer), and the self-attention operation is also performed to obtain H I Vector, which is also one of the inputs of the OTK fusion model, where

[0111] In some optional implementations of the embodiments of the present application, after step S207, the following steps are further included:

[0112] Input the concatenated feature vectors into two multi-layer perceptron layers respectively to obtain two multi-layer perceptron features;

[0113] Map the two multi-layer perceptual features to the general features of the feature vector dimension respectively to obtain the enhanced concatenated feature vector;

[0114] The above step S208 specifically includes the following steps:

[0115] The decoder of the BART model is used to decode the enhanced concatenated feature vector to obtain the generated text of the conference summary.

[0116] In the embodiment of the present application, the concatenated vector ZG is input into two MLP layers. An MLP consists of a fully connected layer and an activation function. Finally, the features are mapped to half of the features of the concatenated dimension, which is consistent with the HD or HI dimension.

[0117] In some optional implementations of the embodiments of the present application, the above step S208 specifically includes the following steps:

[0118] Inputting the concatenated feature vector into the third density layer to perform a third linear change operation to obtain third linear change data;

[0119] Performing a third feature weighting operation on the third linear change data according to the self-attention mechanism to obtain a third weighted feature vector;

[0120] The third weighted feature vector is input into the decoder of the BART model for decoding to obtain the conference summary generation result.

[0121] In the embodiment of the present application, the fusion model output of OTK is finally input into a density layer composed of a full connection, and then the self attention is calculated. Finally, the output of the self attention is input into the decoder of BART for decoding to generate conference summaries of different perspectives. The different perspective summaries depend on which perspective encoded Levi graph is used when inputting, and the perspective can be T, S or Gen.

[0122] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0123] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0124] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0125] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0126] Embodiment 2

[0127] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a summary generation device for a conversation, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0128] like Figure 3 As shown, the summary generation device 200 applied to a conversation in an embodiment of the present application includes:

[0129] The original data acquisition module 210 is used to acquire the original conversation data to be processed;

[0130] The annotation module 220 is used to perform annotation operation on the original conversation data to obtain annotated conversation data;

[0131] A graph encoding module 230 is used to construct a Levi graph according to the annotated conversation data, and perform a graph encoding operation on the annotated conversation data according to the Levi graph to obtain graph encoded data;

[0132] A first feature weighting module 240, configured to perform a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector;

[0133] A text encoding module 250, configured to perform text encoding operations on the annotated dialogue data according to an encoder of the BART model to obtain text encoding data;

[0134] A second feature weighting module 260, configured to perform a second feature weighting operation on the text encoding data to obtain a second weighted feature vector;

[0135] A feature concatenation module 270 is used to concatenate the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a concatenated feature vector;

[0136] The decoding module 280 is used to perform a decoding operation on the concatenated feature vector according to the decoder of the BART model to obtain the generated text of the conference summary.

[0137] In an embodiment of the present application, a summary generation device 200 applied to a conversation is provided, including: an original data acquisition module 210, used to acquire original conversation data to be processed; a labeling module 220, used to label the original conversation data to obtain labeled conversation data; a graph encoding module 230, used to construct a Levi graph according to the labeled conversation data, and perform a graph encoding operation on the labeled conversation data according to the Levi graph to obtain graph encoded data; a first feature weighting module 240, used to perform a first feature weighting operation on the graph encoded data to obtain a first weighted feature vector; a text encoding module 250, used to perform a text encoding operation on the labeled conversation data according to an encoder of a BART model to obtain text encoded data; a second feature weighting module 260, used to perform a second feature weighting operation on the text encoded data to obtain a second weighted feature vector; a feature splicing module 270, used to perform a splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced ​​feature vector; a decoding module 280, used to perform a decoding operation on the spliced ​​feature vector according to a decoder of a BART model to obtain a conference summary generation text. Compared with the prior art, the present application performs Levi graph encoding on speakers from different perspectives and uses the perspective encoding features as input to control the decoder to generate summaries of the same conference content viewed by different roles.

[0138] In some optional implementations of the embodiments of the present application, the first feature weighting module includes:

[0139] The maximum pooling submodule is used to input the graph encoding data into the RGCN layer for maximum pooling operation to obtain the maximum pooling data;

[0140] A first linear transformation submodule, used for inputting the maximum pooling data into the first density layer to perform a first linear transformation operation to obtain first linear change data;

[0141] The first feature weighting submodule is used to perform a first feature weighting operation on the first linear change data according to the self-attention mechanism to obtain a first weighted feature vector.

[0142] In some optional implementations of the embodiments of the present application, the second feature weighting module includes:

[0143] A second linear change submodule, used for inputting the text encoding data into the second density layer to perform a second linear change operation to obtain second linear change data;

[0144] The second feature weighting submodule is used to perform a second feature weighting operation on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

[0145] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0146] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0147] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0148] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for the method for generating a summary of a conversation. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0149] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions of the summary generation method applied to the conversation.

[0150] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0151] The computer device provided in this application performs Levi graph encoding on speakers from different perspectives and uses the perspective encoding features as input to control the decoder to generate summaries of the same meeting content viewed by different roles.

[0152] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the summary generation method applied to the conversation as described above.

[0153] The computer-readable storage medium provided in the present application performs Levi graph encoding on speakers from different perspectives and uses the perspective encoding features as input to control the decoder to generate summaries of the same meeting content viewed by different roles.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0155] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A summary generation method applied to a conversation, characterized in that: The steps include: Get the raw conversation data to be processed; Performing a labeling operation on the original conversation data to obtain labeled conversation data; Constructing a Levi graph according to the annotated conversation data, and performing a graph encoding operation on the annotated conversation data according to the Levi graph to obtain graph encoded data; Performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector; Performing text encoding operation on the annotated conversation data according to the encoder of the BART model to obtain text encoded data; Performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector; Performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a spliced ​​feature vector; The concatenated feature vector is decoded according to the decoder of the BART model to obtain the generated text of the conference summary.

2. The method for generating a summary for a conversation according to claim 1, characterized in that: The step of performing a first feature weighting operation on the graph coded data to obtain a first weighted feature vector specifically comprises the following steps: Input the graph encoding data into the RGCN layer for maximum pooling operation to obtain maximum pooling data; Inputting the maximum pooling data into a first density layer to perform a first linear transformation operation to obtain first linear change data; A first feature weighting operation is performed on the first linear change data according to a self-attention mechanism to obtain the first weighted feature vector.

3. The method for generating a summary for a conversation according to claim 1, characterized in that: The step of performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector specifically comprises the following steps: Inputting the text encoding data into a second density layer to perform a second linear change operation to obtain second linear change data; A second feature weighting operation is performed on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

4. The method for generating a summary for a conversation according to claim 1, characterized in that: After the step of performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to the OTK fusion model to obtain a spliced ​​feature vector, the following step is also included: Input the concatenated feature vectors into two multi-layer perceptron layers respectively to obtain two multi-layer perceptron features; Respectively mapping the two multi-layer perceptual features to general features of feature vector dimensions to obtain enhanced concatenated feature vectors; The decoder according to the BART model decodes the concatenated feature vector to obtain the step of generating the conference summary text, which specifically includes the following steps: The enhanced concatenated feature vector is decoded according to the decoder of the BART model to obtain the generated text of the conference summary.

5. The method for generating a summary for a conversation according to claim 1, characterized in that: The decoder according to the BART model decodes the concatenated feature vector to obtain the step of generating the conference summary text, which specifically includes the following steps: Inputting the concatenated feature vector into a third density layer to perform a third linear change operation to obtain third linear change data; Performing a third feature weighting operation on the third linear change data according to a self-attention mechanism to obtain a third weighted feature vector; The third weighted feature vector is input into the decoder of the BART model for decoding to obtain a conference summary generation result.

6. A summary generation device for conversation, characterized in that: include: The original data acquisition module is used to acquire the original conversation data to be processed; A labeling module, used for labeling the original conversation data to obtain labeled conversation data; A graph encoding module, used for constructing a Levi graph according to the annotated conversation data, and performing a graph encoding operation on the annotated conversation data according to the Levi graph to obtain graph encoded data; A first feature weighting module, used for performing a first feature weighting operation on the graph encoding data to obtain a first weighted feature vector; A text encoding module, used for performing a text encoding operation on the annotated dialogue data according to an encoder of a BART model to obtain text encoding data; A second feature weighting module, used for performing a second feature weighting operation on the text encoding data to obtain a second weighted feature vector; A feature splicing module, used for performing a splicing operation on the first weighted feature vector and the second weighted feature vector according to an OTK fusion model to obtain a spliced ​​feature vector; The decoding module is used to perform a decoding operation on the concatenated feature vector according to a decoder of the BART model to obtain a generated text of the conference summary.

7. The device for generating a summary for a conversation according to claim 6, characterized in that: The first feature weighting module includes: The maximum pooling submodule is used to input the graph encoding data into the RGCN layer for performing a maximum pooling operation to obtain maximum pooling data; A first linear transformation submodule, used for inputting the maximum pooling data into a first density layer to perform a first linear transformation operation to obtain first linear change data; The first feature weighting submodule is used to perform a first feature weighting operation on the first linear change data according to a self-attention mechanism to obtain the first weighted feature vector.

8. The device for generating a summary for a conversation according to claim 6, characterized in that: The second feature weighting module includes: A second linear change submodule, used for inputting the text encoding data into a second density layer to perform a second linear change operation to obtain second linear change data; The second feature weighting submodule is used to perform a second feature weighting operation on the second linear change data according to the self-attention mechanism to obtain a second weighted feature vector.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the summary generation method applied to a conversation as claimed in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the summary generation method applied to a conversation as claimed in any one of claims 1 to 5 are implemented.

Images