A method, apparatus, electronic device, and storage medium for abstract generation

Abstract

The present invention provides a method, apparatus, electronic device and storage medium for abstract generation, which are applied to the technical field of data processing. The method includes: obtaining text data to be processed, where the text data includes a plurality of words; inputting the text data and a model prompt template into a first network model to obtain an annotated text, where the model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the plurality of words; inputting the text data and the annotated text into a second network model to obtain summary information of the text data; where the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method, device, electronic device, and storage medium for generating abstracts. Background Art

[0002] With the increasing growth of multi-source event information, the forms and contents of events are constantly diversifying, and the massive text information makes the event analysis task increasingly difficult. To assist the efficient analysis of event texts, it is necessary to automatically identify key entities and events in the texts and rewrite information briefs.

[0003] In the prior art, information compression and abstract generation of long event texts and multi-document event texts can be achieved through pre-trained language models. The mainstream event text information processing technology mainly performs domain fine-tuning learning based on the language model on specific downstream datasets. High-quality model output results usually require training with a large amount of manually annotated supervision data. However, manual annotation not only has a high cost but also is difficult to obtain in large quantities. Summary of the Invention

[0004] The present invention provides a method, device, electronic device, and storage medium for generating abstracts to solve the problems in the prior art that manual annotation not only has a high cost but also is difficult to obtain in large quantities.

[0005] The present invention provides a method for generating an abstract, including: obtaining text data to be processed, where the text data includes a plurality of words; inputting the text data and a model prompt template into a first network model to obtain an annotated text, where the model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the plurality of words; inputting the text data and the annotated text into a second network model to obtain summary information of the text data; where the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

[0006] According to the method for generating an abstract provided by the present invention, before obtaining the text data to be processed, the method further includes: obtaining an event text training set, where the event text training set includes training texts, named entities and events manually annotated; inputting the training texts and the model prompt template into the first network model to obtain a first output information; updating model parameters of the first network model based on the first output information and the named entities and events manually annotated; where both the Query layer and the Value layer of the first network model include a trainable LoRA module, and the LoRA module includes two layers of trainable fully connected layers.

[0007] According to the present invention, there is provided a method for generating an abstract. Updating the model parameters of the first network model based on the first output information and the manually annotated named entities and events includes: calculating the model loss between the first output information and the manually annotated named entities and events using a cross-entropy loss function; updating the model parameters of the first network model based on the model loss.

[0008] According to the present invention, there is provided a method for generating an abstract. The event text training set further includes manually written abstracts. After obtaining the event text training set, the method further includes: inputting the training text and the first output information into the second network model to obtain a second output information; updating the model parameters of the second network model based on the second output information and the manually written abstracts; wherein, the second network model includes a target Transformer network and a cross-attention module, the target Transformer network is used to encode the training text and the first output information, and the cross-attention module is used to learn the context relationship between the first output information and the training text.

[0009] According to the present invention, there is provided a method for generating an abstract. After updating the model parameters of the second network model based on the second output information and the manually written abstracts, the method further includes: determining target output information with an accuracy rate higher than a first threshold from the second output information; adding the target output information to the event text training set.

[0010] The present invention also provides a device for generating an abstract, including: an acquisition module and a processing module; the acquisition module is used to acquire text data to be processed, and the text data includes a plurality of words; the processing module is used to input the text data and a model prompt template into a first network model to obtain an annotated text, the model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the plurality of words; inputting the text data and the annotated text into a second network model to obtain summary information of the text data; wherein, the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

[0011] According to the present invention, there is provided a summary generation device. The acquisition module is used to acquire an event text training set, and the event text training set includes training texts, manually annotated named entities and events. The processing module is used to input the training texts and the model prompt template into the first network model to obtain first output information, and update the model parameters of the first network model based on the first output information and the manually annotated named entities and events. Wherein, both the Query layer and the Value layer of the first network model include a trainable LoRA module, and the LoRA module includes two layers of trainable fully connected layers.

[0012] According to the present invention, there is provided a summary generation device. The processing module is used to calculate the model loss of the first output information and the manually annotated named entities and events by using a cross-entropy loss function, and update the model parameters of the first network model based on the model loss.

[0013] According to the present invention, there is provided a summary generation device. The event text training set further includes a manually written summary. The processing module is used to input the training texts and the first output information into the second network model to obtain second output information, and update the model parameters of the second network model based on the second output information and the manually written summary. Wherein, the second network model includes a target Transformer network and a cross-attention module. The target Transformer network is used to encode the training texts and the first output information, and the cross-attention module is used to learn the context relationship between the first output information and the training texts.

[0014] According to the present invention, there is provided a summary generation device. The processing module is used to determine target output information with an accuracy rate higher than a first threshold from the second output information, and add the target output information to the event text training set.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the summary generation method as described in any one of the above are implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the summary generation method as described in any one of the above are implemented.

[0017] The abstract generation method, device, electronic device, and storage medium provided by the present invention can obtain text data to be processed, where the text data includes multiple words; input the text data and a model prompt template into a first network model to obtain an annotated text, and the model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the multiple words; input the text data and the annotated text into a second network model to obtain the abstract information of the text data; where the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate the abstract information of the text data based on the annotated text. Through this solution, the first network model and the second network model can be integrated to generate the abstract of the text data, and a progressive semi-supervised training strategy is designed to enable the model to learn event information extraction and information compression based on the pseudo-labels generated in the previous round, overcoming the problem of high costs of manual annotation and writing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is one of the flow diagrams of the abstract generation method provided by the present invention;

[0020] Figure 2 is another flow diagram of the abstract generation method provided by the present invention;

[0021] Figure 3 is the structural diagram of the abstract generation device provided by the present invention;

[0022] Figure 4 is the structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0024] It should be noted that in the embodiments of the present invention, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0025] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0026] For the convenience of clearly describing the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0027] Some exemplary embodiments are described in the embodiments of the present invention for the purpose of illustration. It should be understood that the present invention can be implemented in other ways not specifically shown in the drawings.

[0028] The above implementation manners will be elaborated in detail below in conjunction with specific embodiments and drawings.

[0029] As Figure 1 shown, the embodiments of the present invention provide a summary generation method, which can be applied to a summary generation device. The summary generation method may include S101 - S103:

[0030] S101. The summary generation device obtains text data to be processed.

[0031] Wherein, the above text data includes a plurality of words.

[0032] S102. The abstract generation device inputs the text data and the model prompt template into the first network model to obtain an annotated text.

[0033] Among them, the above model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the multiple words. The first network model is used to annotate named entities and events in the text data.

[0034] It should be noted that the model prompt template can be a prompt template based on the prior knowledge of the current task, such as "Identify named entities and events in the following text". The role of the prompt template is to quickly activate the learning ability of the pre-trained language model for domain fine-tuning in the current task.

[0035] S103. The abstract generation device inputs the text data and the annotated text into the second network model to obtain the abstract information of the text data.

[0036] Among them, the above second network model is used to generate the abstract information of the text data according to the annotated text.

[0037] In the embodiments of the present invention, the first network model and the second network model can be integrated to generate the abstract of the text data. A progressive semi-supervised training strategy is designed to enable the model to learn event information extraction and information compression based on the pseudo-labels generated in the previous round, overcoming the problem of high costs of manual annotation and writing.

[0038] As Figure 2 shown, the abstract generation method provided by the embodiments of the present invention may further include S104 - S106:

[0039] S104. The abstract generation device obtains an event text training set.

[0040] Among them, the above event text training set includes training texts, manually annotated named entities and events, and manually written abstracts.

[0041] After obtaining the event text training set, text preprocessing can be performed first. The text preprocessing includes word segmentation, part-of-speech recognition, removal of text noise, symbol normalization, text deduplication, etc. For example, jieba Chinese word segmentation, regular expressions and other conventional text matching methods can be used to clean abnormal symbols and garbled characters, etc. Then, the annotator can mark the key named entities and events on a small number of event texts in the event text training set and manually write the abstract. That is, select the key event elements in the training text, and obtain the event abstract, requiring the annotator to abstract, compress and restate the original long text in the form of writing an abstract for a small number of samples.

[0042] For example, for a text D=(W1, W2,..., Wn ), the annotated text is D Ano =(Ano1, Ano2, ..., Ano n ), Ano∈{0, 1}, where Ano=1 indicates an annotated word and Ano=0 indicates an unannotated word. The text length of the summary of text D is less than n and meets the requirements of the briefing of the relevant event.

[0043] S105 : The summary generating device inputs the training text and the model prompt template into the first network model to obtain first output information.

[0044] The Query layer and the Value layer of the first network model each include a trainable LoRA module, and the LoRA module includes two trainable fully connected layers.

[0045] Optionally, the summary generation device can perform named entity recognition and event recognition based on a prompt learning strategy. The named entity recognition and event recognition tasks require finding correctly matched fragments of multiple nouns and behavioral events in the training text. During the training phase of the machine learning model, the input text sequence is divided into multiple word blocks by the word segmenter of the first network model, and the first network model models the text sequence as a deep semantic representation through a multi-layer fully connected network and an attention network. The prompt learning strategy can utilize the generalization performance of the pre-trained language model on a variety of natural language tasks, and can activate the context modeling ability of the language model in the task by adding prompt learning word blocks that can participate in training optimization to the prefixes of the training text. At the same time, in order to accelerate the entity and event recognition training of this step, a small number of optimizable model parameters are added in the training phase, and other network parameters of the language model are not involved in the training.

[0046] Specifically, the input of the first network model is input = [prompt, D], where prompt represents the model prompt template and D represents the training text. During the training phase, the first network model can use a pluggable parameter efficient fine-tuning technology based on LoRA. Other parameters in the first network model do not participate in the model update.

[0047] The LoRA modules are inserted into the Query layer and the Value layer of the Transformer network respectively. The LoRA module inserts two trainable fully connected layers and r< <d。LoRA模块的输出 for:

[0048]

[0049] Here, α represents the scale constant.

[0050] During the inference stage, the parameters of the LoRA module can be incorporated into W0, maintaining the throughput speed of the original language model:

[0051]

[0052] S106. The abstract generation device updates the model parameters of the first network model based on the first output information and the manually annotated named entities and events.

[0053] Optionally, the abstract generation device may calculate the model loss between the first output information and the manually annotated named entities and events using the cross-entropy loss function; and update the model parameters of the first network model based on the model loss.

[0054] Specifically, the abstract generation device may use the cross-entropy loss function (Cross-Entropy Loss) to supervise the training of the first network model. The model prediction value is The reference value is the text y=(Ano1, Ano2, …, Ano n ), Ano∈{0, 1}, for the category C∈{0, 1}, the loss of the first network model is:

[0055]

[0056] S107. The abstract generation device inputs the training text and the first output information into the second network model to obtain the second output information.

[0057] Among them, the second network model includes a target Transformer network and a cross-attention module. The target Transformer network is used to encode the training text and the first output information, and the cross-attention module is used to learn the context relationship between the first output information and the training text.

[0058] Optionally, the abstract generation device may use the first output information as a feature, jointly with the original training text as the input, and optimize the second network model based on the parameter-efficient fine-tuning algorithm to learn to output the true event abstract. Using the first output information, the cross-attention module is used to fuse its semantic representation into the context semantic modeling of the original training text. Furthermore, by using the parameter-efficient fine-tuning technology for the second network model to add a small number of optimizable model parameters, training acceleration can be achieved, ensuring the model inference speed.

[0059] Specifically, the abstract generation device may incorporate the named entities and events identified by the first network model as features into the original Encoder-Decoder architecture of the second network model, through special characters<sep>to divide the found entities and events, the length of the sequence is m, and the starting point of the training text can be represented by a special character <s>representation

[0060] To model the context representation relationship between key entities and events for the training text, a new Transformer network layer can be added to the second network model to encode them, and a Cross Attention module is adopted to learn their context relationship, enabling the original training text to quickly capture information related to summary generation. For the event text representation from the Encoder and the representation from entities and events The Cross Attention module needs to learn three representations: Q = H doc ·W Q , K = H ent ·W K , V = H ent ·W V , The output of the Cross Attention module is:

[0061]

[0062] S108. The summary generation device updates the model parameters of the second network model based on the second output information and the manually written summary.

[0063] Optionally, after updating the model parameters of the second network model based on the second output information and the manually written summary, the summary generation device can determine target output information with an accuracy rate higher than a first threshold from the second output information; add the target output information to the event text training set.

[0064] Specifically, the first network model and the second network model obtained through parameter-efficient fine-tuning training can perform inference and prediction on unseen samples to obtain their key entities and events and generate summaries. The summary generation device can determine target output information with an accuracy rate higher than a first threshold from the second output information output by the second network model, and include the target output information as a reference output in the dataset in the next round of the training phase. Repeat the above steps for multiple rounds to gradually expand the alternative reference outputs generated by the model until all event text datasets are recognized by the model for events and summaries are generated.

[0065] The above mainly introduces the solution provided by the embodiments of the present invention from the perspective of methods. To implement the above functions, it includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0066] For the abstract generation method provided by the embodiments of the present invention, the execution subject can be an abstract generation device, or a control module for abstract generation in the abstract generation device. In the embodiments of the present invention, taking the abstract generation device executing the abstract generation method as an example, the abstract generation device provided by the embodiments of the present invention is described.

[0067] It should be noted that the embodiments of the present invention can divide the function modules of the abstract generation device according to the above method examples. For example, each function module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module. Optionally, the division of modules in the embodiments of the present invention is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0068] As Figure 3 shown, the embodiments of the present invention provide an abstract generation device 300. The abstract generation device 300 includes: an acquisition module 301 and a processing module 302. The acquisition module 301 can be used to acquire text data to be processed, and the text data includes a plurality of words; the processing module 302 can be used to input the text data and a model prompt template into a first network model to obtain an annotated text. The model prompt template is used to prompt the first network model to identify named entities and events in the input text, and the annotated text includes annotation information for some of the plurality of words; input the text data and the annotated text into a second network model to obtain summary information of the text data; wherein, the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

[0069] Optionally, the obtaining module 301 is configured to obtain an event text training set, where the event text training set includes training texts, manually annotated named entities and events; the processing module 302 is configured to input the training texts and the model prompt template into the first network model to obtain first output information; update model parameters of the first network model based on the first output information and the manually annotated named entities and events; where the Query layer and the Value layer of the first network model each include a trainable LoRA module, and the LoRA module includes two trainable fully connected layers.

[0070] Optionally, the processing module 302 is configured to calculate a model loss of the first output information and the manually annotated named entities and events by using a cross-entropy loss function; update model parameters of the first network model based on the model loss.

[0071] Optionally, the event text training set further includes a manually written abstract; the processing module 302 is configured to input the training texts and the first output information into the second network model to obtain second output information; update model parameters of the second network model based on the second output information and the manually written abstract; where the second network model includes a target Transformer network and a cross-attention module, the target Transformer network is configured to encode the training texts and the first output information, and the cross-attention module is configured to learn a context relationship between the first output information and the training texts.

[0072] Optionally, the processing module 302 is configured to determine target output information with an accuracy rate higher than a first threshold from the second output information; add the target output information to the event text training set.

[0073] In an embodiment of the present invention, the first network model and the second network model can be fused to generate an abstract of text data, and a progressive semi-supervised training strategy is designed to enable the model to learn event information extraction and information compression based on the pseudo-labels generated in the previous round, overcoming the problem of high costs of manual annotation and writing.

[0074] Figure 4 An entity structure schematic diagram of an electronic device is illustrated, such as Figure 4 As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logical instructions in the memory 430 to execute the abstract generation method, which includes: obtaining the text data to be processed, where the text data includes a plurality of words; inputting the text data and the model prompt template into the first network model to obtain the annotated text, where the model prompt template is used to prompt the first network model to identify the named entities and events in the input text, and the annotated text includes the annotation information for some of the plurality of words; inputting the text data and the annotated text into the second network model to obtain the summary information of the text data; where the first network model is used to annotate the named entities and events in the text data, and the second network model is used to generate the summary information of the text data according to the annotated text.

[0075] In addition, when the logical instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0076] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the abstract generation method provided by each of the above methods. The method includes: obtaining text data to be processed, where the text data includes a plurality of words; inputting the text data and a model prompt template into a first network model to obtain an annotated text. The model prompt template is used to prompt the first network model to identify named entities and events in the input text. The annotated text includes annotation information for some of the plurality of words; inputting the text data and the annotated text into a second network model to obtain summary information of the text data; where the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

[0077] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the abstract generation method provided by each of the above. The method includes: obtaining text data to be processed, where the text data includes a plurality of words; inputting the text data and a model prompt template into a first network model to obtain an annotated text. The model prompt template is used to prompt the first network model to identify named entities and events in the input text. The annotated text includes annotation information for some of the plurality of words; inputting the text data and the annotated text into a second network model to obtain summary information of the text data; where the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text.

[0078] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort. [[ID=�]]

[0079] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / s> < / sep>

Claims

1. A method for generating an abstract, characterized in that Including: Obtain the text data to be processed, where the text data includes multiple words; Input the text data and the model prompt template into the first network model to obtain an annotated text. The model prompt template is used to prompt the first network model to identify named entities and events in the input text. The annotated text includes annotation information for some of the multiple words; Input the text data and the annotated text into the second network model to obtain summary information of the text data; Wherein, the first network model is used to annotate named entities and events in the text data, and the second network model is used to generate summary information of the text data according to the annotated text; Before obtaining the text data to be processed, the method further includes: Obtain an event text training set, where the event text training set includes training texts, manually annotated named entities and events; Input the training text and the model prompt template into the first network model to obtain a first output information; Update the model parameters of the first network model based on the first output information and the manually annotated named entities and events; Wherein, both the Query layer and the Value layer of the first network model include a trainable LoRA module, and the LoRA module includes two layers of trainable fully connected layers; The event text training set further includes a manually written summary. After obtaining the event text training set, the method further includes: Input the training text and the first output information into the second network model to obtain a second output information; Update the model parameters of the second network model based on the second output information and the manually written summary; Wherein, the second network model includes a target Transformer network and a cross-attention module. The target Transformer network is used to encode the training text and the first output information, and the cross-attention module is used to learn the context relationship between the first output information and the training text.

2. The abstract generation method according to claim 1, wherein The updating the model parameters of the first network model based on the first output information and the manually annotated named entities and events includes: Calculate the model loss between the first output information and the manually annotated named entities and events by using a cross-entropy loss function; Update the model parameters of the first network model based on the model loss.

3. The abstract generation method according to claim 1, wherein After updating the model parameters of the second network model based on the second output information and the manually written summary, the method further includes: Determine target output information with an accuracy rate higher than a first threshold from the second output information; Add the target output information to the event text training set.

4. A summary generation device, characterized in that, Including: An acquisition module and a processing module; The acquisition module is used to obtain the text data to be processed, where the text data includes multiple words; The processing module is configured to input the text data and the model prompt template into a first network model to obtain an annotated text. The model prompt template is used to prompt the first network model to identify named entities and events in the input text. The annotated text includes annotation information for some of the multiple words. The processing module is further configured to input the text data and the annotated text into a second network model to obtain summary information of the text data. Among them, the first network model is used to annotate named entities and events in text data, and the second network model is used to generate summary information of text data based on the annotated text. The acquisition module is configured to acquire an event text training set, which includes training texts, manually annotated named entities and events. The processing module is configured to input the training text and the model prompt template into the first network model to obtain a first output information. The model parameters of the first network model are updated based on the first output information and the manually annotated named entities and events. Among them, both the Query layer and the Value layer of the first network model include a trainable LoRA module, and the LoRA module includes two trainable fully connected layers. The event text training set further includes a manually written summary. After acquiring the event text training set, the processing module is configured to input the training text and the first output information into the second network model to obtain a second output information. The model parameters of the second network model are updated based on the second output information and the manually written summary. Among them, the second network model includes a target Transformer network and a cross-attention module. The target Transformer network is used to encode the training text and the first output information, and the cross-attention module is used to learn the context relationship between the first output information and the training text.

5. The abstract generation device according to claim 4, wherein The processing module is configured to calculate the model loss between the first output information and the manually annotated named entities and events by using a cross-entropy loss function. The model parameters of the first network model are updated based on the model loss.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the summary generation method according to any one of claims 1 to 3.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps in the summary generation method according to any one of claims 1 to 3.

Images