Method and related device for generating public opinion text abstract based on pegarus and entity chain

Abstract

The method and related device for generating public opinion text summary based on PEGASUS and entity chain, comprising: obtaining public opinion text as training data, and preprocessing the training data; the preprocessed training data is processed again to obtain a pseudo summary, and the pseudo summary is segmented to obtain entity words; the summary entity chain is obtained by concatenating the entity words, and the summary entity chain and the pseudo summary are merged as label data; the training data and the label data are input into PEGASUS for training; the trained model first generates an entity chain based on the input text, and then generates a summary based on the input text and the entity chain, calculates the fidelity and diversity scores of the summary, and selects the optimal summary as the final output. The present application improves the diversity and fidelity of automatic generation of public opinion text summary, and solves the problems of insufficient diversity, content illusion and entity degradation in generating public opinion text summary.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of Internet information services, and relates to an opinion text abstract generation method based on PEGASUS and entity chains and a related device. BACKGROUND

[0002] In recent years, with the support of user-generated content functions in various Internet applications, especially the flourishing of information publishing platforms such as Internet social networks, blogs, microblogs and the like, the channels for individuals to express their own opinions are increasingly smooth and the means are increasingly convenient. However, when a network public opinion event occurs, the emotions, opinions and attitudes of the public held towards the event are quickly expressed in the form of text within a short period of time, and the quantity of text data on the Internet rapidly increases, so it is urgent to objectively and accurately know the emotions, opinions and attitudes of the public towards the public opinion event. In order to accurately find effective information in the massive Internet data when a public opinion event occurs, it is necessary to perform a "dimension reduction" processing on various types of text, and text abstract is an important means. At present, text abstracts can be divided into extractive abstracts and generative abstracts according to the output type: extractive text abstract and generative text abstract. The extractive abstract extracts key sentences and key words from the source document to form an abstract, and the abstract is entirely derived from the original text. The generative abstract allows new words and phrases to be generated to form an abstract according to the original text.

[0003] The extractive method selects key words and key sentences from the original text to form an abstract. This method naturally has a low error rate in grammar and syntax, ensuring a certain effect. Traditional extractive abstract methods use graph methods, clustering and the like to complete unsupervised abstracts, including Lead-3, textrank. The currently popular supervised abstract method first extracts various features at the word and sentence levels, such as the length of the sentence, the position of the sentence, the TF-IDF value of the words in the sentence and the like, and then uses machine learning algorithms to extract sentences. Or the extractive abstract based on neural network often models the problem as two types of tasks: sequence labeling and sentence ranking.

[0004] Although the extractive abstract has certain guarantees in grammar and syntax, it also faces certain problems, such as content selection errors, poor coherence and poor flexibility. The generative abstract attempts to generate an abstract by understanding the meaning of the original text, which may use words in the original text or new words to express, and is more flexible in word selection than the extractive abstract. In recent years, with the widespread application of large-scale language models, the generative text abstract technology has improved in terms of fidelity and fluency, but is slightly insufficient in terms of diversity, and is prone to content hallucination and entity degradation. SUMMARY

[0005] The application aims to provide an opinion text summary generation method based on PEGASUS and entity chains and related devices to solve the problems of insufficient diversity of generated opinion text summaries, content hallucination and entity degradation.

[0006] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0007] In the first aspect of the application, an opinion text summary generation method based on PEGASUS and entity chains is provided, comprising:

[0008] Obtaining opinion text as training data and preprocessing the training data;

[0009] Performing secondary processing on the preprocessed training data to obtain a pseudo summary, and performing word segmentation on the pseudo summary to obtain entity words;

[0010] Obtaining an entity chain of the summary by concatenating the entity words, and merging the entity chain of the summary and the pseudo summary as label data;

[0011] Sending the training data and the label data into PEGASUS for training;

[0012] PEGASUS, which stands for Pre-training with Extracted Gap-sentences for Abstractive Summarization, designs an unsupervised pre-training task (referred to as Gap Sentence Generation) for the text summary task, randomly covers several complete sentences in the document, and uses a Transformer encoder-decoder model to generate the covered sentences. The pre-training task matches the actual text summary task.

[0013] The trained model first generates an entity chain based on the input text, and then generates a summary based on the input text and the entity chain, calculates the fidelity and diversity scores of the summary, and selects the optimal summary as the final output.

[0014] Optionally, the preprocessing of the training data comprises:

[0015] Using opinion text as training data, cleaning the opinion text data, and removing symbols, expressions, special characters and URLs in the original data.

[0016] Optionally, the secondary processing of the preprocessed training data to obtain a pseudo summary and the word segmentation of the pseudo summary to obtain entity words comprises:

[0017] Using the TextRank algorithm to assist in selecting key sentences of the training data manually, shielding or deleting the key sentences in the training data, and taking the key sentences as the pseudo summary of the remaining sentences.

[0018] The pseudo summary is segmented by using the jieba segmentation algorithm, and the part-of-speech of the segmented words is screened, and the nouns, place names and time names are reserved as entity words.

[0019] Optionally, the summary entity chain is obtained by concatenating the entity words, and the summary entity chain and the pseudo summary are merged as label data, comprising:

[0020] The entity words of the same sentence are concatenated using '|', the sentence entity chain is composed, the entity chains of different sentences are concatenated using '|||', the entire summary entity chain is constructed, [CONTENT] is used as the header of the summary entity chain, [SUMMARY] is used as the division between the summary entity chain and the pseudo summary, and the summary entity chain and the pseudo summary are merged as label data.

[0021] Optionally, the model is trained, comprising:

[0022] The maximum length of the training data input into the model is set to 500, and the words exceeding the maximum length are discarded, and the words less than the maximum length are padded; during training, the learning rate lr is set to 0.00001, the training rounds epochs are 50 rounds, the batch size is 2, the gradient clipping is performed, the learning rate warm-up strategy is used, and the early stopping strategy is used, that is, when the model does not have an effect within 3 rounds, the model is saved, and the training is ended.

[0023] Optionally, in the process of generating the summary based on the input text and the entity chain, the generation of the entity chain is controlled by using core sampling, and the generation of the summary is controlled by using beam search.

[0024] Optionally, an over-generation strategy is used in the generation stage, and PEGASUS generates 20 summaries at the same time, a discriminator module is used to calculate the fidelity and diversity scores of the summaries respectively, and finally the final score of the summary is obtained by weighted summation, and the optimal summary is selected as the final output.

[0025] In the second aspect of the present application, a public opinion text summary generation system based on PEGASUS and entity chain is provided, characterized in that it comprises:

[0026] A data acquisition module is configured to acquire public opinion text as training data and pre-process the training data;

[0027] An entity word obtaining module is configured to perform secondary processing on the pre-processed training data to obtain a pseudo summary, and perform segmentation on the pseudo summary to obtain entity words;

[0028] A label data obtaining module is configured to obtain a summary entity chain by concatenating the entity words, and merge the summary entity chain and the pseudo summary as label data;

[0029] The training module is configured to input training data and label data into the PEGASUS for training.

[0030] The output module is configured to generate an entity chain based on the input text, generate an abstract based on the input text and the entity chain, calculate the fidelity and diversity scores of the abstract, and select an optimal abstract as a final output.

[0031] In a third aspect, the present application provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the public opinion text abstract generation method based on PEGASUS and an entity chain when executing the computer program.

[0032] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the public opinion text abstract generation method based on PEGASUS and an entity chain when executed by a processor.

[0033] Compared with the prior art, the present application has the following technical effects:

[0034] The public opinion text abstract generation method based on PEGASUS and an entity chain,

[0035] The entire method solves the problems of entity degradation and content hallucination in the text generation process by constructing an entity chain to generate an abstract; the generation process adopts over-generation, and the fidelity and diversity scores of the abstract are calculated to select an optimal abstract as a final output, which can improve the fidelity and diversity of the generated abstract.

[0036] Further, the public opinion text data is cleaned to reduce the interference of symbolic expressions, special characters, and URLs on model training. Although symbolic expressions, special characters, and URLs contain certain information, they account for a low proportion in the training data and can interfere with PEGASUS training.

[0037] Further, the entity words are concatenated to form an entity chain, and the entity chain contains entity information of the pseudo abstract; the entity chain is added to the training process, so that the model can better capture the entity information in the abstract and alleviate the problems of entity degradation and content hallucination.

[0038] Further, the abstract entity chain and the pseudo abstract are merged as label data, and the model first generates an entity chain and then generates an abstract when generating, which is equivalent to knowledge enhancement when generating an abstract, and solves the problem of entity degradation in generating an abstract.

[0039] Further, the hyperparameter setting of the trained model is set to the hyperparameter obtained by the inventor in practice, gradient clipping is performed to alleviate the problem of gradient explosion in the training process, learning rate preheating is adopted to improve the training stability, and the early stopping mechanism is adopted to prevent model overfitting.

[0040] Further, the beam search, also known as Beam Search, selects the k candidates with the maximum conditional probability from all combinations based on the output sequence of the previous step at each time step of text generation, as the candidate output sequence at the time step. Always keep k candidates. Finally, the optimal one is selected from the k candidates as the final output.

[0041] Nucleus sampling, also known as Nucleus sampling, constructs a minimum candidate set by fixing the probability of the candidate set and the proportion in the entire probability distribution, and randomly selects a candidate from the set as the output each time.

[0042] Using nucleus sampling to control entity chain generation, the generated entity chain is more diverse; using beam search to control summary generation makes the generated summary more faithful. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The flowchart of the present application.

[0044] Figure 2 It is a PEGASUS and entity chain based public opinion text summary generation method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application will be described in detail below in conjunction with the drawings and specific embodiments.

[0046] I. Training phase

[0047] S1, in the training phase, using public opinion text as training data, cleaning the public opinion text data, removing symbols, expressions, special characters and URLs in the original data;

[0048] S2, using TextRank algorithm assisted by manual selection of training data key sentences, shielding or deleting key sentences in the training data, and taking it as a pseudo summary of the remaining sentences;

[0049] S3, using jieba word segmentation algorithm to segment the pseudo summary, screening the part of speech, and retaining the noun, place name and time noun as entity words;

[0050] S4, using'|'to concatenate the entity words of the same sentence to form the sentence entity chain, and using'|||'to concatenate the entity chains of different sentences to form the entire summary entity chain;

[0051] S5, using [CONTENT] as the head of the summary entity chain, [SUMMARY] as the split of the summary entity chain and the pseudo summary, merging the summary entity chain and the pseudo summary as the label data;

[0052] S6, set the maximum length of the training data of the input model to 500, discard the words that exceed the length, and fill in the insufficient words; set the learning rate lr to 0.00001 during training, the training round epochs to 50 rounds, the batch size to 2, perform gradient clipping, use the learning rate warm-up strategy, and use the early stopping strategy, that is, when the model does not improve in 3 rounds, save the model and end the training; the hyperparameters and training strategies given here are only examples, and can be set according to the situation in actual application;

[0053] S7, send the training data and the label data into PEGASUS for training.

[0054] II. Generation phase

[0055] S8, in the generation phase, the model first generates an entity chain based on the input text, and then generates a summary based on the input text and the entity chain;

[0056] S9, use core sampling to control the generation of the entity chain to enrich the diversity of the generated summary, and set the core sampling hyperparameter p to 0.9;

[0057] S10, use beam search to control the generation of the summary to improve the fidelity of the generated summary, and set the beam search hyperparameter k to 10;

[0058] S11, the hyperparameters p and k given here are only examples, and can be set according to the situation in actual application;

[0059] S12, the generation phase also uses the over-generation strategy, PEGASUS generates 20 summaries at the same time, uses the discriminator module to calculate the fidelity and diversity scores of the summaries respectively, and finally gets the final score of the summary by weighted summation, and selects the optimal summary as the final output.

[0060] Using the over-generation strategy, the discriminator module is divided into a similarity calculation submodule and a diversity calculation submodule; the similarity calculation submodule uses the BERT model to project the over-generated summaries and the input text into the same vector space, uses the contrast learning strategy to make the similarity scores of the summaries similar to the input text higher and the similarity scores of the summaries not similar to the input text lower; the similarity calculation submodule outputs the similarity score s of the summary;

[0061] The diversity calculation submodule calculates the diversity score d of the over-generated summaries, and the calculation method is as follows:

[0062] The discriminator module finally outputs the summary considering the text fidelity and diversity, defines y = as + b as the final score, and selects the summary with the maximum y as the final output;

[0063] To ensure that s and d have the same dimension, normalization processing should be performed before weighted summation.

[0064] The above scheme provided by the embodiment of the present application performs knowledge enhancement through the prediction of entity chains, controls summary generation through the combination of kernel sampling and beam search, and finally selects the summary with the highest fidelity and the best diversity through the discriminator, thereby solving the problems of insufficient diversity, content hallucination and entity degradation of the generated public opinion text summary.

[0065] In another embodiment of the present application, a public opinion text summary generation system based on PEGASUS and entity chains is provided, which can be used to implement the above-mentioned public opinion text summary generation method based on PEGASUS and entity chains. Specifically, the system comprises:

[0066] A data acquisition module is configured to acquire public opinion text as training data and pre-process the training data.

[0067] An entity word obtaining module is configured to perform secondary processing on the pre-processed training data to obtain a pseudo summary, and perform word segmentation on the pseudo summary to obtain entity words.

[0068] A label data obtaining module is configured to obtain an abstract entity chain by concatenating the entity words, and merge the abstract entity chain and the pseudo summary as label data.

[0069] A training module is configured to input the training data and the label data into PEGASUS for training.

[0070] An output module is configured to generate an entity chain based on the input text after training the model, generate a summary based on the input text and the entity chain, calculate the fidelity and diversity scores of the summary, and select the optimal summary as the final output.

[0071] The division of modules in the embodiment of the present application is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used. In addition, the functional modules in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0072] In another embodiment of the present application, a computer device is provided, which comprises a processor and a memory, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method flow or corresponding function; the processor in the embodiment of the present application can be used for the operation of the public opinion text summarization generation method based on PEGASUS and entity chain.

[0073] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the computer device, and is used to store programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the computer device, and of course can also include the expansion storage medium supported by the computer device. The computer readable storage medium provides a storage space, and the storage space stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the public opinion text summarization generation method based on PEGASUS and entity chain in the above-mentioned embodiments.

[0074] Please refer to Figure 2

[0075] Application example: for the text "# Zhang Shuai first into the Cincinnati Open 16# Chinese player Zhang Shuai 17 in the women's tennis association Cincinnati Open second round of the game, with 63 and 64 defeated Russian player Alexander, the first time in the career of the 16 strong. Next, the Chinese gold flower will face the 2nd seed, Estonia player Kontaveit, to win a quarterfinal ticket."

[0076] Its pseudo summary is: Zhang Shuai first into the Cincinnati Open 16

[0077] Entity chain is: Zhang Shuai|Cincinnati|tennis|open

[0078] The entity chain and the pseudo summary form the tag data: [CONTENT] Zhang Shuai|Cincinnati|tennis|open [SUMMARY] Zhang Shuai first into the Cincinnati Open 16

[0079] Model generation: Zhang Shuai|Cincinnati|tennis|open [SUMMARY] Zhang Shuai first into the Cincinnati Open 16 [EOS]

[0080] The model generates 20 summaries, calculates the similarity score and the diversity score of the 20 summaries, and finally selects the summary with the highest score as the final output: Zhang Shuai first into the Cincinnati Open 16.

[0081] The above scheme provided by the embodiment of the present application is to enhance the knowledge by predicting the entity chain, control the summary generation by combining the kernel sampling and beam search, and finally select the summary with the highest loyalty and the best diversity by the discriminator, so as to solve the problems of insufficient diversity, content hallucination and entity degradation of the generated public opinion text summary.

[0082] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0083] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0084] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0085] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0086] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the technical solutions of the present application. Although the present application is described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or equivalent replacements without departing from the spirit and scope of the present application, and any modifications or equivalent replacements without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A public opinion text abstract generation method based on PEGASUS and entity chain, characterized in that, The method comprises the following steps: Obtain public opinion text as training data and preprocess the training data; Perform secondary processing on the preprocessed training data to obtain a pseudo summary, and perform word segmentation on the pseudo summary to obtain entity words; Obtain an abstract entity chain by concatenating the entity words, and merge the abstract entity chain and the pseudo summary as label data; Send the training data and the label data to PEGASUS for training; The trained model first generates an entity chain based on the input text, and then generates a summary based on the input text and the entity chain, calculates the fidelity and diversity scores of the over-generated summary, and selects the optimal summary as the final output; The method of obtaining an abstract entity chain by concatenating entity words and merging the abstract entity chain and the pseudo summary as label data comprises the following steps: Use ‘|’ to concatenate entity words in the same sentence to form a sentence entity chain, use ‘|||’ to concatenate different sentence entity chains to form an entire abstract entity chain, use [CONTENT] as the head of the abstract entity chain, and use [SUMMARY] as the division between the abstract entity chain and the pseudo summary, and merge the abstract entity chain and the pseudo summary as label data; In the process of generating an entity chain based on the input text and generating a summary based on the input text and the entity chain, kernel sampling is used to control the generation of the entity chain, and beam search is used to control the generation of the summary. 2.The PEGASUS and entity chain based public opinion text summarization generation method according to claim 1, characterized in that, The preprocessing of the training data comprises the following steps: Use public opinion text as training data, clean the public opinion text data, and remove symbols, expressions, special characters and URLs in the original data. 3.The PEGASUS and entity chain based public opinion text summarization generation method according to claim 1, characterized in that, The secondary processing of the preprocessed training data to obtain a pseudo summary and the word segmentation of the pseudo summary to obtain entity words comprises the following steps: Use the TextRank algorithm to select key sentences of the training data manually, shield or delete the key sentences in the training data, and use the key sentences as the pseudo summary of the remaining sentences; Use the jieba word segmentation algorithm to perform word segmentation on the pseudo summary, filter the word types, and retain the noun, place name and time name as entity words. 4.The PEGASUS and entity chain based public opinion text summarization generation method according to claim 1, characterized in that, The training model comprises the following steps: Set the maximum length of the training data input into the model to 500, discard the words that exceed the length, and fill in the words that are insufficient; set the learning rate lr to 0.00001 during training, set the training rounds epochs to 50 rounds, set the batch size to 2, perform gradient clipping, use the learning rate warm-up strategy, and use the early stopping strategy, that is, when the model does not improve within 3 rounds, save the model and end the training. 5.The PEGASUS and entity chain based public opinion text summarization generation method according to claim 1, characterized in that, In the generation stage, the over-generation strategy is used, PEGASUS generates 20 summaries at the same time, uses a discriminator module to calculate the fidelity and diversity scores of the summaries, finally obtains the final score of the summaries by weighted summation, and selects the optimal summary as the final output.

6. The system for generating public opinion text summary based on PEGASUS and entity chain, characterized in that, The method comprises the following steps: A data acquisition module is configured to obtain public opinion text as training data and preprocess the training data; An entity word obtaining module is configured to perform secondary processing on the preprocessed training data to obtain a pseudo summary, and perform word segmentation on the pseudo summary to obtain entity words; A label data obtaining module is configured to obtain an abstract entity chain by concatenating the entity words, and merge the abstract entity chain and the pseudo summary as label data; The training module is configured to input training data and label data into the PEGASUS for training. The output module is configured to generate an entity chain based on the input text, generate an abstract based on the input text and the entity chain, calculate the fidelity and diversity scores of the abstract, and select the optimal abstract as the final output. The abstract entity chain is obtained by concatenating entity words in series, and the abstract entity chain and the pseudo abstract are merged as label data, including: The entity words in the same sentence are concatenated using '|', the sentence entity chain is composed, the entity chains of different sentences are concatenated using '|||', the entire abstract entity chain is constructed, [CONTENT] is used as the header of the abstract entity chain, [SUMMARY] is used as the division between the abstract entity chain and the pseudo abstract, and the abstract entity chain and the pseudo abstract are merged as label data; During the process of generating an abstract based on the input text and the entity chain, kernel sampling is used to control the generation of the entity chain, and beam search is used to control the generation of the abstract.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the public opinion text abstract generation method based on PEGASUS and an entity chain according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the steps of the public opinion text abstract generation method based on PEGASUS and an entity chain according to any one of claims 1 to 5.

Images