Abstract generation method, device, electronic device and storage medium

By combining text sentences and using pre-trained abstract recognition model, candidate digests with high consistency are identified, which solves the problem of inconsistent summary and text, and improves the accuracy and user experience of the summary.

CN114595684BActive Publication Date: 2024-11-29BEIJING SANKUAI ONLINE TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210129446.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-11
Publication Date
2024-11-29
Estimated Expiration
2042-02-11

AI Technical Summary

Technical Problem

In the prior art, the abstract is inconsistent with the text content, resulting in a decrease in user experience and forming a ‘title party’.

Method used

By obtaining the pending text, combining multiple sentences to generate candidate digests, and using a pre-trained digest recognition model, the most suitable digest is identified based on the consistency between the candidate digest and the text. The model improves the accuracy of the abstract by comparative learning training, considering the consistency between the text sample and the reference digest sample, and the consistency between the candidate digest and the text sample.

Benefits of technology

Improve the consistency between the summary and text content, so that the generated summary better characterizes the content of the pending text, avoiding information redundancy and missing key information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114595684B_ABST
    Figure CN114595684B_ABST
Patent Text Reader

Abstract

The present application provides a summary generation method, device, electronic device and storage medium, which relates to the field of information processing technology and aims to improve the consistency between the summary and the content of the text. The method includes: obtaining a text to be processed; combining at least two sentences in the text to be processed to generate multiple candidate summaries; using a pre-trained summary recognition model, respectively determining the consistency between the multiple candidate summaries and the text to be processed, and identifying the summary of the text to be processed from the multiple candidate summaries according to the consistency corresponding to each of the multiple candidate summaries; wherein the pre-trained summary recognition model is obtained by comparative learning and training a preset model based on the consistency between the text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a summary generation method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of the Internet, the text data in the network has exploded. In order to provide users with more user-friendly information, it is usually pushed to users to establish a stable user expectation and feedback mechanism. The summary is a bridge connecting users and texts. The summary can directly guide users to read the content of the text. However, if the description of the summary is inconsistent with the content of the text, it is easy to form a "clickbait" and thus reduce the user experience. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention provide a method, device, electronic device and storage medium for generating a summary, so as to improve the consistency between the summary and the text content.

[0004] According to a first aspect of an embodiment of the present invention, a method for generating a summary is provided, the method comprising:

[0005] Get the text to be processed;

[0006] Combining at least two sentences in the text to be processed to generate multiple candidate summaries;

[0007] Determining the consistency between the plurality of candidate summaries and the text to be processed respectively through a pre-trained summary recognition model, and identifying the summary of the text to be processed from the plurality of candidate summaries according to the consistency corresponding to each of the plurality of candidate summaries;

[0008] The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between a text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.

[0009] Optionally, before combining at least two sentences in the text to be processed to generate a plurality of candidate summaries, the method further includes performing at least one of the following preprocessing on the text to be processed:

[0010] De-duplication of sentences contained in another sentence in the text to be processed;

[0011] Pre-combining any two sentences in the text to be processed, and when the ratio of the length of the common text of the two sentences involved in the pre-combination to the length of the single sentence is greater than a preset ratio, in the step of generating multiple candidate summaries, giving up combining the two sentences involved in the pre-combination;

[0012] For two sentences in the text to be processed that are arranged in sequence and contain the same text at the beginning and the end, deduplication of the same text contained in one of the two sentences;

[0013] Combining at least two sentences in the text to be processed to generate multiple candidate summaries, including:

[0014] At least two preprocessed sentences in the text to be processed are combined to generate a plurality of candidate summaries.

[0015] Optionally, the summary recognition model is trained according to the following steps:

[0016] Acquire a text sample, a reference summary sample of the text sample, and a plurality of candidate summary samples of the text sample, and input them into a preset model;

[0017] Determine the text-candidate summary consistency between the text sample and each of its candidate summary samples by using the preset model, and determine the text-reference summary consistency between the text sample and its reference summary sample;

[0018] Determine the preset model after the first contrastive learning and the second contrastive learning as the summary recognition model; wherein the first contrastive learning is a contrastive learning between the reference summary sample of the text sample and each candidate summary sample, performed by the preset model according to the text-reference summary consistency and the text-candidate summary consistency corresponding to each candidate summary sample;

[0019] The second contrastive learning is contrastive learning between every two candidate summary samples of the text samples performed by the preset model according to the text-candidate summary consistency corresponding to each of the plurality of candidate summary samples.

[0020] Optionally, it also includes:

[0021] Determining text similarities between the plurality of candidate summary samples and the reference summary sample respectively;

[0022] In the order of text similarity from large to small, corresponding serial numbers are added to the multiple candidate summary samples respectively; the loss value of the second contrastive learning is determined according to the following steps: when the consistency between the text with the preceding serial number and the candidate summary is less than the consistency between the text with the succeeding serial number and the candidate summary, the loss value is determined; or, when the consistency between the text with the preceding serial number and the candidate summary is greater than the consistency between the text with the succeeding serial number, and the difference between the consistency between the two is less than a calibration threshold, the loss value is determined, wherein the calibration threshold is proportional to the absolute value of the serial number difference; the determined loss values ​​are accumulated to obtain the loss value of the second contrastive learning.

[0023] Optionally, determining text similarities between the plurality of candidate summary samples and the reference summary sample respectively includes:

[0024] For each candidate summary sample, the text similarity between the multiple candidate summary samples and the reference summary sample is determined according to the number of phrases respectively included in the candidate summary sample and the reference summary sample, the number of phrases included in the same text between the two, and a preset length control factor.

[0025] Optionally, the consistency between the text sample and its reference summary sample is determined according to the following steps:

[0026] Determine the vector representation of the text sample and its reference summary sample respectively through the preset model;

[0027] Determine the cosine similarity between the vector representations of the text sample and its reference summary sample by using the preset model, and determine the cosine similarity as the consistency between the text sample and its reference summary sample;

[0028] The agreement between a text sample and each of its candidate summary samples is determined by following the steps below:

[0029] Determine the vector representation of the text sample and each candidate summary sample respectively through the preset model;

[0030] The cosine similarity between the vector representations of the text sample and each candidate summary sample is determined by the preset model, and the cosine similarity is determined as the consistency between the text sample and each candidate summary sample.

[0031] A second aspect of an embodiment of the present invention provides a summary generation device, the device comprising:

[0032] An acquisition module is used to obtain the text to be processed;

[0033] A generating module, used for combining at least two sentences in the text to be processed to generate a plurality of candidate summaries;

[0034] A recognition module, used to determine the consistency between the multiple candidate summaries and the text to be processed respectively through a pre-trained summary recognition model, and to recognize the summary of the text to be processed from the multiple candidate summaries according to the consistency corresponding to each of the multiple candidate summaries;

[0035] The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between a text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.

[0036] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the summary generation method described in the first aspect of the embodiment is implemented.

[0037] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the summary generation method described in the first aspect of the embodiment are implemented.

[0038] The embodiments of the present invention include the following advantages:

[0039] In this embodiment, after obtaining the text to be processed, at least two sentences in the text to be processed are combined to generate multiple candidate summaries, and then a pre-trained summary recognition model is used to determine the consistency of the multiple candidate summaries with the text to be processed, and the summary of the text to be processed is identified from the multiple candidate summaries according to the consistency corresponding to each of the multiple candidate summaries; and the pre-trained summary recognition model is obtained by comparative learning and training of a preset model according to the consistency between the text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.

[0040] By combining at least two sentences in the text to be processed to obtain a candidate summary, and then processing and analyzing the consistency between the candidate summary and the content of the text, unlike the method of analyzing the probability of a sentence as a summary and then combining them, this solution converts the summary from the sentence level to the summary level, avoiding the problem of information redundancy and missing sentences containing key information; when training the summary recognition model, the consistency between the text sample and the reference summary sample, as well as the consistency between each candidate summary sample and the text sample are considered at the same time, so that the candidate summary with the content consistency of the text to be processed can be more accurately identified. The technical solution of this application can improve the consistency of the content of the summary and the text, so that the generated summary can better represent the content of the text to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] FIG1 is a flowchart of a method for generating a summary according to an embodiment of the present invention;

[0043] FIG2 is a flowchart of the steps of training a summary recognition model in an embodiment of the present invention;

[0044] FIG. 3 is a schematic diagram of the structure of a summary generation device in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0046] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0047] In order to solve the problem of consistency between summary and text, this application proposes a summary generation method, which can improve the content consistency between summary and text, so that the generated summary can better represent the content of the text to be processed.

[0048] 1 , a flowchart of a method for generating a summary in an embodiment of the present invention is shown. As shown in FIG1 , the method for generating a summary may specifically include the following steps:

[0049] S101, obtaining text to be processed.

[0050] The text to be processed refers to the text that does not carry the abstract created by the publisher, or the text of the abstract created by the publisher does not meet the requirements.

[0051] S102: Combine at least two sentences in the text to be processed to generate multiple candidate summaries.

[0052] In a feasible implementation of this embodiment, a rule-based sentence segmentation method may be used to segment sentences in the text to be processed according to the rules, and unreasonable short sentences may be spliced ​​using a sliding window to generate a final plurality of sentences.

[0053] Then, the multiple segmented sentences are combined. Combining means combining different sentences together as candidate summary samples. At least two sentences are combined during the combination. The number of sentences in the combination can be customized or determined using a greedy algorithm. For the sake of convenience, in this embodiment, the number of sentences in the combination is determined to be two.

[0054] The combination may be a combination of two sentences arranged in text order, or a combination of two random sentences, thereby obtaining multiple candidate summaries.

[0055] In a feasible implementation manner, before combining at least two sentences in the text to be processed to generate a plurality of candidate summaries, the method further includes performing at least one of the following preprocessing on the text to be processed:

[0056] A1: Remove duplicate sentences contained in another sentence in the text to be processed.

[0057] Some sentences in the text to be processed are repeated, so the multiple sentences that have been segmented are deduplicated. A specific deduplication method is to deduplicate a sentence that is included in another sentence.

[0058] A2: Pre-combining any two sentences in the text to be processed, and when the ratio of the length of the common text of the two sentences involved in the pre-combination to the length of a single sentence is greater than a preset ratio, in the step of generating multiple candidate summaries, giving up combining the two sentences involved in the pre-combination.

[0059] Enumerate the pre-combinations of any two sentences, and then find the length of the same text contained in the two sentences. If the length of the same text is greater than a certain proportion, for example, greater than 50% of the length of any of the two sentences, it can be represented that the meanings described by the two sentences are similar, and the two sentences participating in the pre-combination are abandoned for combination.

[0060] A3: For two sentences in the text to be processed that are arranged in sequence and contain the same text at the beginning and the end, duplicates are removed from the same text contained in one of the two sentences.

[0061] For two sentences that are arranged smoothly, if the end of the sentence that is arranged first and the beginning of the sentence that is arranged second contain the same text, then the same text contained in one of the two sentences is deduplicated.

[0062] When generating multiple candidate summaries, at least two preprocessed sentences in the text to be processed are combined to reduce sentences or texts with repeated meanings or invalid meanings, so that the sentences in the combination for generating the candidate summaries can better or more cover the content of the text to be processed.

[0063] S103 , determining respectively the consistency between the plurality of candidate summaries and the text to be processed by using a pre-trained summary recognition model, and identifying a summary of the text to be processed from the plurality of candidate summaries according to the consistency corresponding to each of the plurality of candidate summaries.

[0064] The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between a text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.

[0065] Text samples include historically published texts with summaries created by the publisher, and their summaries are reference summary samples. Text samples can also be historically published texts without summaries. When training the summary recognition model, for these texts without summaries, summaries are manually written based on the content of the text as reference summary samples for the text samples. It is generally believed that the summary written based on the content of the text has the highest consistency with the text.

[0066] The summary recognition model identifies the consistency of each candidate summary based on the consistency between the reference summary sample and the text sample, and then can identify the candidate summary with the best consistency among multiple candidate summaries. The summary recognition model can quickly and accurately identify the summary of the text to be processed from multiple candidate summaries.

[0067] In actual implementation, multiple candidate summaries are input into the summary recognition model, which can determine the consistency between each candidate summary and the text to be processed, and select the summary of the text to be processed from the multiple candidate summaries based on the consistency of each candidate summary.

[0068] This method combines at least two sentences in the text to be processed to obtain a candidate summary, and then processes and analyzes the consistency between the candidate summary and the content of the text. Unlike the method of analyzing the probability of a sentence as a summary and then combining them, this solution converts the summary from the sentence level to the summary level, avoiding the problem of information redundancy and missing sentences containing key information; when training the summary recognition model, it also considers the consistency between the text sample and the reference summary sample, as well as the consistency between each candidate summary sample and the text sample, so that the candidate summary with the content consistency of the text to be processed can be more accurately identified. The technical solution of this application can improve the consistency of the content of the summary and the text, so that the generated summary can better represent the content of the text to be processed.

[0069] 2 , a flowchart of a training method for a summary recognition model in an embodiment of the present invention is shown. As shown in FIG2 , the training method may specifically include the following steps:

[0070] S201: Acquire a text sample, a reference summary sample of the text sample, and a plurality of candidate summary samples of the text sample.

[0071] In this embodiment, after obtaining the text sample, a corresponding reference summary sample is manually written according to the content of the text sample, or a text carrying a summary prepared by the publisher is directly obtained as the text sample, and the summary it carries is used as the reference summary sample. The consistency between the reference summary sample and the text sample is the highest by default.

[0072] For the text sample, the process of determining the candidate summary sample includes sentence segmentation and combination, wherein the sentence segmentation and combination method is consistent with the sentence segmentation and combination method in step S102, and thus will not be described in detail here.

[0073] S202: Determine text similarities between the plurality of candidate summary samples and the reference summary sample, and add corresponding serial numbers to the plurality of candidate summary samples in descending order of text similarities.

[0074] During the training phase, after generating multiple candidate summary samples, they need to be sorted according to their similarity to the reference summary samples in order to train a model that can distinguish the quality of the candidate summary samples.

[0075] In a feasible implementation, a similarity index commonly used in the field of automatic summarization and translation, ROUGE (ECALL-ORIENTED UNDERSTUDY FOR GISTING EVALUATION), is used to measure the similarity between multiple candidate summary samples and a reference summary sample.

[0076] ROUGE-F calculates the similarity between candidate summary samples and reference summary samples by counting the proportion of identical tuples to the number of tuples in each of them. That is, ROUGE-F is similar to the F-Measure in machine learning. The F-measure is the weighted harmonic mean of precision and recall. In this embodiment, the proportion of identical tuples to candidate summary sample tuples is precision, and the proportion to reference summary sample tuples is recall. Specifically:

[0077]

[0078] in:

[0079]

[0080]

[0081] The tuple in the formula includes words and phrases. The calculated ROUGE-F is the text similarity between the candidate summary sample and the reference summary sample. Then, multiple candidate summary samples are sorted in descending order according to the text similarity, and corresponding serial numbers are added to the multiple candidate summary samples. For example, the candidate summary sample with the first text similarity is ranked 1, the candidate summary sample with the second text similarity is ranked 2, and so on. The multiple candidate summary samples are ranked in sequence.

[0082] Since the text summary will be folded or staged if it is too long during the display process, it is also necessary to control the length of the summary. In a feasible implementation, determining the text similarity between multiple candidate summary samples and the reference summary sample includes:

[0083] For each candidate summary sample, the text similarity between the multiple candidate summary samples and the reference summary sample is determined according to the number of phrases respectively included in the candidate summary sample and the reference summary sample, the number of phrases included in the same text between the two, and a preset length control factor.

[0084] For example, when using the similarity index ROUGE to measure the similarity between multiple candidate summary samples and the reference summary sample, the length control factor β is introduced, and the formula of ROUGE-F is as follows:

[0085]

[0086] The problem of long candidate summary samples being ranked higher can be alleviated by using the length control factor β. By presetting the specific value of the length control factor β, the summary recognition model's preference for length can be guided.

[0087] S203: Inputting the text sample, the reference summary sample of the text sample, and the sorted candidate summary samples of the text sample into a preset model.

[0088] In this implementation, a preset model is trained based on the text sample, the reference summary sample, and the sorted plurality of candidate summary samples, thereby obtaining a summary recognition model.

[0089] S204: Determine the text-candidate summary consistency between the text sample and each of its candidate summary samples through the preset model, and determine the text-reference summary consistency between the text sample and its reference summary sample.

[0090] Specifically, the consistency between the text sample and its reference summary sample is determined according to the following steps:

[0091] Determine the vector representation of the text sample and its reference summary sample respectively through the preset model;

[0092] The cosine similarity between the vector representations of the text sample and its reference summary sample is determined by the preset model, and the cosine similarity is determined as the consistency between the text sample and its reference summary sample.

[0093] The agreement between a text sample and each of its candidate summary samples is determined by following the steps below:

[0094] Determine the vector representation of the text sample and each candidate summary sample respectively through the preset model;

[0095] The cosine similarity between the vector representations of the text sample and each candidate summary sample is determined by the preset model, and the cosine similarity is determined as the consistency between the text sample and each candidate summary sample.

[0096] Regarding the method of determining the vector representation of the text sample and its reference summary sample and determining the vector representation of the text sample and each candidate summary sample, in a feasible implementation, the preset model includes a twin BERT (Bidirectional Encoder Representation from Transformers, a pre-trained language representation model) network to obtain the vector representation. The twin network used in this implementation includes two sub-networks with the same structure and shared parameters, which are used to extract the vector representation of the text sample, the reference summary sample, and multiple candidate summary samples respectively.

[0097] Because the CLS position in the embedding vector output by the twin BERT network usually represents the overall vector representation, in this embodiment, the CLS position of the embedding vector of the text sample is used as the vector representation of the text sample; the CLS position of the embedding vector of each of the multiple candidate summary samples is used as the vector representation of each of the multiple candidate summary samples, and the CLS position of the embedding vector of the reference summary sample is used as the vector representation of the reference summary sample.

[0098] In this implementation, the cosine similarity in the vector space is used to measure the consistency. For example, the consistency between the reference summary sample or the candidate summary sample and the text sample can be expressed as:

[0099]

[0100] In the formula, the text is a text sample, and the summary can be a reference summary sample corresponding to the text sample, or a candidate summary sample generated according to the text sample.

[0101] By setting the target loss threshold, the preset can be trained multiple times to ensure that the preset model obtains a consistent

[0102] According to the text candidate summary consistency between the text sample and each of its candidate summary samples, and the text-reference summary consistency between the text sample and its reference summary sample, the preset model is trained by comparative learning to obtain a summary recognition model.

[0103] In the process of contrastive learning, if there are n candidate summary samples, there should be n+1 parameters involved in the learning, including:

[0104]

[0105] S205: Perform a first comparative learning, wherein the first comparative learning is a comparative learning between the reference summary sample of the text sample and each candidate summary sample performed by the preset model according to the text-reference summary consistency and the text-candidate summary consistency corresponding to each candidate summary sample.

[0106] S206: Performing a second comparative learning, wherein the second comparative learning is comparative learning between every two candidate summary samples of the text samples performed by the preset model according to the text-candidate summary consistency corresponding to each of the plurality of candidate summary samples.

[0107] Contrastive learning is a popular self-supervised learning method in the field of machine learning. The main purpose of contrastive learning is to learn to distinguish samples of different categories, such as finding texts that belong to the same category as a specific text in many paragraphs of text.

[0108] In the embodiment of the present application, the reference summary sample has the highest consistency with the text sample, so the preset model must first distinguish the reference summary text from the candidate summary text, that is, the preset model performs the first comparative learning; in addition, the content consistency of the candidate summary sample that is closer to the reference summary sample should be higher, and this difference is proportional to the similarity between the candidate summary sample and the reference summary sample, and then the preset model is subjected to the second comparative learning so that the model can be used to train to distinguish different candidate summary samples.

[0109] The first contrastive learning is based on contrastive learning between each candidate summary sample and the reference summary sample. The input of the preset model is the consistency between the text and the reference summary, and the consistency between multiple different texts and candidate summaries. The first loss value during the training process is The formula is as follows:

[0110]

[0111] in, is the text-reference summary agreement expressed as cosine similarity; It is the cosine similarity value of the consistency between the text of the i-th candidate summary sample and the candidate summary and the content vector, expressed by cosine similarity. represent Must be big.

[0112] Since the reference summary sample and the text sample have the best consistency, that is, the corresponding vector cosine value is the largest, if , it means that the model does not distinguish the relationship between "text-reference summary" and "text-candidate summary" well, so a loss value will be generated. Otherwise, no loss value will be generated. Finally, through training, the preset model can identify the reference summary sample and the text sample with the highest consistency, that is, the vector of the reference summary sample is closest to the vector of the text sample.

[0113] The second contrastive learning is based on contrastive learning between different candidate summary samples. The input of the preset model is the consistency of multiple different text-candidate summaries.

[0114] Multiple candidate summary samples have been sorted before, and the consistency corresponding to the candidate summary sample with the earlier serial number should be greater than the consistency corresponding to the candidate summary sample with the later serial number, and the greater the absolute value of the difference between the serial numbers of two candidate summary samples, the greater the difference in the consistency of the text candidate summaries corresponding to the two candidate summary samples should be.

[0115] Therefore, when learning is performed, when the consistency between the text with the preceding sequence number and the candidate summary is less than the consistency between the text with the following sequence number and the candidate summary, the loss value is determined;

[0116] Or, when the consistency between the text with the preceding sequence number and the candidate summary is greater than the consistency between the text with the following sequence number, and the difference between the two consistency is less than a calibration threshold, the loss value is determined, wherein the calibration threshold is proportional to the absolute value of the sequence number difference.

[0117] The determined loss values ​​are accumulated to obtain the loss value of the second contrastive learning. Specifically, the second loss value during the training process The formula is as follows:

[0118]

[0119] Among them, the greater the difference between the two candidate summary samples, the greater the difference between the two candidate summary samples; since the candidate summary samples are sorted in advance according to the similarity with the reference summary sample, the candidate summary samples with smaller numbers are closer to the reference summary sample than the candidate summary samples with larger numbers. Therefore, if , which means that the model does not distinguish the consistency between the i-th candidate summary sample and the j-th candidate summary sample and the content, and a loss value will be generated. Multiple loss values ​​are accumulated to obtain the loss value of the second contrastive learning. Otherwise, no loss will be generated. Finally, through training, the preset model can identify the difference in consistency between different candidate summary samples and text samples.

[0120] S207: Determine the preset model after the first contrast learning and the second contrast learning as the summary recognition model.

[0121] Through the first contrastive learning and the second contrastive learning, the loss values ​​obtained from the two contrastive learning are added together and used as the final loss value to train the model to obtain a summary recognition model.

[0122] The embodiment of the present application proposes to explicitly model the consistency between the summary and the text, and then train a summary recognition model through the first contrastive learning and the second contrastive learning training method; when it is necessary to generate a summary for the text to be processed, the pre-trained summary recognition model is used to combine at least two sentences in the text to be processed to generate multiple candidate summaries, determine the vector representation of each of the multiple candidate summaries and the vector representation of the text to be processed, and then determine the cosine similarity between the vector representation of each of the multiple candidate summaries and the vector representation of the text to be processed, so that the summary of the text to be processed can be identified from the multiple candidate summaries, and the selected summary has the best consistency with the text to be processed.

[0123] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0124] FIG3 is a schematic diagram of the structure of a summary generation device according to an embodiment of the present invention. As shown in FIG3 , a summary generation device, a second aspect of an embodiment of the present invention provides a summary generation device, the device comprising:

[0125] An acquisition module 100 is used to acquire the text to be processed;

[0126] A generating module 200, configured to combine at least two sentences in the text to be processed to generate a plurality of candidate summaries;

[0127] The recognition module 300 is used to determine the consistency between the multiple candidate summaries and the text to be processed respectively through a pre-trained summary recognition model, and to recognize the summary of the text to be processed from the multiple candidate summaries according to the consistency corresponding to each of the multiple candidate summaries;

[0128] The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between a text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples.

[0129] Optionally, the device further comprises at least one of the following modules:

[0130] A first preprocessing module, used for removing duplicates from a sentence contained in another sentence in the text to be processed;

[0131] A second preprocessing module is used to pre-combine any two sentences in the text to be processed, and when the ratio of the length of the same text of the two sentences involved in the pre-combination to the length of the single sentence is greater than a preset ratio, in the step of generating multiple candidate summaries, the combination of the two sentences involved in the pre-combination is abandoned;

[0132] The third preprocessing module is used to remove duplicates of the same text contained in one of two sentences that are arranged in sequence and contain the same text at the beginning and the end in the text to be processed.

[0133] Optionally, the device comprises:

[0134] A sample acquisition module, used to acquire a text sample, a reference summary sample of the text sample, and a plurality of candidate summary samples of the text sample, and input them into a preset model;

[0135] A sample processing module, used to determine the text-candidate summary consistency between the text sample and each of its candidate summary samples through the preset model, and determine the text-reference summary consistency between the text sample and its reference summary sample;

[0136] A model determination module, configured to determine a preset model after the first contrast learning and the second contrast learning as the summary recognition model;

[0137] The first contrastive learning is a contrastive learning between the reference summary sample of the text sample and each candidate summary sample, performed by the preset model according to the text-reference summary consistency and the text-candidate summary consistency corresponding to each candidate summary sample;

[0138] The second contrastive learning is contrastive learning between every two candidate summary samples of the text samples performed by the preset model according to the text-candidate summary consistency corresponding to each of the plurality of candidate summary samples.

[0139] Optionally, the device further comprises:

[0140] A similarity determination module, used to determine the text similarities between the plurality of candidate summary samples and the reference summary sample respectively;

[0141] A sorting module, used to add corresponding serial numbers to the plurality of candidate summary samples in descending order of text similarity;

[0142] The second contrastive learning module includes:

[0143] A loss value determination unit is used to determine the loss value when the consistency degree between the text with the preceding sequence number and the candidate summary is less than the consistency degree between the text with the succeeding sequence number and the candidate summary; or, when the consistency degree between the text with the preceding sequence number and the candidate summary is greater than the consistency degree between the text with the succeeding sequence number and the difference between the consistency degrees between the two is less than a calibration threshold, wherein the calibration threshold is proportional to the absolute value of the sequence number difference;

[0144] The loss value accumulation unit is used to accumulate the determined loss values ​​to obtain the loss value of the second comparative learning.

[0145] Optionally, the similarity determination module includes:

[0146] The similarity determination unit is used to determine, for each candidate summary sample, the text similarity between the plurality of candidate summary samples and the reference summary sample according to the number of phrases respectively included in the candidate summary sample and the reference summary sample, the number of phrases included in the common texts of the two, and a preset length control factor.

[0147] Optionally, the sample processing module includes:

[0148] A first generating unit, configured to determine the vector representation of the text sample and its reference summary sample respectively through the preset model;

[0149] A first determining unit, configured to determine the cosine similarity between the vector representations of the text sample and its reference summary sample through the preset model, and determine the cosine similarity as the consistency between the text sample and its reference summary sample;

[0150] A second generating unit, configured to determine the vector representation of the text sample and each candidate summary sample respectively through the preset model;

[0151] The second determining unit is used to determine the cosine similarity between the vector representations of the text sample and each candidate summary sample through the preset model, and determine the cosine similarity as the consistency between the text sample and each candidate summary sample.

[0152] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.

[0153] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the summary generation method described in the first aspect of the embodiment is implemented.

[0154] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the summary generation method described in the first aspect of the embodiment are implemented.

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0156] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, apparatuses or computer program products. Therefore, embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The form of the product.

[0158] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0162] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0163] The above is a detailed introduction to a summary generation method, device, electronic device and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating a summary, characterized in that: The method comprises: Get the text to be processed; Combining at least two sentences in the text to be processed to generate multiple candidate summaries; Determining the consistency between the plurality of candidate summaries and the text to be processed respectively through a pre-trained summary recognition model, and identifying the summary of the text to be processed from the plurality of candidate summaries according to the consistency corresponding to each of the plurality of candidate summaries; The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between the text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples; The summary recognition model is trained according to the following steps: Acquire a text sample, a reference summary sample of the text sample, and a plurality of candidate summary samples of the text sample, and input them into a preset model; Determining the text-candidate summary consistency between the text sample and each of its candidate summary samples through the preset model, and determining the text-reference summary consistency between the text sample and its reference summary sample; Determining the preset model after the first contrastive learning and the second contrastive learning as the summary recognition model; Among them, the first contrastive learning is the contrastive learning between the reference summary sample of the text sample and each candidate summary sample performed by the preset model according to the text-reference summary consistency and the text-candidate summary consistency corresponding to each candidate summary sample; the second contrastive learning is the contrastive learning between every two candidate summary samples of the text sample performed by the preset model according to the text-candidate summary consistency corresponding to each of the multiple candidate summary samples.

2. The method according to claim 1, characterized in that: Before combining at least two sentences in the text to be processed to generate multiple candidate summaries, the method further includes performing at least one of the following preprocessing on the text to be processed: removing duplicates from a sentence in the text to be processed that is contained in another sentence; pre-combining any two sentences in the text to be processed, and when the ratio of the length of the same text in the two sentences involved in the pre-combination to the length of a single sentence is greater than a preset ratio, giving up combining the two sentences involved in the pre-combination in the step of generating multiple candidate summaries; for two sentences in the text to be processed that are arranged in sequence and contain the same text at the beginning and the end, removing duplicates from the same text contained in one of the two sentences; Combining at least two sentences in the text to be processed to generate multiple candidate summaries, including: At least two preprocessed sentences in the text to be processed are combined to generate a plurality of candidate summaries.

3. The method according to claim 2, characterized in that Also includes: Determining text similarities between the plurality of candidate summary samples and the reference summary sample respectively; Adding corresponding serial numbers to the plurality of candidate summary samples respectively in descending order of text similarity; The loss value of the second contrastive learning is determined according to the following steps: When the consistency between the text with the previous sequence number and the candidate summary is less than the consistency between the text with the next sequence number and the candidate summary, a loss value is determined; Or, when the consistency between the text with the preceding sequence number and the candidate summary is greater than the consistency between the text with the following sequence number, and the difference between the consistency between the two is less than a calibration threshold, the loss value is determined, wherein the calibration threshold is proportional to the absolute value of the sequence number difference; The determined loss values ​​are accumulated to obtain the loss value of the second comparative learning.

4. The method according to claim 3, characterized in that Determining text similarities between the plurality of candidate summary samples and the reference summary sample respectively includes: For each candidate summary sample, the text similarity between the multiple candidate summary samples and the reference summary sample is determined according to the number of phrases respectively included in the candidate summary sample and the reference summary sample, the number of phrases included in the same text between the two, and a preset length control factor.

5. The method according to any one of claims 1 to 4, characterized in that: The degree of consistency between the text sample and its reference summary sample is determined according to the following steps: Determine the vector representation of the text sample and its reference summary sample respectively through the preset model; Determine the cosine similarity between the vector representations of the text sample and its reference summary sample by using the preset model, and determine the cosine similarity as the consistency between the text sample and its reference summary sample; The agreement between a text sample and each of its candidate summary samples is determined by following the steps below: Determine the vector representation of the text sample and each candidate summary sample respectively through the preset model; The cosine similarity between the vector representations of the text sample and each candidate summary sample is determined by the preset model, and the cosine similarity is determined as the consistency between the text sample and each candidate summary sample.

6. A summary generation device, characterized in that: The device comprises: An acquisition module is used to obtain the text to be processed; A generating module, used for combining at least two sentences in the text to be processed to generate a plurality of candidate summaries; A recognition module, used to determine the consistency between the multiple candidate summaries and the text to be processed respectively through a pre-trained summary recognition model, and to recognize the summary of the text to be processed from the multiple candidate summaries according to the consistency corresponding to each of the multiple candidate summaries; The pre-trained summary recognition model is obtained by performing comparative learning training on a preset model according to the consistency between the text sample and its reference summary sample, and the consistency between the text sample and each of its candidate summary samples; The summary recognition model is trained according to the following steps: Acquire a text sample, a reference summary sample of the text sample, and a plurality of candidate summary samples of the text sample, and input them into a preset model; Determining the text-candidate summary consistency between the text sample and each of its candidate summary samples through the preset model, and determining the text-reference summary consistency between the text sample and its reference summary sample; Determining the preset model after the first contrastive learning and the second contrastive learning as the summary recognition model; Among them, the first contrastive learning is the contrastive learning between the reference summary sample of the text sample and each candidate summary sample performed by the preset model according to the text-reference summary consistency and the text-candidate summary consistency corresponding to each candidate summary sample; the second contrastive learning is the contrastive learning between every two candidate summary samples of the text sample performed by the preset model according to the text-candidate summary consistency corresponding to each of the multiple candidate summary samples.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the abstract generation method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the abstract generation method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Text abstract generation method and device, terminal equipment and storage medium

    CN112347758A

  • Abstraction extraction model training method and device, abstract extraction method and device and electronic equipment

    CN112560479A