A Summary Reordering Method and System Combining Extractive and Generative Approaches
By combining extractive and generative methods and utilizing contextual relationships and rewriting models to optimize summarization, the problem of insufficient summarization accuracy in existing technologies is solved, resulting in more accurate and coherent summaries.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUILIN UNIV OF ELECTRONIC TECH
- Filing Date
- 2022-12-20
- Publication Date
- 2026-05-26
AI Technical Summary
Existing extractive and generative summarization methods are insufficient in terms of accuracy. Extractive methods may contain irrelevant or redundant information, while generative methods may deviate from the semantics of the original text.
Combining extractive and generative methods, this paper generates summary sentences by extracting key sentences from the text, and optimizes the summary by leveraging contextual relationships and a rewriting model. The summary generation is optimized by training the rewriting model and using binary cross-entropy loss.
It improves the accuracy and coherence of the abstracts, making the generated abstracts more consistent with the original text and more accurate.
Smart Images
Figure CN116010591B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of abstract generation technology, and in particular to an abstract reordering method and system that combines extractive and generative methods. Background Technology
[0002] Text summarization is broadly divided into two categories: extractive methods and generative methods. Each has its advantages and disadvantages. Extractive methods extract important text segments from a document and connect them to form a summary. Compared to generative summarization, it has advantages in content selectivity and authenticity; however, the extracted sentences may contain irrelevant or redundant information, and because it doesn't consider context, it may have low coherence. Generative methods use conditional language models to generate summaries, removing irrelevant information, but the generated summaries may be difficult to understand and deviate from the original semantics.
[0003] Therefore, regardless of whether an extractive or generative method is used, the accuracy of the generated summaries is low. Summary of the Invention
[0004] To improve the accuracy of generated summaries, this invention provides a summary reordering method and system that combines extractive and generative methods.
[0005] In a first aspect, to address the aforementioned technical problems, this invention provides a summary reordering method combining extractive and generative methods, comprising the following steps:
[0006] S1, retrieve the first article, which contains multiple first sentences and multiple original summary sentences;
[0007] S2, extract each first sentence to obtain multiple first extracted sentences. The first extracted sentences are the iconic sentences in the first article;
[0008] S3, based on each first extracted sentence and each original summary sentence, determine the first summary sentence corresponding to each first extracted sentence. The first summary sentence is the summary predicted based on the first extracted sentence.
[0009] S4. Based on each first summary sentence, determine the token corresponding to each first summary sentence. The token can be a character or a word.
[0010] S5: Determine the first target digest based on each token.
[0011] The beneficial effects of the summary reordering method combining extraction and generative methods provided by this invention are as follows: Using an extraction method, multiple first extracted sentences and original summary sentences are obtained from a first article. Then, a first summary sentence is determined using each first extracted sentence and original summary sentence. Next, a generative method is used to generate a token from the first summary sentence, and a first target summary sentence is determined from the token. This application combines extraction and generative methods to modify the extracted first summary sentence into a token, thereby improving the accuracy of summary generation. Furthermore, it filters out the first target summary with higher accuracy from the token, further enhancing the accuracy of the generated summary.
[0012] Based on the above technical solution, the summary reordering method of the present invention, which combines extraction and generative methods, can be further improved as follows.
[0013] Furthermore, the method also includes:
[0014] Obtain the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence;
[0015] Based on each first extracted sentence and each original summary sentence, determine the first summary sentence corresponding to each first extracted sentence, including:
[0016] Based on each first extracted sentence, the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence, the first summary sentence corresponding to each first extracted sentence is determined using a first formula, wherein the first formula is:
[0017]
[0018] Where |Y| represents the total number of sentences extracted in the first round, and argmax(x) represents the search function, where x represents the input of the search function. E represents the output of the search function. j E represents the j-th first extracted sentence. <j Y represents the context corresponding to the j-th first extracted sentence. <j Let X represent the context corresponding to the j-th original summary sentence, and Y represent the first article. j Let P(Y) represent the first summary sentence corresponding to the j-th first extracted sentence. j |E j E <j ,Y <j X) represents the output of the search function in the form of sequence probabilities.
[0019] The beneficial effect of adopting the above-mentioned further scheme is that when generating the first summary sentence corresponding to each first extracted sentence, the context corresponding to the first extracted sentence and the context corresponding to the original summary are taken into consideration, making the generated first summary sentence more consistent with the original text content and more accurate.
[0020] Furthermore, the method also includes:
[0021] Generate a first sequence from each first summary sentence, and generate a second sequence from each first extracted sentence;
[0022] Based on each first summary sentence, determine the token corresponding to each first summary sentence, including:
[0023] Based on the first and second sequences, the token corresponding to each first summary sentence is determined using a second formula, whereby:
[0024]
[0025] in, The output result is represented in the form of sequence probabilities, Y′ k Y′ represents the k-th token. <k G represents the tokens preceding the k-th token. X′ Indicates the first sequence, X' represents the second sequence, and X' represents the first summary sentence.
[0026] The beneficial effect of adopting the above-mentioned further scheme is that, based on the first sequence and the second sequence, the first summary sentence and the first extracted sentence correspond one-to-one, so that the first summary sentence can be accurately indexed to the first extracted sentence and modified into a token according to the original text content, thereby improving the accuracy of summary generation.
[0027] Furthermore, based on each token, the first target digest is determined, including:
[0028] Based on each token, the weight corresponding to each token is determined using the third formula, which is:
[0029]
[0030] Where |Y′| represents the total number of tokens, w j Let Y' represent the weights, where Y' is the weight of Y'. k If w is included in the first sequence and the second sequence, then j =γ, where γ represents the network parameters, if Y′ k If w is not included in the first or second sequence, then j =1, where Loss represents the loss function of the rewriter;
[0031] The first target digest is determined based on each weight and each token.
[0032] The beneficial effect of adopting the above-mentioned further scheme is that weights are assigned to each second summary through the third formula so as to determine the first target summary based on the weights.
[0033] Furthermore, steps S2 to S5 above are determined by rewriting the model, which is trained through the following steps:
[0034] S21, retrieve the second article, which contains multiple second sentences;
[0035] S22, extract each second sentence to obtain multiple extracted second sentences, which are the key sentences in the second article;
[0036] S23, Based on each second extracted sentence, determine the second summary sentence corresponding to each second extracted sentence using the fourth formula, whereby the fourth formula is:
[0037]
[0038] Where E represents the set of second summary sentences, |E| represents the total number of second extracted sentences, and X represents each second sentence. k E represents the second extracted sentence. j This represents the j-th second summary sentence;
[0039] S24, Based on each second summary sentence, determine the corresponding third summary sentence for each second summary sentence using the fifth formula, whereby:
[0040]
[0041] Here, argmax(x) represents the search function, where x represents the input of the search function. P(Y|E,S) represents the output of the search function, where Y represents the third summary sentence and S represents the second article.
[0042] S25. Based on each third summary sentence and evaluation index, determine the second target summary, which is a landmark summary predicted based on the evaluation index.
[0043] S26. Based on the second target summary and each third summary sentence, determine the binary cross-entropy loss, which is the difference between the second target summary and each third summary sentence;
[0044] S27. If the binary cross-entropy loss meets the preset termination condition, the initial model that meets the preset termination condition is used as the rewritten model. If the binary cross-entropy loss does not meet the preset termination condition, the network parameters of the initial network are adjusted, and the initial model is retrained based on the adjusted network parameters until the binary cross-entropy loss of the initial model meets the preset termination condition.
[0045] The beneficial effect of adopting the above-mentioned further scheme is that during the training of the rewritten model, the third summary sentence generated by the initial model is used to determine the second target summary through evaluation metrics, and then the initial model is allowed to learn the differences between the second target summary and each third summary sentence, thereby improving the accuracy of the initial model in generating summaries.
[0046] Furthermore, based on the various third summary sentences and evaluation metrics, the second target summary is determined, including:
[0047] Based on the third summary sentences and evaluation indicators, the second target summary is determined using the sixth formula, which is:
[0048]
[0049] in, This represents the second objective summary, μ represents the evaluation index, (C1),…,(C m ) represents each third summary sentence, and argmax(x) represents the search function, where x represents the input of the search function.
[0050] The beneficial effect of adopting the above-mentioned further scheme is that the second target summary is determined from the third summary sentences through the sixth formula, so as to obtain the best summary sentence, i.e. the second target summary, from each third summary sentence.
[0051] Furthermore, based on the second objective summary and each of the third summary sentences, the binary cross-entropy loss is determined, including:
[0052] Based on the second objective summary and each of the third summary sentences, the binary cross-entropy loss is determined using the seventh formula, which is:
[0053]
[0054] in, Represents binary cross-entropy loss. This indicates that the initial model is parameterized by θ, when At that time, y i =1, At that time, y i =0.
[0055] The beneficial effect of adopting the above-mentioned further scheme is that, through the seventh formula, the initial model learns the differences between the second target summary and each third summary sentence, thereby improving the accuracy of the initial model in generating the summary.
[0056] Secondly, the present invention provides a summary reordering system combining extractive and generative methods, comprising:
[0057] The first article acquisition module is used to acquire the first article, which contains multiple first sentences and multiple original summary sentences.
[0058] The first sentence extraction module is used to extract each first sentence, resulting in multiple first extracted sentences. The first extracted sentences are the iconic sentences in the first article.
[0059] The first summary sentence acquisition module is used to determine the first summary sentence corresponding to each first extracted sentence based on each first extracted sentence and each original summary sentence. The first summary sentence is the summary predicted based on the first extracted sentence.
[0060] The token acquisition module is used to determine the token corresponding to each first summary sentence based on each first summary sentence. The token is the summary after modifying the first summary sentence.
[0061] The first target digest acquisition module is used to determine the first target digest based on each token.
[0062] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of a summary reordering method combining extractive and generative methods as described above.
[0063] Fourthly, the present invention also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the steps of a summary reordering method combining extractive and generative methods as described above. Attached Figure Description
[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0065] Figure 1 This is a flowchart illustrating a summary reordering method combining extractive and generative approaches according to an embodiment of the present invention.
[0066] Figure 2This is a schematic diagram of a summary reordering system that combines extractive and generative methods according to an embodiment of the present invention. Detailed Implementation
[0067] The following embodiments are further explanations and supplements to the present invention and do not constitute any limitation on the present invention.
[0068] The following describes, with reference to the accompanying drawings, an embodiment of the present invention, a summary reordering system combining extractive and generative methods.
[0069] like Figure 1 As shown, this invention provides a summary reordering method combining extractive and generative methods. This method is applied to a terminal device. In this application, the terminal device is the executing entity, and the solution is described. The terminal device can be a computer, server, etc., used to execute the steps of a summary reordering method combining extractive and generative methods.
[0070] This invention provides a summary reordering method combining extractive and generative methods, comprising the following steps:
[0071] S1, retrieve the first article, which contains multiple first sentences and multiple original summary sentences;
[0072] S2, extract each first sentence to obtain multiple first extracted sentences. The first extracted sentences are the iconic sentences in the first article;
[0073] S3, based on each first extracted sentence and each original summary sentence, determine the first summary sentence corresponding to each first extracted sentence. The first summary sentence is the summary predicted based on the first extracted sentence.
[0074] S4. Based on each first summary sentence, determine the token corresponding to each first summary sentence. The token is a character or word obtained from the first summary sentence.
[0075] S5: Determine the first target digest based on each token.
[0076] Optionally, in S2, some distinctive sentences are extracted from each first sentence as the first extracted sentence. The distinctive sentence refers to the sentence that can express the core content of the first article. For example, the first extracted sentence may be the first sentence related to the research purpose, specific implementation method, results or conclusions of the first article.
[0077] Optionally, when generating the first sentence from the extracted first sentence, since it is impossible to determine whether the first extracted sentence contains irrelevant or redundant information, and the contextual relationship is not considered, the coherence of the first summary sentence may be affected. Therefore, the method further includes:
[0078] Obtain the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence;
[0079] Based on each first extracted sentence and each original summary sentence, determine the first summary sentence corresponding to each first extracted sentence, including:
[0080] Based on each first extracted sentence, the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence, the first summary sentence corresponding to each first extracted sentence is determined using a first formula, wherein the first formula is:
[0081]
[0082] Where |Y| represents the total number of sentences extracted in the first round, and argmax(x) represents the search function, where x represents the input of the search function. E represents the output of the search function. j E represents the j-th first extracted sentence. <j Y represents the context corresponding to the j-th first extracted sentence. <j Let X represent the context corresponding to the j-th original summary sentence, and Y represent the first article. j Let P(Y) represent the first summary sentence corresponding to the j-th first extracted sentence. j |E j E <j ,Y <j X) represents the output of the search function in the form of sequence probabilities.
[0083] In this embodiment, the first formula takes into account the context of the first extracted sentence and the context of the original summary sentence when generating the first summary sentence, so that the generated first summary sentence takes into account the context of the original text and has higher coherence.
[0084] Optionally, the method further includes:
[0085] Generate a first sequence from each first summary sentence, and generate a second sequence from each first extracted sentence;
[0086] Based on each first summary sentence, determine the token corresponding to each first summary sentence, including:
[0087] Based on the first and second sequences, the token corresponding to each first summary sentence is determined using a second formula, whereby:
[0088]
[0089] in, The output result is represented in the form of sequence probabilities, Y′ k Y′ represents the k-th token. <k G represents the tokens preceding the k-th token. X′ Indicates the first sequence, X' represents the second sequence, and X' represents the first summary sentence.
[0090] Optionally, when generating the first sequence and the second sequence, each first summary sentence and each first extracted sentence can be converted into a vector representation through a shared trap table. At this time, each first summary sentence in the first sequence has the same label as the corresponding first extracted sentence in the second sequence. Through this label, the first summary sentence can be located to the position of the corresponding first extracted sentence in the first article.
[0091] Optionally, the first target digest is determined based on each token, including:
[0092] Based on each token, the weight corresponding to each token is determined using the third formula, which is:
[0093]
[0094] Where |Y′| represents the total number of tokens, w j Let Y' represent the weights, where Y' is the weight of Y'. k If w is included in the first sequence and the second sequence, then j =γ, where γ represents the network parameters, if Y′ k If w is not included in the first or second sequence, then j =1, where Loss represents the loss function of the rewriter;
[0095] The first target digest is determined based on each weight and each token.
[0096] Optionally, the weight represents the significance of each token. The more important the token, the greater its weight. Therefore, the tokens can be combined into a first target digest based on their weights.
[0097] Optionally, steps S2 to S5 above are determined by rewriting the model, which is trained through the following steps:
[0098] S21, retrieve the second article, which contains multiple second sentences;
[0099] S22, extract each second sentence to obtain multiple extracted second sentences, which are the key sentences in the second article;
[0100] S23, Based on each second extracted sentence, determine the second summary sentence corresponding to each second extracted sentence using the fourth formula, whereby the fourth formula is:
[0101]
[0102] Where E represents the set of second summary sentences, |E| represents the total number of second extracted sentences, and X represents each second sentence. k E represents the second extracted sentence. j This represents the j-th second summary sentence;
[0103] S24, Based on each second summary sentence, determine the corresponding third summary sentence for each second summary sentence using the fifth formula, whereby:
[0104]
[0105] Here, argmax(x) represents the search function, where x represents the input of the search function. P(Y|E,S) represents the output of the search function, where Y represents the third summary sentence and S represents the second article.
[0106] S25. Based on each third summary sentence and evaluation index, determine the second target summary, which is a landmark summary predicted based on the evaluation index.
[0107] S26. Based on the second target summary and each third summary sentence, determine the binary cross-entropy loss, which is the difference between the second target summary and each third summary sentence;
[0108] S27. If the binary cross-entropy loss meets the preset termination condition, the initial model that meets the preset termination condition is used as the rewritten model. If the binary cross-entropy loss does not meet the preset termination condition, the network parameters of the initial network are adjusted, and the initial model is retrained based on the adjusted network parameters until the binary cross-entropy loss of the initial model meets the preset termination condition.
[0109] Optionally, the dataset in this embodiment contains multiple second articles. Each second article in the dataset is randomly divided into two equal parts, one part being used as the training set and the other part as the test set. It should be noted that each second article in the training set is used as an input to the initial model, and the initial model is trained through S21-S27.
[0110] Optionally, in S22, some distinctive sentences are extracted from each second sentence as second extraction sentences. The distinctive sentences are those that can express the core content of the second article. For example, the second extraction sentences can be second sentences related to the research purpose, specific implementation methods, results, or conclusions of the second article.
[0111] Optionally, before determining the second target summary, since the output of argmax(x) is represented in the form of sequence probabilities, it is necessary to decode the sequence probabilities. The decoding method can be a combination of bundle search, diversified bundle search, top-k sampling and top-p sampling. By using multiple methods for joint decoding, the problem of huge search space and difficulty in encoding global constraints such as syntacticity, coherence and factual consistency when using a single bundle search method can be solved.
[0112] Optionally, the evaluation metric refers to the standard for evaluating the accuracy of the summary generated by the initial model. Once the evaluation metric is preset, the score of each decoded third summary sentence can be obtained based on the evaluation metric, i.e.:
[0113] S μ ={μ(C1),…,μ(C m )};
[0114] Among them, S μ Let μ be the set of ratings, and (C1), ..., (C2) be the evaluation index. m ) represents each third summary sentence. After obtaining the score of each third summary sentence, the third summary sentence corresponding to the highest score can be used as the second target summary.
[0115] Optionally, the second target summary is determined based on each third summary sentence and evaluation metric, including:
[0116] Based on the third summary sentences and evaluation indicators, the second target summary is determined using the sixth formula, which is:
[0117]
[0118] in, This represents the second objective summary, μ represents the evaluation index, (C1),…,(C m ) represents each third summary sentence, and argmax(x) represents the search function, where x represents the input of the search function.
[0119] Optionally, the binary cross-entropy loss is determined based on the second target summary and each of the third summary sentences, including:
[0120] Based on the second objective summary and each of the third summary sentences, the binary cross-entropy loss is determined using the seventh formula, which is:
[0121]
[0122] in, Represents binary cross-entropy loss. This indicates that the initial model is parameterized by θ, when At that time, y i =1, At that time, y i =0.
[0123] Optionally, after training the initial model using all training sets through S21-S27, a trained rewritten model can be obtained. The rewritten model is then tested using a test set to obtain multiple test summaries. The rewritten model is evaluated by calculating the recall rate at different thresholds. Since different test summaries may receive the same score, the recall rate of each test summary at the randomized unified ranking baseline at threshold k is:
[0124]
[0125] Where, m best R is the number of test summaries, k is the threshold, and R is the number of test summaries. k represents recall, and m represents the test summary.
[0126] Optionally, the summaries output by the rewriting model can also be evaluated by human review in order to improve the rewriting model in the future.
[0127] like Figure 2 As shown, an embodiment of the present invention provides a summary reordering system combining extractive and generative methods, comprising:
[0128] The first article acquisition module 201 is used to acquire the first article, which contains multiple first sentences and multiple original summary sentences.
[0129] The first sentence extraction module 202 is used to extract each first sentence to obtain multiple first extracted sentences. The first extracted sentences are the iconic sentences in the first article.
[0130] The first summary sentence acquisition module 203 is used to determine the first summary sentence corresponding to each first extracted sentence based on each first extracted sentence and each original summary sentence. The first summary sentence is the summary predicted based on the first extracted sentence.
[0131] The token acquisition module 204 is used to determine the token corresponding to each first summary sentence based on each first summary sentence. The token is the summary after modifying the first summary sentence.
[0132] The first target digest acquisition module 205 is used to determine the first target digest based on each token.
[0133] Optionally, the system may also include:
[0134] The context acquisition module is used to acquire the context corresponding to each first extracted sentence, as well as the context corresponding to each original summary sentence;
[0135] The first summary sentence acquisition module 203 is also used for:
[0136] Based on each first extracted sentence, the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence, the first summary sentence corresponding to each first extracted sentence is determined using a first formula, wherein the first formula is:
[0137]
[0138] Where |Y| represents the total number of sentences extracted in the first round, and argmax(x) represents the search function, where x represents the input of the search function. E represents the output of the search function. j E represents the j-th first extracted sentence. <j Y represents the context corresponding to the j-th first extracted sentence. <j Let X represent the context corresponding to the j-th original summary sentence, and Y represent the first article. j Let P(Y) represent the first summary sentence corresponding to the j-th first extracted sentence. j |E j E <j ,Y <j X) represents the output of the search function in the form of sequence probabilities.
[0139] Optionally, the system may also include:
[0140] The sequence generation module is used to generate a first sequence from each first summary sentence and a second sequence from each first extracted sentence;
[0141] The token acquisition module 204 is also used for:
[0142] Based on the first and second sequences, the token corresponding to each first summary sentence is determined using a second formula, whereby:
[0143]
[0144] in, The output result is represented in the form of sequence probabilities, Y′ k Y′ represents the k-th token. <k G represents the tokens preceding the k-th token. X′ Indicates the first sequence, X' represents the second sequence, and X' represents the first summary sentence.
[0145] Optionally, the first target summary acquisition module 205 is also used for:
[0146] Based on each token, the weight corresponding to each token is determined using the third formula, which is:
[0147]
[0148] Where |Y′| represents the total number of tokens, w j Let Y' represent the weights, where Y' is the weight of Y'. k If w is included in the first sequence and the second sequence, then j =γ, where γ represents the network parameters, if Y′ k If w is not included in the first or second sequence, then j =1, where Loss represents the loss function of the rewriter;
[0149] The first target digest is determined based on each weight and each token.
[0150] Optionally, the system also includes a model training module, which includes:
[0151] The second article acquisition module is used to acquire a second article, which contains multiple second sentences.
[0152] The second sentence extraction module is used to extract each second sentence, resulting in multiple second extracted sentences. The second extracted sentences are the key sentences in the second article.
[0153] The second summary sentence acquisition module is used to determine the second summary sentence corresponding to each second extracted sentence based on each second extracted sentence, using a fourth formula. The fourth formula is:
[0154]
[0155] Where E represents the set of second summary sentences, |E| represents the total number of second extracted sentences, and X represents each second sentence. k E represents the second extracted sentence. j This represents the j-th second summary sentence;
[0156] The third summary sentence acquisition module is used to determine the third summary sentence corresponding to each second summary sentence based on the second summary sentences, using a fifth formula. The fifth formula is:
[0157]
[0158] Here, argmax(x) represents the search function, where x represents the input of the search function. P(Y|E,S) represents the output of the search function, where Y represents the third summary sentence and S represents the second article.
[0159] The second target summary acquisition module is used to determine the second target summary based on each third summary sentence and evaluation index. The second target summary is a landmark summary predicted based on the evaluation index.
[0160] The binary cross-entropy loss acquisition module is used to determine the binary cross-entropy loss based on the second target summary and each third summary sentence. The binary cross-entropy loss is the difference between the second target summary and each third summary sentence.
[0161] The judgment module is used to determine if the binary cross-entropy loss meets the preset termination condition, and to use the initial model that meets the preset termination condition as the rewritten model. If the binary cross-entropy loss does not meet the preset termination condition, the network parameters of the initial network are adjusted, and the initial model is retrained based on the adjusted network parameters until the binary cross-entropy loss of the initial model meets the preset termination condition.
[0162] Optionally, the second target summary acquisition module described above is also used for:
[0163] Based on the third summary sentences and evaluation indicators, the second target summary is determined using the sixth formula, which is:
[0164]
[0165] in, This represents the second objective summary, μ represents the evaluation index, (C1),…,(C m ) represents each third summary sentence, and argmax(x) represents the search function, where x represents the input of the search function.
[0166] Optionally, the binary cross-entropy loss acquisition module described above is also used for:
[0167] Based on the second objective summary and each of the third summary sentences, the binary cross-entropy loss is determined using the seventh formula, which is:
[0168]
[0169] in, Represents binary cross-entropy loss. This indicates that the initial model is parameterized by θ, when At that time, y i =1, At that time, y i =0.
[0170] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements some or all of the steps of the above-described summary reordering method combining extractive and generative methods.
[0171] The electronic device can be a computer, and its program is computer software. The parameters and steps of the electronic device of the present invention can be referred to the parameters and steps of the summary reordering method combining extraction and generative methods in the above-described embodiment, and will not be repeated here.
[0172] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be embodied in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which contains computer-readable program code. Computer-readable storage media can be, for example, but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof.
[0173] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0174] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A summary reordering method combining extractive and generative methods, characterized in that, Includes the following steps: S1, Obtain the first article, which contains multiple first sentences and multiple original summary sentences; S2, extract each of the first sentences to obtain multiple first extracted sentences, where the first extracted sentences are the iconic sentences in the first article; S3, based on each of the first extracted sentences and each of the original summary sentences, determine the first summary sentence corresponding to each of the first extracted sentences, wherein the first summary sentence is the summary predicted based on the first extracted sentence; S4, based on each of the first summary sentences, determine the token corresponding to each of the first summary sentences, wherein the token is a character or a word; S5, determine the first target digest based on each of the tokens; The method also includes: Obtain the context corresponding to each of the first extracted sentences, and the context corresponding to each of the original summary sentences; The step of determining the first summary sentence corresponding to each first extracted sentence based on each of the first extracted sentences and each of the original summary sentences includes: Based on each of the first extracted sentences, the context corresponding to each of the first extracted sentences, and the context corresponding to each of the original summary sentences, a first summary sentence corresponding to each of the first extracted sentences is determined using a first formula, wherein the first formula is: Where |Y| represents the total number of sentences extracted in the first round, and argmax(x) represents the search function, where x represents the input of the search function. This represents the output of the search function. This represents the j-th first extracted sentence. This represents the context corresponding to the j-th first extracted sentence. Let X represent the context corresponding to the j-th original summary sentence, and let X represent the first article. This represents the first summary sentence corresponding to the j-th first extracted sentence. This indicates that the output of the search function is represented in the form of sequence probabilities; The method also includes: Generate a first sequence from each of the first summary sentences, and generate a second sequence from each of the first extracted sentences; The step of determining the token corresponding to each of the first summary sentences includes: Based on the first sequence and the second sequence, a token corresponding to each of the first summary sentences is determined using a second formula, wherein the second formula is: in, The output result is represented in the form of sequence probabilities. This represents the k-th token. This represents the tokens preceding the k-th token. Indicates the first sequence, Indicates the second sequence, This indicates the first summary sentence.
2. The method according to claim 1, characterized in that, The step of determining the first target digest based on each of the tokens includes: Based on each of the tokens, the weight corresponding to each token is determined using a third formula, wherein the third formula is: in, This represents the total number of tokens. Denotes the weight, where if If it is included in the first sequence and the second sequence, then = , Indicates network parameters, if If it is not included in the first sequence and the second sequence, then = , Represents the loss function of the rewriter; The first target digest is determined based on each of the weights and each of the tokens.
3. The method according to claim 1, characterized in that, Steps S2 to S5 are determined by a rewritten model, which is trained through the following steps: S21, Obtain the second article, which contains multiple second sentences; S22, extract each of the second sentences to obtain multiple extracted second sentences, which are the iconic sentences in the second article; S23, based on each of the second extracted sentences, determine the second summary sentence corresponding to each second extracted sentence using the fourth formula, wherein the fourth formula is: ； in, Let |E| represent the set of second summary sentences, and |E| represent the total number of second extracted sentences. This refers to each of the second sentences. This indicates the second extracted sentence. This represents the j-th second summary sentence; S24, based on each of the second summary sentences, determine the third summary sentence corresponding to each of the second summary sentences using a fifth formula, wherein the fifth formula is: ； Here, argmax(x) represents the search function, where x represents the input of the search function. This represents the output of the search function. The output of the search function is represented in the form of sequence probabilities, where Y represents the third summary sentence and S represents the second article; S25, based on each of the third summary sentences and evaluation indicators, determine the second target summary, which is a landmark summary predicted based on the evaluation indicators; S26, determine the binary cross-entropy loss based on the second target summary and each of the third summary sentences, wherein the binary cross-entropy loss is the difference between the second target summary and each of the third summary sentences; S27. If the binary cross-entropy loss satisfies the preset termination condition, the initial model that satisfies the preset termination condition is used as the rewritten model. If the binary cross-entropy loss does not satisfy the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained according to the adjusted network parameters until the binary cross-entropy loss of the initial model satisfies the preset termination condition.
4. The method according to claim 3, characterized in that, The determination of the second target summary based on each of the third summary sentences and evaluation metrics includes: Based on the third summary sentences and evaluation indicators, the second target summary is determined using a sixth formula, wherein the sixth formula is: ； in, This represents a summary of the second objective. Indicates the evaluation indicators, Let represent each third summary sentence, and argmax(x) represent the search function, where x represents the input of the search function.
5. The method according to claim 4, characterized in that, The step of determining the binary cross-entropy loss based on the second target summary and each of the third summary sentences includes: Based on the second target summary and each of the third summary sentences, the binary cross-entropy loss is determined using the seventh formula, wherein the seventh formula is: ； in, Represents binary cross-entropy loss. This indicates that the initial model is parameterized by θ, when hour, =1, hour, =0.
6. A summary reordering system combining extractive and generative methods, characterized in that, include: The first article acquisition module is used to acquire the first article, which contains multiple first sentences and multiple original summary sentences. The first sentence extraction module is used to extract each of the first sentences to obtain multiple first extracted sentences, wherein the first extracted sentences are the iconic sentences in the first article; The first summary sentence acquisition module is used to determine a first summary sentence corresponding to each first extracted sentence based on each first extracted sentence and each original summary sentence, wherein the first summary sentence is a summary predicted based on the first extracted sentence; The token acquisition module is used to determine the token corresponding to each first digest sentence based on each first digest sentence, wherein the token is a digest after modifying the first digest sentence; The first target digest acquisition module is used to determine the first target digest based on each of the tokens; The system also includes: The context acquisition module is used to acquire the context corresponding to each first extracted sentence, as well as the context corresponding to each original summary sentence; The first summary sentence acquisition module 203 is also used for: Based on each first extracted sentence, the context corresponding to each first extracted sentence, and the context corresponding to each original summary sentence, the first summary sentence corresponding to each first extracted sentence is determined using a first formula, wherein the first formula is: Where |Y| represents the total number of sentences extracted in the first round, and argmax(x) represents the search function, where x represents the input of the search function. This represents the output of the search function. This represents the j-th first extracted sentence. This represents the context corresponding to the j-th first extracted sentence. Let X represent the context corresponding to the j-th original summary sentence, and let X represent the first article. This represents the first summary sentence corresponding to the j-th first extracted sentence. This indicates that the output of the search function is represented in the form of sequence probabilities; The system also includes: The sequence generation module is used to generate a first sequence from each first summary sentence and a second sequence from each first extracted sentence; The token acquisition module 204 is also used for: Based on the first and second sequences, the token corresponding to each first summary sentence is determined using a second formula, whereby: in, The output result is represented in the form of sequence probabilities. This represents the k-th token. This represents the tokens preceding the k-th token. Indicates the first sequence, Indicates the second sequence, This indicates the first summary sentence.
7. An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of a summary reordering method combining extractive and generative methods as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the steps of a summary reordering method combining extractive and generative methods as described in any one of claims 1 to 5.