A method, device and electronic device for generating document summary

Abstract

The present invention discloses a method, device and electronic device for generating a document summary. The method comprises: obtaining a target document to be processed; obtaining a summary generation model for processing the target document; processing the target document using the summary generation model to obtain a corresponding document summary; wherein the summary model is trained by the following method: obtaining a plurality of training samples; obtaining a probability score expression of each reference word in each reference summary at a corresponding decoding moment according to each training sample respectively and taking the undetermined parameters of the summary generation model as variables; obtaining a penalty factor corresponding to each training sample according to the probability score expression of the first reference word in each reference summary at a corresponding decoding moment respectively; constructing a loss function according to the probability score expression of each reference word at a corresponding decoding moment and the penalty factor corresponding to each training sample, and determining the undetermined parameters according to the loss function.

Description

Technical Field

[0001] The present invention relates to the technical field of summary generation, and more specifically, to a method for generating a document summary, a device for generating a document summary, an electronic device, and a computer-readable storage medium. Background Art

[0002] In recent years, due to the strong practicality of summary tasks and their necessity in various scenarios, more and more people have begun to study summary tasks. There are two methods for document summary tasks: extractive and generative. The extractive method extracts a few sentences that can express the main content of the document from the document to form a summary, which is simple and easy. The generative method is to restate the main content of the document based on the understanding of the document. The generated sentences and even words may not be in the document. This method is very similar to the process of human summary writing.

[0003] In the prior art, a summary generation model is usually used to generate a document summary corresponding to a target document. The existing summary generation model is generally a sequence-to-sequence model, which decodes or encodes word by word at the encoding end and the decoding end.

[0004] If the performance of the trained summary generation model is low, then the quality of the document summary obtained by processing the target document using the summary generation model will be poor.

[0005] Therefore, it is very valuable to propose a solution to generate document summaries by training a summary generation model to supervise the correctness of the first word. Summary of the invention

[0006] An object of the present invention is to provide a new technical solution for generating document summaries.

[0007] According to a first aspect of the present invention, there is provided a method for generating a document summary, comprising:

[0008] Get the target document to be processed;

[0009] Acquire a summary generation model for processing the target document;

[0010] Processing the target document using the summary generation model to obtain a corresponding document summary;

[0011] The summary model is trained by the following method:

[0012] Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words;

[0013] According to each training sample, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained by taking the undetermined parameters of the summary generation model as variables;

[0014] According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the penalty factor corresponding to each training sample is obtained;

[0015] According to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, a loss function is constructed, and the undetermined parameters are determined according to the loss function.

[0016] Optionally, the step of obtaining the penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time includes:

[0017] According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the probability of the set dictionary word at the decoding time corresponding to each first reference word is obtained;

[0018] Determine the decoding time corresponding to each first reference word respectively, whether a set number of set dictionary words with the greatest probability hit the corresponding first reference word, and obtain the hit result of each first reference word;

[0019] According to the hit result of each first reference word, the penalty factor of the corresponding training sample is obtained.

[0020] Optionally, if the hit result of the first reference word is yes, the penalty factor of the corresponding training sample is set to a set value;

[0021] If the hit result of the first reference word is negative, the penalty factor of the corresponding training sample is set according to the probability score expression of the first reference word with a negative hit result at the corresponding decoding time.

[0022] Optionally, the step of constructing a loss function according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample includes:

[0023] According to the probability expression of each reference word in each training sample at the corresponding decoding time, the loss term corresponding to each training sample is obtained;

[0024] According to the preset weights, the loss term and penalty factor corresponding to each training sample are weighted summed to obtain the initial loss expression corresponding to each training sample;

[0025] The loss function is obtained according to the initial loss expression of each training sample.

[0026] Optionally, the step of determining the undetermined parameter according to the loss function includes:

[0027] Setting the initial value of the pending parameter;

[0028] Substituting the undetermined parameters after setting the initial values ​​into the loss function for iterative processing;

[0029] When the undetermined parameter obtained by the iterative process meets the convergence condition, the iterative process is terminated to determine the value of the undetermined parameter; otherwise, the iterative process is continued.

[0030] Optionally, the convergence condition includes:

[0031] The number of iterative processing reaches a preset number threshold; and / or,

[0032] Substituting the undetermined parameters obtained by the iterative processing into the loss function, the obtained training result value is less than or equal to the preset result threshold.

[0033] Optionally, the summary generation model is used to generate a summary of the target document to be processed.

[0034] Optionally, after determining the undetermined parameter according to the loss function, the step further includes:

[0035] According to each training sample, a probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample is obtained;

[0036] According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained;

[0037] Constructing a new loss function according to the loss function and the bag-of-words loss expression corresponding to each training sample;

[0038] The undetermined parameters are re-determined according to the new loss function to complete the fine-tuning training of the summary generation model.

[0039] Optionally, the step of obtaining the bag-of-words loss expression corresponding to each training sample according to the probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample comprises:

[0040] Sum the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample to obtain the probability and expression corresponding to each reference word in each training sample;

[0041] According to the probability and expression corresponding to each reference word in each training sample, the bag-of-words loss expression corresponding to each training sample is obtained.

[0042] Optionally, the method further includes:

[0043] According to a preset training cycle, the step of training the summary generation model is performed.

[0044] Optionally, the method further includes:

[0045] Provide a setting entry for setting application scenarios of summary generation models;

[0046] Obtain the application scenario input through the setting entry;

[0047] The step of obtaining a plurality of training samples includes: obtaining corresponding training samples according to the input application scenario.

[0048] Optionally, the method further includes:

[0049] Determine the application scenarios for which the trained summary generation model is suitable;

[0050] Find application projects that match the application scenario;

[0051] The trained summary generation model is input into the application project.

[0052] According to a second aspect of the present invention, there is provided a method for generating a document summary, comprising:

[0053] Get the target document to be processed;

[0054] Acquire a summary generation model for processing the target document;

[0055] Processing the target document using the summary generation model to obtain a corresponding document summary;

[0056] The summary model is trained by the following method:

[0057] Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words;

[0058] According to each training sample, the undetermined parameters of the summary generation model are used as variables to obtain the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample;

[0059] According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained;

[0060] A new loss function is constructed according to the bag-of-words loss expression corresponding to each training sample; and the undetermined parameters are re-determined according to the new loss function.

[0061] According to a third aspect of the present invention, there is provided a device for generating a document summary, comprising:

[0062] A document acquisition module is used to acquire the target document to be processed;

[0063] A model acquisition module, used to acquire a summary generation model for processing the target document;

[0064] A document processing module, used to process the target document using the summary generation model to obtain a corresponding document summary;

[0065] Wherein, the model acquisition module is also used for:

[0066] Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words;

[0067] According to each training sample, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained by taking the undetermined parameters of the summary generation model as variables;

[0068] According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the penalty factor corresponding to each training sample is obtained;

[0069] Constructing a loss function according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, and determining the undetermined parameter according to the loss function;

[0070] Alternatively, the model acquisition module is further used for:

[0071] Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words;

[0072] According to each training sample, the undetermined parameters of the summary generation model are used as variables to obtain the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample;

[0073] According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained;

[0074] A new loss function is constructed according to the bag-of-words loss expression corresponding to each training sample; and the undetermined parameters are re-determined according to the new loss function.

[0075] According to a fourth aspect of the present invention, there is provided an electronic device, comprising:

[0076] The device according to the third aspect of the present invention; or,

[0077] A processor and a memory, wherein the memory is used to store instructions, and the instructions are used to control the processor to execute the method according to the first aspect or the second aspect of the present invention.

[0078] According to a fifth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in the first aspect or the second aspect of the present invention.

[0079] In an embodiment of the present invention, according to each training sample and the corresponding reference summary, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained with the undetermined parameters of the summary generation model as variables, and then the penalty factor corresponding to each training sample is obtained according to the probability expression of the first reference word in each reference summary at the corresponding decoding time. According to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, a loss function is constructed; and the undetermined parameters are determined according to the loss function, and the summary generation model obtained by training is used to process the target document to be processed to obtain the corresponding document summary. By obtaining the penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the problem of the summary generation model predicting the first word of the reference summary in each training sample incorrectly is punished, so that the summary generation model is encouraged to predict the first word correctly, thereby improving the performance of the summary generation model, and further improving the accuracy of the generated document summary.

[0080] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0082] Figure 1 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to the first embodiment of the present invention.

[0083] Figure 2 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to a second embodiment of the present invention.

[0084] Figure 3 A flowchart of a method for generating a document summary according to an embodiment of the present invention is shown.

[0085] Figure 4 A flowchart of a method for training a summary generation model according to a first embodiment of the present invention is shown.

[0086] Figure 5 A flowchart of a method for training a summary generation model according to a second embodiment of the present invention is shown.

[0087] Figure 6 A first example flow chart of a method for generating a document summary according to an embodiment of the present invention is shown.

[0088] Figure 7 A flowchart showing a second example of the method for generating a document summary according to an embodiment of the present invention.

[0089] Figure 8 A flowchart showing a third example of the method for generating a document summary according to an embodiment of the present invention.

[0090] Fig. 9 A principle block diagram of a device for generating a document summary according to an embodiment of the present invention is shown.

[0091] Fig.10 A principle block diagram of an electronic device according to a third embodiment of the present invention is shown. DETAILED DESCRIPTION

[0092] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0093] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0094] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0095] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0096] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0097] <Hardware Configuration>

[0098] Figure 1 and Figure 2 is a block diagram of a hardware configuration of an electronic device 1000 that can be used to implement the method of any embodiment of the present invention.

[0099] In one embodiment, Figure 1 As shown, the electronic device 1000 may be a server 1100 .

[0100] Server 1100 provides a business point for processing, database, and communication facilities. Server 1100 can be a monolithic server or a distributed server across multiple computers or computer data centers. The server can be of various types, such as but not limited to, a network server, a news server, a mail server, a message server, an advertising server, a file server, an application server, an interactive server, a database server, or a proxy server. In some embodiments, each server may include hardware, software, or an embedded logic component or a combination of two or more such components for executing the appropriate functions supported or implemented by the server. For example, a server such as a blade server, a cloud server, etc., or a server group consisting of multiple servers, which may include one or more of the above-mentioned types of servers, etc.

[0101] In this embodiment, the server 1100 can be as follows Figure 1 As shown, it includes a processor 1110 , a memory 1120 , an interface device 1130 , a communication device 1140 , a display device 1150 , and an input device 1160 .

[0102] In this embodiment, the server 1100 may also include a speaker, a microphone, etc., which are not limited here.

[0103] The processor 1110 may be a dedicated server processor, or a desktop processor or a mobile processor that meets the performance requirements, which is not limited here. The memory 1120 includes, for example, ROM (read-only memory), RAM (random access memory), non-volatile memory such as a hard disk, etc. The interface device 1130 includes, for example, various bus interfaces, such as a serial bus interface (including a USB interface), a parallel bus interface, etc. The communication device 1140 is capable of wired or wireless communication, for example. The display device 1150 is, for example, a liquid crystal display, an LED display, a touch display, etc. The input device 1160 may include, for example, a touch screen, a keyboard, etc.

[0104] In this embodiment, the memory 1120 of the server 1100 is used to store instructions, which are used to control the processor 1110 to operate to at least perform the method according to any embodiment of the present invention. A technician can design instructions according to the scheme disclosed in the present invention. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0105] Despite Figure 1 , multiple devices of the server 1100 are shown; however, the present invention may only involve some of the devices, for example, the server 1100 only involves the memory 1120 and the processor 1110 .

[0106] In one embodiment, the electronic device 1000 may be a terminal device 1200 such as a PC or a laptop computer used by an operator, which is not limited here.

[0107] In this embodiment, refer to Figure 2 As shown, the terminal device 1200 may include a processor 1210, a memory 1220, an interface device 1230, a communication device 1240, a display device 1250, an input device 1260, a speaker 1270, a microphone 1280, and the like.

[0108] The processor 1210 may be a mobile version processor. The memory 1220 may include, for example, a ROM (read-only memory), a RAM (random access memory), a non-volatile memory such as a hard disk, etc. The interface device 1230 may include, for example, a USB interface, a headphone interface, etc. The communication device 1240 may, for example, be capable of wired or wireless communication. The communication device 1240 may include a short-range communication device, for example, any device for short-range wireless communication based on short-range wireless communication protocols such as Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. The communication device 1240 may also include a remote communication device, for example, any device for WLAN, GPRS, 2G / 3G / 4G / 5G remote communication. The display device 1250 may, for example, be a liquid crystal display, a touch display, etc. The input device 1260 may include, for example, a touch screen, a keyboard, etc. The user may input / output voice information through the speaker 1270 and the microphone 1280.

[0109] In this embodiment, the memory 1220 of the terminal device 1200 is used to store instructions, and the instructions are used to control the processor 1210 to operate to at least perform the method according to any embodiment of the present invention. A technician can design instructions according to the scheme disclosed in the present invention. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0110] Despite Figure 2 , multiple devices of the terminal device 1200 are shown, but the present invention may only involve some of the devices, for example, the terminal device 1200 only involves the memory 1220, the processor 1210, and the display device 1250.

[0111] <Method Example>

[0112] In this embodiment, a method for generating a document summary is provided. The method for generating a document summary may be implemented by an electronic device. The electronic device may be Figure 1 The server 1100 shown or Figure 2 The terminal device 1200 is shown.

[0113] according to Figure 3 As shown, the method for generating a document summary in this embodiment may include the following steps S3100 to S3300:

[0114] Step S3100, obtaining the target document to be processed.

[0115] Step S3200: Obtain a summary generation model for processing the target document.

[0116] Step S3300: Use the summary generation model to process the target document to obtain a corresponding document summary.

[0117] The summary generation model may be obtained by training through the first embodiment or the second embodiment shown below.

[0118] <First Embodiment>

[0119] In the existing summary generation scenario, the generation of the next word is based on the previously generated word. If the previous word is generated correctly, the probability of the next word being correct will also be high. On the contrary, if the previous word is biased, the bias will become larger and larger, resulting in poor quality of the final generated summary. Therefore, this embodiment improves the performance of the summary generation model by supervising the correctness of the first word.

[0120] like Figure 4 As shown, the method for training a summary generation model may include steps S4100 to S4500: Step S4100, obtaining multiple training samples.

[0121] Each training sample includes a document and a corresponding reference abstract, the reference abstract includes multiple reference words, and each reference word has its own arrangement order in the corresponding reference abstract. The word in this embodiment can be a character, a vocabulary or a punctuation mark.

[0122] The document may be a sequence with a length greater than the reference abstract, that is, the number of words contained in each document may be greater than the number of words in the corresponding reference abstract.

[0123] Step S4200 , based on each training sample, using the undetermined parameters of the summary generation model as variables, obtain a probability score expression of each reference word in each reference summary at the corresponding decoding time.

[0124] In this example, the summary generation model can be a sequence-to-sequence model, so the summary generation model can specifically include an encoding end and a decoding end. The encoding process of the encoding end actually uses the memory function of the recurrent neural network. The word vectors are input into the network in sequence through the sequence relationship of the context in the input document. For the recurrent neural network, the network will output a result each time, but the difference in encoding is that it only retains the last hidden state, which is equivalent to condensing the entire sentence together and storing it as a content vector for the subsequent decoding end. The structure of the decoding end and the encoding end is almost the same. The only difference is that in the decoding process, the subsequent results are obtained based on the previous results. In the encoding process, a document is input, and every word in this document is known, while the decoding process is equivalent to knowing nothing. First, an identifier is required to indicate the beginning of a sentence, and then it is input into the network to obtain the first output as the first word of the summary, and then the first word is used as the next input of the decoding end, and the output is used as the second word of the summary. The cycle continues, and the summary output by the network is obtained in this way.

[0125] In the process of training the summary generation model, if the reference summary of the training sample contains T reference words, then the summary generation model needs to perform T decoding for the training sample. In this embodiment, the t-th decoding time of the training sample can be used as the decoding time corresponding to the t-th reference word in the reference summary of the training sample, where t∈[1,T].

[0126] Specifically, when the number of training samples in this round of training summary generation model is N, the pending parameter of the summary generation model can be set to X, and the probability expression of the t-th reference word in the reference summary of the n-th training sample at the corresponding decoding time can be expressed as: P n,t (X), n∈[1,N].

[0127] Step S4300 , obtaining a penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time.

[0128] The first reference word may be a reference word in the corresponding reference summary whose ranking value is 1. The decoding time corresponding to the first reference word may be the first decoding time of the corresponding training sample.

[0129] In the case where the number of training samples in this round of training summary generation model is N, if the probability expression of the t-th reference word in the reference summary of the n-th training sample at the corresponding decoding time is expressed as P n,t(X), n∈[1,N], then the probability expression of the first reference word in the reference summary of the nth training sample at the corresponding decoding time can be expressed as P n,1 (X), n∈[1,N].

[0130] In one embodiment of the present invention, the step of obtaining the penalty factor corresponding to the nth training sample according to the probability score expression of the first reference word in the reference summary of the nth training sample at the corresponding decoding time may include the following steps S4310 to S4330:

[0131] Step S4310, according to the probability expression of the first reference word in the reference summary of the nth training sample at the corresponding decoding time, obtain the probability of the set dictionary word at the decoding time corresponding to the first reference word.

[0132] Specifically, a large number of set dictionary words may be pre-stored in the summary generation model, and the set dictionary words include all reference words in all reference summaries. During the decoding process of the summary generation model, the probability of each set dictionary word at each decoding time may be obtained.

[0133] Therefore, this step S4310 can specifically be based on the probability expression of the first reference word in the reference summary of the nth training sample to obtain the probability of each set dictionary word at the decoding time corresponding to the first reference word in the reference summary of the nth training sample.

[0134] Step S4320, determining the decoding time corresponding to the first reference word of the reference summary of the nth training sample, whether a set number of set dictionary words with the highest probability hit the first reference word, and obtaining the hit result of the first reference word.

[0135] Since the set dictionary words include all the reference words, the obtained probability of the set dictionary words also includes the probability of the first reference word of the reference summary of the nth training sample.

[0136] Specifically, each set dictionary word may be sorted in descending order of probability, and the sorting value of each set dictionary word may be obtained. It is determined whether the set dictionary words whose sorting value is less than or equal to a set threshold (the size of the set threshold is the same as the size of the set number) contain the first reference word of the reference summary of the nth training sample. If so, the hit result of the first reference word of the reference summary of the nth training sample is yes; if not, the hit result of the first reference word of the reference summary of the nth training sample is no.

[0137] Step S4330, obtaining the penalty factor of the nth training sample according to the hit result of the first reference word.

[0138] In one embodiment, if the hit result of the first reference word of the reference summary of the nth training sample is yes, the penalty factor of the nth training sample is set to a set value; if the hit result of the first reference word of the reference summary of the nth training sample is no, the penalty factor of the nth training sample is set according to the probability expression of the first reference word of the reference summary of the nth training sample at the corresponding decoding time.

[0139] The setting value may be set in advance according to the application scenario or specific requirements. For example, the setting value may be 0, then if the hit result of the first reference word of the reference summary of the nth training sample is yes, the penalty factor of the nth training sample is set to 0.

[0140] When the number of training samples in this round of training summary generation model is N, if the hit result of the first reference word of the reference summary of the nth training sample is no, the probability score expression of the first reference word of the reference summary of the nth training sample at the corresponding decoding time is expressed as P n,1 (X), then the penalty factor for the nth training sample can be set to -log P n,1 (X), n∈[1,N].

[0141] Step S4400, constructing a loss function according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample.

[0142] In one embodiment, step S4400 may further include steps S4410 to S4430 as shown below:

[0143] Step S4410, determining the loss term corresponding to each training sample according to the probability score expression of each reference word in each training sample at the corresponding decoding time.

[0144] In one embodiment, the negative logarithm of the probability score expression of each reference word in each training sample at the corresponding decoding moment may be determined respectively, and then the negative logarithms corresponding to all reference words in each training sample may be summed respectively to obtain the loss term corresponding to each training sample.

[0145] In the case where the number of training samples in this round of training summary generation model is N, if the reference summary of the nth training sample includes T reference words, the probability expression of the tth reference word in the reference summary of the nth training sample at the corresponding decoding time is expressed as P n,t(X), t∈[1, T], n∈[1, N], then, according to the probability expression of each reference word in the nth training sample at the corresponding decoding time, the loss term corresponding to the nth training sample can be determined as:

[0146] Step S4420, according to the preset weights, weighted sums are performed on the loss terms and penalty factors corresponding to each training sample to obtain the initial loss expression corresponding to each training sample.

[0147] Specifically, the preset weights may be the weights of the loss term and the penalty factor, so that the weights of the loss term of different training samples are the same, and the weights of the penalty factors of different training samples are the same. For example, the weight of the loss term may be set to λ1, and the weight of the penalty factor may be set to λ2.

[0148] Then, when the number of training samples in this round of training summary generation model is N, the loss term corresponding to the nth training sample is and the penalty factor -log P n,1 (X) is weighted summed to obtain the initial loss expression corresponding to the nth training sample, which can be expressed as

[0149] Step S4430, obtaining a loss function according to the initial loss expression of each training sample.

[0150] In one embodiment, the initial loss expression of each training sample may be summed to obtain the loss function.

[0151] For example, when the number of training samples in this round of training summary generation model is N, the initial loss expression corresponding to the nth training sample is expressed as L n , n∈[1,N], then the loss function can be expressed as

[0152] In another embodiment, the initial loss expression of each training sample may be averaged to obtain the loss function.

[0153] For example, when the number of training samples in this round of training summary generation model is N, the initial loss expression corresponding to the nth training sample is expressed as L n , n∈[1,N], then the loss function can be expressed as

[0154] Step S4500, determining the undetermined parameters according to the loss function, and completing the training of the summary generation model.

[0155] In one embodiment, step S4500 may further include steps S4510 to S4530 as shown below:

[0156] Step S4510, setting the initial value of the parameter to be determined.

[0157] The undetermined parameter X in this embodiment may be m coefficients {X1, X2, ..., X m}.

[0158] In one embodiment of the present invention, the initial value of the undetermined parameter may be the value of the undetermined parameter of the summary generation model obtained through preliminary training in advance. The preliminary training in this embodiment may be training based on a loss function obtained by summing or averaging the loss items corresponding to each training sample.

[0159] In another embodiment of the present invention, the initial value of the parameter to be determined can be set to a random number in a preset value range. The preset value range can be set according to the application scenario or application requirements. For example, the preset value range is set to 0-1 so that the coefficients {X1, X2, ..., X m The initial value of} is a random number between 0 and 1.

[0160] Step S4520, substitute the undetermined parameters after setting the initial values ​​into the loss function and perform iterative processing.

[0161] Specifically, the loss function may be solved according to the maximum likelihood estimation method, and the undetermined parameters when the result value of the loss function is minimized may be determined to complete this iterative process.

[0162] Assume that this iteration is the k+1th iteration (the initial value of k is 0 and increases by 1 with each iteration), and the undetermined parameter before this iteration is X k , the corresponding feature weight after iteration is X k+1 .

[0163] Step S4530, when the undetermined parameters obtained by the iterative processing meet the convergence conditions, the iterative processing is terminated and the values ​​of the undetermined parameters are determined; otherwise, the iterative processing is continued.

[0164] In this example, the convergence condition can be set according to specific application scenarios or application requirements.

[0165] For example, the convergence condition is that the number of iterative processing is greater than a preset number threshold. The preset number threshold can be set according to engineering experience or experimental simulation results, for example, it can be set to 10000. Correspondingly, assuming that the number of iterative processing is k+1, the number threshold is itemNums, and the corresponding convergence condition is: k≥itemNums.

[0166] For another example, the convergence condition is that the undetermined parameters obtained by iterative processing are substituted into the loss function, and the obtained training result value is less than or equal to a preset result threshold.

[0167] The result threshold can be set based on engineering experience or experimental simulation results. For example, the result threshold can be set to L th Assume that the undetermined parameter determined after the k+1th iteration is X k+1 , the corresponding training result value is: L(X k+1 ), the convergence condition can be:

[0168] L(X k+1 )≤L th .

[0169] In one example, the convergence condition is to satisfy any one of the convergence conditions in the above two examples. The specific convergence condition has been described in the above two examples and will not be repeated here.

[0170] The summary generation model trained by the method of the embodiment of the present invention can be used to generate a summary of a target document to be processed. Specifically, the target document to be processed can be input into the trained summary generation model to obtain a document summary corresponding to the target document.

[0171] In actual applications, the document content that can be obtained by the user may change over time, which may affect the actual generation effect of the document summary corresponding to the target document. Therefore, in this embodiment, the summary generation model can also be trained according to the preset training cycle. Figure 3 The steps of training the summary generation model shown enable the summary generation model to adaptively adapt to the changes of the above factors affecting the summary generation effect over time, and generate document summaries more accurately, so as to achieve summary generation that actually meets user needs.

[0172] The training period can be set according to specific application scenarios or application requirements, for example, it can be set to 1 day.

[0173] In an embodiment of the present invention, according to each training sample and the corresponding reference summary, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained with the pending parameters of the summary generation model as variables, and then the penalty factor corresponding to each training sample is obtained according to the probability expression of the first reference word in each reference summary at the corresponding decoding time. According to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, a loss function is constructed; and the pending parameters are determined according to the loss function to complete this round of training of the summary generation model.

[0174] In the summary generation scenario, the generation of the next word is based on the previously generated word. If the previous word is generated correctly, the probability of the next word being correct will also be high. On the contrary, if the previous word is biased, the bias will become larger and larger, resulting in poor quality of the final generated summary. Therefore, according to the probability expression of the first reference word in each reference summary at the corresponding decoding moment, the penalty factor corresponding to each training sample is obtained. The penalty factor will penalize the problem of incorrect prediction of the first word of the reference summary in each training sample by the summary generation model. In this way, the summary generation model can be encouraged to predict the first word correctly, thereby improving the performance of the summary generation model, and further improving the accuracy of the document summary of the target document generated by the summary generation model.

[0175] Furthermore, in the process of obtaining the penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time, it is not mandatory to obtain the highest probability of the first word of the reference summary, but only requires that the first word of the reference summary is in the set number of set dictionary words with the highest probability at the corresponding decoding time. In this way, it is possible to allow the summary generation model to have a small deviation in the prediction of the first word of the reference summary in the training sample, thereby improving the performance of the summary generation model.

[0176] In the process of training the summary generation model, each document in the training sample generally has only one corresponding reference summary for training. For the summaries of the same document, different people will write them differently, and even if they use exactly the same words, there will be differences in the word order of expression. However, the summary generation model in the above embodiment is trained in the manner of maximum likelihood estimation, and is calculated one-to-one according to the word order of the reference summary. The training samples are not from the same person, and the expression style and so on will be different. Calculating the loss function in this one-to-one correspondence manner will limit the generation ability of the model and the diversity of the generated summaries.

[0177] In order to solve this problem, when training the summary generation model, the word order, grammar, syntax, etc. of the reference summary can be ignored, and the reference summary can be regarded as a collection of multiple reference words, namely, a bag of words. Specifically, it is not required that the summary generated by the summary generation model based on the document corresponding to the reference summary is the same as the reference summary to be correct, but it is considered that the summary generated by the summary generation model based on the document corresponding to the reference summary contains the same bag of words as the reference summary, which is also a good document summary.

[0178] In this embodiment, after executing the training steps of steps S4100 to S4500, the method may further include steps S4600 to S4900 as shown below, and fine-tune the summary generation model obtained by training in steps S4100 to S4500 again:

[0179] Step S4600: Obtain, based on each training sample, a probability score expression of each reference word in each training sample at each decoding moment of the corresponding training sample.

[0180] When the number of training samples in this round of training summary generation model is N, the reference summary of the nth (n∈[1, N]) training sample contains T reference words, then the nth training sample can have corresponding T decoding moments.

[0181] For example, the probability expression of the tth (t∈[1,T])th reference word in the nth training sample at the ith (i∈[1,T])th decoding time of the nth training sample can be expressed as P n,t,i , where the values ​​of i and t can be the same or different.

[0182] Step S4700, obtaining the bag-of-words loss item corresponding to each training sample according to the probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample.

[0183] In one embodiment, step S4700 may further include steps S4710 to S4720 as shown below:

[0184] Step S4710, respectively sum the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample to obtain the probability and expression corresponding to each reference word in each training sample.

[0185] The probability expression of the tth (t∈[1,T]) reference word in the nth training sample at the ith (i∈[1,T]) decoding time of the nth training sample can be expressed as P n,t,i In the case of , the probability and expression corresponding to the tth (t∈[1,T])th reference word in the nth training sample can be expressed as:

[0186] Step S4720, obtaining the bag-of-words loss expression corresponding to each training sample according to the probability and expression corresponding to each reference word in each training sample.

[0187] Specifically, the probability and negative logarithm of the expression corresponding to each reference word in each training sample may be determined respectively, and then the sum of the negative logarithms corresponding to all reference words in each training sample may be determined respectively to obtain the bag-of-words loss item corresponding to each training sample.

[0188] The probability and expression corresponding to the tth (t∈[1,T])th reference word in the nth training sample is expressed as In the case of , the bag-of-words loss expression of the nth training sample can be expressed as:

[0189] Step S4800, constructing a new loss function according to the bag-of-words loss expression corresponding to each training sample.

[0190] In one embodiment, the bag-of-words loss expression corresponding to each training sample may be summed to obtain a new loss function.

[0191] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as Then, the new loss function can be expressed as

[0192] In one embodiment, the bag-of-words loss expression corresponding to each training sample may be averaged to obtain a new loss function.

[0193] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as Then, the new loss function can be expressed as

[0194] In one embodiment, the bag-of-words loss expression corresponding to each training sample and the initial loss expression of each training sample obtained in the aforementioned step S4420 may be weightedly summed according to preset weights to obtain a new loss expression for each training sample; then the new loss expression for each training sample may be summed or averaged to obtain a new loss function.

[0195] Specifically, the weights of the initial loss expression and the bag-of-words loss expression can be set in advance respectively. The weights of the initial loss expressions of different training samples are the same, and the weights of the bag-of-words loss expressions of different training samples are the same. For example, the weight of the initial loss expression can be set to λ3, and the weight of the bag-of-words loss expression can be set to λ4.

[0196] Then, when the number of training samples in this round of fine-tuning the training summary generation model is N, the initial loss expression for the nth training sample obtained in step S4420 is L n , and bag-of-words loss expression By performing weighted summation, the new loss expression for the nth training sample can be expressed as:

[0197]

[0198] The new loss function can be obtained by summing the new loss expressions of each training sample.

[0199] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as L n ′, n∈[1,N], then the new loss function can be expressed as

[0200] Alternatively, the new loss expression for each training sample can be averaged to obtain a new loss function.

[0201] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as L n ′, n∈[1,N], then the new loss function can be expressed as

[0202] In one embodiment, the bag-of-words loss expression corresponding to each training sample and the loss item of each training sample obtained in the aforementioned step S4410 may be weightedly summed according to preset weights to obtain a new loss expression for each training sample; and then the new loss expression for each training sample may be summed or averaged to obtain a new loss function.

[0203] Specifically, the weights of the loss term and the bag-of-words loss expression can be set in advance respectively. The weights of the loss term of different training samples are the same, and the weights of the bag-of-words loss expressions of different training samples are the same. For example, the weight of the loss term can be set to λ3′, and the weight of the bag-of-words loss expression can be set to λ4′.

[0204] Then, when the number of training samples in this round of fine-tuning the training summary generation model is N, the loss term for the nth training sample obtained in step S4410 is And the bag-of-words loss expression By performing weighted summation, the new loss expression for the nth training sample can be expressed as:

[0205]

[0206] The new loss function can be obtained by summing the new loss expressions of each training sample.

[0207] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as L n ′, n∈[1,N], then the new loss function can be expressed as

[0208] Alternatively, the new loss expression for each training sample can be averaged to obtain a new loss function.

[0209] For example, when the number of training samples in this round of training summary generation model is N, the bag-of-words loss expression corresponding to the nth training sample is expressed as L n ′, n∈[1,N], then the new loss function can be expressed as

[0210] Step S4900: re-determine the pending parameters according to the new loss function to complete the fine-tuning training of the summary generation model.

[0211] The execution method of step S4900 can refer to the aforementioned step S4500 and will not be repeated here.

[0212] In this embodiment, by re-fine-tuning the summary generation model using the bag-of-words loss expression, the diversity of document summaries generated by the trained summary generation model can be improved. Moreover, the summary generation model trained by this embodiment can generate document summaries that are correctly expressed and well summarized, but have a different word order from the reference summary.

[0213] In one embodiment, the method of the present invention may further include the following steps:

[0214] A setting entry for setting an application scenario of a summary generation model is provided; and an application scenario input through the setting entry is obtained.

[0215] Specifically, the user can determine the specific application scenario of the summary generation model according to his own needs, and input the application scenario through the setting entry. The application scenario can be, for example, any scenario where a document summary needs to be generated.

[0216] On this basis, step S4100 can be further performed as follows: obtaining corresponding training samples according to the input application scenario.

[0217] In this way, by training the summary generation model with training samples matching the input application scenario, the trained summary generation model can be adapted to the input application scenario.

[0218] In one embodiment, the method of the present invention may further include the following steps:

[0219] Determine an application scenario to which the trained summary generation model is applicable; find an application project that matches the application scenario; and input the trained summary generation model into the application project.

[0220] In this embodiment, the trained summary generation model is input into an application project that matches the application scenario to which it is applicable, so that the trained summary generation model is used in the corresponding application project to process the target document in the application project to obtain the corresponding document summary.

[0221] In one embodiment, the target document, the document summary generated according to the target document, the documents in the training sample and the reference summary are of the same language type, which may be, for example but not limited to, Chinese.

[0222] In one embodiment, the target document to be processed may be a travel note, an article, news, or a product manual, etc. Correspondingly, the generated document summary may be a travel note summary, an article summary, a news summary, or a product manual summary.

[0223] <Second Embodiment>

[0224] In the process of training the summary generation model, each document in the training sample generally has only one corresponding reference summary for training. For the summaries of the same document, different people will write them differently, and even if they use exactly the same words, there will be differences in the word order of expression. However, the summary generation model in the above embodiment is trained in the manner of maximum likelihood estimation, and is calculated one-to-one according to the word order of the reference summary. The training samples are not from the same person, and the expression style and so on will be different. Calculating the loss function in this one-to-one correspondence manner will limit the generation ability of the model and the diversity of the generated summaries.

[0225] In order to solve this problem, when training the summary generation model, the word order, grammar, syntax, etc. of the reference summary can be ignored, and the reference summary can be regarded as a collection of multiple reference words, namely, a bag of words. Specifically, it is not required that the summary generated by the summary generation model based on the document corresponding to the reference summary is the same as the reference summary to be correct, but it is considered that the summary generated by the summary generation model based on the document corresponding to the reference summary contains the same bag of words as the reference summary, which is also a good document summary.

[0226] according to Figure 5As shown, the training method of the summary generation model in this embodiment may include the following steps S5100 to S5500:

[0227] Step S5100, obtaining multiple training samples.

[0228] Each training sample includes a document and a corresponding reference summary, and the reference summary includes a plurality of reference words. The word in this embodiment can be a character, a vocabulary or a punctuation mark.

[0229] The training samples in this embodiment may be the same as or different from the training samples in step S4100 of the above embodiment.

[0230] Step S5200 , based on each training sample, using the undetermined parameters of the summary generation model as variables, obtain a probability score expression of each reference word in each training sample at each decoding moment of the corresponding training sample.

[0231] The summary generation model in this embodiment may be a summary generation model trained by the above embodiment, or a summary generation model trained by other methods.

[0232] The execution method of step S5200 can refer to the aforementioned step S4600 and will not be repeated here.

[0233] Step S5300, obtaining a bag-of-words loss expression corresponding to each training sample according to a probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample.

[0234] The execution method of step S5300 can refer to the aforementioned step S4700 and will not be repeated here.

[0235] Step S5400, constructing a new loss function according to the bag-of-words loss expression corresponding to each training sample.

[0236] The execution method of step S5400 can refer to the aforementioned step S4800 and will not be repeated here.

[0237] Step S5500: re-determine the pending parameters according to the new loss function to complete the fine-tuning training of the summary generation model.

[0238] The execution method of step S5500 may refer to the aforementioned step S4500 or step S4900, which will not be repeated here.

[0239] In this embodiment, by re-fine-tuning the summary generation model using the bag-of-words loss expression, the diversity of document summaries generated by the trained summary generation model can be improved. Moreover, the summary generation model trained by this embodiment can generate document summaries that are correctly expressed and well summarized, but have a different word order from the reference summary.

[0240] <Example 1>

[0241] The following will be combined Figure 6 The method of generating a document summary by training the summary generation model in the first embodiment is further described. Step S6001: obtaining a plurality of training samples.

[0242] The training sample of this embodiment includes a document and a corresponding reference abstract. Each reference abstract includes a plurality of reference words, and each reference word may have a word order in the corresponding training sample.

[0243] Step S6002, based on each training sample, using the undetermined parameters of the summary generation model as variables, obtain a probability score expression of each reference word in each reference summary at the corresponding decoding time.

[0244] When the number of training samples in this round of training summary generation model is N, the pending parameter of summary generation model can be set to X, and the probability expression of the t-th reference word in the reference summary of the n-th training sample at the corresponding decoding time can be expressed as: P n,t (X), n∈[1,N].

[0245] Step S6003, obtaining the probability of the set dictionary word at the decoding time corresponding to each first reference word according to the probability expression of the first reference word in the reference summary of each training sample at the corresponding decoding time.

[0246] The probability expression of the first reference word in the reference summary of the nth training sample at the corresponding decoding time can be expressed as P n,1 (X), n∈[1,N].

[0247] Step S6004, respectively determining the decoding time corresponding to each first reference word, and whether a set number of set dictionary words with the highest probability hit the corresponding first reference word, to obtain the hit result of each first reference word.

[0248] Step S6005, obtaining the penalty factor of each training sample according to the hit result of each first reference word.

[0249] If the hit result of the first reference word of the reference summary of the nth training sample is yes, the penalty factor of the nth training sample is set to a set value of 0.

[0250] If the hit result of the first reference word of the reference summary of the nth training sample is no, the probability score expression of the first reference word of the reference summary of the nth training sample at the corresponding decoding time is expressed as P n,1 (X), then the penalty factor for the nth training sample can be set to -log P n,1 (X), n∈[1,N].

[0251] Step S6006, determining the loss term corresponding to each training sample according to the probability score expression of each reference word in each training sample at the corresponding decoding time.

[0252] The loss term corresponding to the nth training sample can be:

[0253] Step S6007, according to the preset weights, weighted sums are performed on the loss terms and penalty factors corresponding to each training sample to obtain the initial loss expression corresponding to each training sample.

[0254] The weight of the loss term can be set to λ1, and the weight of the penalty factor can be set to λ2. The initial loss expression corresponding to the nth training sample can be expressed as:

[0255]

[0256] Step S6008, summing the initial loss expression of each training sample to obtain a loss function.

[0257] The loss function can be expressed as

[0258] Step S6009, determining the undetermined parameters according to the loss function, and completing this round of training of the summary generation model.

[0259] Step S6010, obtaining the target document to be processed.

[0260] Step S6011, using the summary generation model trained through steps S6001 to S6009 to process the target document to obtain a corresponding document summary.

[0261] <Example 2>

[0262] Example 2 is based on Example 1 and explains an implementation step to solve the problem that the document summaries generated by the summary generation model obtained in Example 1 may be relatively simple. Figure 7 As shown, it may include:

[0263] Step S7001, based on each training sample, obtain the probability score expression of each reference word in each training sample at each decoding time of the corresponding training sample.

[0264] The probability expression of the tth (t∈[1,T]) reference word in the nth training sample at the ith (i∈[1,T]) decoding time of the nth training sample can be expressed as P n,t,i .

[0265] Step S7002, summing up the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, to obtain the probability and expression corresponding to each reference word in each training sample.

[0266] The probability and expression corresponding to the tth (t∈[1,T])th reference word in the nth training sample can be expressed as:

[0267] Step S7003, obtaining the bag-of-words loss expression corresponding to each training sample according to the probability and expression corresponding to each reference word in each training sample.

[0268] The bag-of-words loss expression for the nth training sample can be expressed as:

[0269] Step S7004, according to the preset weights, weighted summation is performed on the bag-of-words loss expression corresponding to each training sample and the initial loss expression of each training sample obtained in step S5007 in Example 1 to obtain a new loss expression for each training sample.

[0270] The weight of the initial loss expression can be set to λ3, the weight of the bag-of-words loss expression can be set to λ4, and the new loss expression for the nth training sample can be expressed as:

[0271]

[0272] Step S7005, summing the new loss expression of each training sample to obtain a new loss function.

[0273] The new loss function can be expressed as

[0274] Step S7006, re-determine the pending parameters according to the new loss function, and complete the fine-tuning training of the summary generation model.

[0275] <Example 3>

[0276] The following will be combined Figure 8The method of generating a document summary by training the summary generation model in the second embodiment is further described.

[0277] Step S8001, obtaining multiple training samples.

[0278] The training sample of this embodiment includes a document and a corresponding reference abstract. Each reference abstract includes a plurality of reference words. Each reference word may have a word order in the corresponding training sample or may not have a word order.

[0279] Step S8002: According to each training sample, the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample is obtained by taking the undetermined parameters of the summary generation model as variables.

[0280] The probability expression of the tth (t∈[1,T]) reference word in the nth training sample at the ith (i∈[1,T]) decoding time of the nth training sample can be expressed as P n,t,i .

[0281] Step S8003, respectively summing the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample to obtain the probability and expression corresponding to each reference word in each training sample.

[0282] The probability and expression corresponding to the tth (t∈[1,T])th reference word in the nth training sample can be expressed as:

[0283] Step S8004, obtaining the bag-of-words loss expression corresponding to each training sample according to the probability and expression corresponding to each reference word in each training sample.

[0284] The bag-of-words loss expression for the nth training sample can be expressed as:

[0285] Step S8005, summing up the bag-of-words loss expression corresponding to each training sample to obtain a new loss function.

[0286] The new loss function can be expressed as

[0287] Step S8006, re-determine the pending parameters according to the new loss function, and complete the fine-tuning training of the summary generation model.

[0288] Step S8007, obtaining the target document to be processed.

[0289] Step S8008, using the summary generation model trained through steps S8001 to S8006 to process the target document to obtain a corresponding document summary.

[0290] <Document Summary Generation Device Embodiment>

[0291] In this embodiment, a document summary generation device 9000 is provided, such as Fig. 9 As shown, it includes a document acquisition module 9100, a model acquisition module 9200 and a document processing module 9300. The document acquisition module 9100 is used to acquire a target document to be processed; the model acquisition module 9200 is used to acquire a summary generation model for processing the target document; and the document processing module 9300 is used to process the target document using the summary generation model to obtain a corresponding document summary.

[0292] In the first embodiment, the method in which the model acquisition module 9200 acquires the target document to be processed includes:

[0293] Acquire multiple training samples, where each training sample includes a document and a corresponding reference summary, where the reference summary is a sequence consisting of multiple reference words;

[0294] According to each training sample, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained by taking the undetermined parameters of the summary generation model as variables;

[0295] According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the penalty factor corresponding to each training sample is obtained;

[0296] According to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, a loss function is constructed; and the undetermined parameters are determined based on the loss function.

[0297] In one embodiment, obtaining the penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time may include:

[0298] According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the probability of the set dictionary word at the decoding time corresponding to each first reference word is obtained;

[0299] Determine the decoding time corresponding to each first reference word respectively, whether a set number of set dictionary words with the greatest probability hit the corresponding first reference word, and obtain the hit result of each first reference word;

[0300] According to the hit result of each first reference word, the penalty factor of the corresponding training sample is obtained.

[0301] In one embodiment, if the hit result of the first reference word is yes, the penalty factor of the corresponding training sample is set to a set value;

[0302] If the hit result of the first reference word is negative, the penalty factor of the corresponding training sample is set according to the probability score expression of the first reference word with a negative hit result at the corresponding decoding time.

[0303] In one embodiment, according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, constructing the loss function may also include:

[0304] According to the probability expression of each reference word in each training sample at the corresponding decoding time, the loss term corresponding to each training sample is obtained;

[0305] According to the preset weights, the loss term and penalty factor corresponding to each training sample are weighted summed to obtain the initial loss expression corresponding to each training sample;

[0306] According to the initial loss expression of each training sample, the loss function is obtained.

[0307] In one embodiment, determining the undetermined parameters according to the loss function may further include:

[0308] Set the initial value of the pending parameter;

[0309] Substitute the undetermined parameters after setting the initial values ​​into the loss function and perform iterative processing;

[0310] When the undetermined parameters obtained by the iterative process meet the convergence conditions, the iterative process is terminated and the values ​​of the undetermined parameters are determined; otherwise, the iterative process is continued.

[0311] In one embodiment, the convergence conditions include:

[0312] The number of iterative processing reaches a preset number threshold; and / or,

[0313] Substitute the undetermined parameters obtained by iterative processing into the loss function, and the obtained training result value is less than or equal to the preset result threshold.

[0314] In one embodiment, the trained summary generation model can be used to generate a summary of a target document to be processed.

[0315] In one embodiment, the model acquisition module 9200 may further include:

[0316] According to each training sample, a probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample is obtained;

[0317] According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained;

[0318] Construct a new loss function based on the loss function and the bag-of-words loss expression corresponding to each training sample;

[0319] The pending parameters are re-determined according to the new loss function to complete the fine-tuning training of the summary generation model.

[0320] In one embodiment, obtaining the bag-of-words loss expression corresponding to each training sample according to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample may also include:

[0321] Sum the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample to obtain the probability and expression corresponding to each reference word in each training sample;

[0322] According to the probability and expression corresponding to each reference word in each training sample, the bag-of-words loss expression corresponding to each training sample is obtained.

[0323] In one embodiment, the model acquisition module 9200 may further include:

[0324] A module for executing the steps of training a summary generation model according to a preset training cycle.

[0325] In the second embodiment, the method in which the model acquisition module 9200 acquires the target document to be processed includes:

[0326] Acquire multiple training samples, wherein each training sample includes a document and a corresponding reference summary, and the reference summary includes multiple reference words;

[0327] According to each training sample, the undetermined parameters of the summary generation model are used as variables to obtain the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample;

[0328] According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained;

[0329] According to the bag-of-words loss expression corresponding to each training sample, a new loss function is constructed; and the pending parameters are re-determined according to the new loss function.

[0330] Those skilled in the art should understand that the document summary generating device 9000 can be implemented in various ways. For example, the document summary generating device 9000 can be implemented by configuring the processor with instructions. For example, the instructions can be stored in a ROM, and when the device is started, the instructions are read from the ROM into a programmable device to implement the document summary generating device 9000. For example, the document summary generating device 9000 can be solidified into a dedicated device (such as an ASIC). The document summary generating device 9000 can be divided into independent units, or they can be combined together for implementation. The document summary generating device 9000 can be implemented by one of the various implementations described above, or can be implemented by a combination of two or more of the various implementations described above.

[0331] In this embodiment, the document summary generating device 9000 can have multiple implementation forms. For example, the document summary generating device 9000 can be a functional module running in any software product or application that provides network access services, or it can be an external embedded component, plug-in, patch, etc. of these software products or applications, or it can also be these software products or applications themselves.

[0332] <Electronic equipment>

[0333] In this embodiment, an electronic device 1000 is also provided. The electronic device 1000 may be Figure 1 The server 1100 shown or Figure 2 The terminal device 1200 is shown.

[0334] On the one hand, the electronic device 1000 may include the aforementioned document summary generating device 8000, which is used to implement the document summary generating method of any embodiment of the present invention; or include the aforementioned document summary generating device 9000, which is used to implement the document summary generating method of any embodiment of the present invention.

[0335] On the other hand, Fig.10 As shown, the electronic device 1000 may further include a processor 1300 and a memory 1400, wherein the memory 1400 is used to store executable instructions; the processor 1300 is used to control the electronic device 1000 to execute the method for generating a document summary according to any embodiment of the present invention under the control of the instructions.

[0336] In this embodiment, the electronic device 1000 may be a mobile phone, a tablet computer, a PDA, a desktop computer, a laptop computer, a workstation, a game console, etc. For example, the electronic device 1000 may be a smart phone installed with an application providing a display service.

[0337] <Computer Readable Storage Medium>

[0338] In this embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating a document summary as described in any embodiment of the present invention is implemented.

[0339] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0340] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0341] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0342] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0343] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0344] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0345] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0346] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0347] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A method for generating a document summary, wherein: include: Get the target document to be processed; Acquire a summary generation model for processing the target document; Processing the target document using the summary generation model to obtain a corresponding document summary; The summary generation model is trained by the following method: Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words; According to each training sample, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained by taking the undetermined parameters of the summary generation model as variables; According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the penalty factor corresponding to each training sample is obtained; According to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, a loss function is constructed, and the undetermined parameters are determined according to the loss function.

2. The method according to claim 1, wherein: The step of obtaining the penalty factor corresponding to each training sample according to the probability expression of the first reference word in each reference summary at the corresponding decoding time comprises: According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the probability of the set dictionary word at the decoding time corresponding to each first reference word is obtained; Determine the decoding time corresponding to each first reference word respectively, whether a set number of set dictionary words with the greatest probability hit the corresponding first reference word, and obtain the hit result of each first reference word; According to the hit result of each first reference word, the penalty factor of the corresponding training sample is obtained.

3. The method according to claim 2, wherein: If the hit result of the first reference word is yes, the penalty factor of the corresponding training sample is set to the set value; If the hit result of the first reference word is negative, the penalty factor of the corresponding training sample is set according to the probability score expression of the first reference word with a negative hit result at the corresponding decoding time.

4. The method according to claim 1, wherein: The step of constructing a loss function according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample includes: According to the probability expression of each reference word in each training sample at the corresponding decoding time, the loss term corresponding to each training sample is obtained; According to the preset weights, the loss term and penalty factor corresponding to each training sample are weighted summed to obtain the initial loss expression corresponding to each training sample; The loss function is obtained according to the initial loss expression of each training sample.

5. The method according to claim 1, wherein: The step of determining the undetermined parameters according to the loss function comprises: Setting the initial value of the pending parameter; Substituting the undetermined parameters after setting the initial values ​​into the loss function for iterative processing; When the undetermined parameter obtained by the iterative process meets the convergence condition, the iterative process is terminated to determine the value of the undetermined parameter; otherwise, the iterative process is continued.

6. The method according to claim 5, wherein: The convergence conditions include: The number of iterative processing reaches a preset number threshold; and / or, Substituting the undetermined parameters obtained by the iterative processing into the loss function, the obtained training result value is less than or equal to the preset result threshold.

7. The method according to claim 1, wherein: The trained summary generation model is used to generate the summary of the target document to be processed.

8. The method according to any one of claims 1 to 7, wherein: After determining the undetermined parameter according to the loss function, the method further includes: According to each training sample, a probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample is obtained; According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained; Constructing a new loss function according to the loss function and the bag-of-words loss expression corresponding to each training sample; The undetermined parameters are re-determined according to the new loss function to complete the fine-tuning training of the summary generation model.

9. The method according to claim 8, wherein: The step of obtaining the bag-of-words loss expression corresponding to each training sample according to the probability expression of each reference word in each training sample at each decoding moment of the corresponding training sample comprises: Sum the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample to obtain the probability and expression corresponding to each reference word in each training sample; According to the probability and expression corresponding to each reference word in each training sample, the bag-of-words loss expression corresponding to each training sample is obtained.

10. The method according to any one of claims 1 to 7, wherein: The method further comprises: Provide a setting entry for setting application scenarios of summary generation models; Obtain the application scenario input through the setting entry; The step of obtaining a plurality of training samples includes: obtaining corresponding training samples according to the input application scenario.

11. The method according to any one of claims 1 to 7, wherein: The method further comprises: Determine the application scenarios for which the trained summary generation model is suitable; Find application projects that match the application scenario; The trained summary generation model is input into the application project.

12. The method according to claim 1, wherein: The target document, the document summary, and the training sample are of the same language type.

13. The method according to claim 12, wherein: The language type is Chinese.

14. The method according to claim 1, wherein: The target document is a travel note, an article, news or a product manual.

15. A method for generating a document summary, wherein: include: Get the target document to be processed; Acquire a summary generation model for processing the target document; Processing the target document using the summary generation model to obtain a corresponding document summary; The summary generation model is trained by the following method: Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words; According to each training sample, the undetermined parameters of the summary generation model are used as variables to obtain the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample; According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained; Constructing a new loss function according to the bag-of-words loss expression corresponding to each training sample; and re-determining the pending parameters according to the new loss function; Among them, before redetermining the pending parameters according to the new loss function, the method also includes: obtaining the penalty factor corresponding to each training sample according to the probability score expression of the first reference word in each reference summary at the corresponding decoding time; constructing a loss function according to the probability score expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, and determining the pending parameters based on the loss function.

16. A device for generating a document summary, wherein: include: A document acquisition module is used to acquire the target document to be processed; A model acquisition module, used to acquire a summary generation model for processing the target document; A document processing module, used to process the target document using the summary generation model to obtain a corresponding document summary; Wherein, the model acquisition module is also used for: Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words; According to each training sample, the probability expression of each reference word in each reference summary at the corresponding decoding time is obtained by taking the undetermined parameters of the summary generation model as variables; According to the probability expression of the first reference word in each reference summary at the corresponding decoding time, the penalty factor corresponding to each training sample is obtained; Constructing a loss function according to the probability expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, and determining the undetermined parameter according to the loss function; Alternatively, the model acquisition module is further used for: Acquire multiple training samples, wherein each of the training samples includes a document and a corresponding reference summary, and the reference summary includes multiple reference words; According to each training sample, the undetermined parameters of the summary generation model are used as variables to obtain the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample; According to the probability expression of each reference word in each training sample at each decoding time of the corresponding training sample, the bag-of-words loss expression corresponding to each training sample is obtained; Constructing a new loss function according to the bag-of-words loss expression corresponding to each training sample; and re-determining the pending parameters according to the new loss function; Among them, before redetermining the pending parameters according to the new loss function, the model acquisition module is also used to: obtain the penalty factor corresponding to each training sample according to the probability score expression of the first reference word in each reference summary at the corresponding decoding time; construct a loss function according to the probability score expression of each reference word at the corresponding decoding time and the penalty factor corresponding to each training sample, and determine the pending parameters based on the loss function.

17. An electronic device, wherein: include: The device according to claim 16; or, A processor and a memory, wherein the memory is used to store instructions, and the instructions are used to control the processor to execute the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 15 when executed by a processor.

Images