Text generation method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a text generation method and device, computer equipment and a computer readable storage medium. The method comprises the steps that a multi-source information set is acquired; preprocessing the multi-source information set to obtain first text information; and performing information extraction and fusion processing on the first text information to obtain target text information. The summary text can be generated in combination with all information in the multi-source information set, and the quality of the generated summary text is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text generation method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] Multi-source document data refers to data from different sources on the same topic. Multi-source document data can be found everywhere in people's daily lives, for example, multiple reports on a hot event. Multi-source document data contains a lot of information. In order to save users' time and energy in reading multi-source document data, it is necessary to summarize the information in the multi-source document data into an overview. However, existing overview generation methods based on multi-source document data have the problem of poor quality of the generated overview. Summary of the invention

[0003] The embodiments of the present application provide a text generation method, apparatus, computer device and computer-readable storage medium, which can generate an overview text by combining all information in a multi-source information set, thereby improving the quality of the generated overview text.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] On the one hand, the present application provides a text generation method, comprising:

[0006] Obtain multi-source information collection;

[0007] Pre-processing the multi-source information set to obtain first text information;

[0008] The first text information is subjected to information extraction and fusion processing to obtain target text information.

[0009] In some implementation schemes of the present application, pre-processing is performed on the multi-source information set to obtain the first text information, including:

[0010] Translating the multi-source information set to obtain second text information;

[0011] The second text information is subjected to information filtering processing to obtain the first text information.

[0012] In some embodiments of the present application, performing information filtering processing on the second text information to obtain the first text information includes:

[0013] Performing a first filtering process on the second text information to obtain third text information, wherein the first filtering process is to filter out documents in the second text information whose number of characters is less than a preset character threshold;

[0014] Performing a second filtering process on the third text information to obtain fourth text information, wherein the second filtering process is to filter out duplicate documents in the third text information;

[0015] The fourth text information is subjected to a third filtering process to obtain the first text information, wherein the third filtering process is to filter out repeated titles in the fourth text information.

[0016] In some implementation schemes of the present application, information extraction and fusion processing are performed on the first text information to obtain target text information, including:

[0017] Performing attention distribution extraction on the first text information to obtain first attention distribution information;

[0018] Determining target word distribution information based on the first attention distribution information;

[0019] Based on the target word distribution information, the target text information is determined.

[0020] In some embodiments of the present application, the text generation method is applied to a text generation model, the text generation model includes a decoder, and attention distribution extraction is performed on the first text information to obtain first attention distribution information, including:

[0021] Encoding the first text information to obtain first coded information and second coded information;

[0022] Determining second attention distribution information based on the first encoding information;

[0023] Determining first similarity information based on the second encoding information and output information of the decoder;

[0024] Based on the second encoding information, determining third similarity information between each sentence in the first text information and other sentences;

[0025] normalizing the third similarity information, and determining the second similarity information based on the normalized third similarity information;

[0026] Determining boundary related information based on the first similarity information and the second similarity information;

[0027] Based on the boundary-related information and the second attention distribution information, the first attention distribution information is determined.

[0028] In some embodiments of the present application, the target text information is determined based on the target word distribution information, including:

[0029] Based on the first attention distribution information, determining context information and first word distribution information, where the first word distribution information is used to characterize the probability of the text generation model copying each word in the first text information;

[0030] Determining second word distribution information and weight information based on the context information and the hidden state information of the decoder, wherein the second word distribution information is used to characterize the probability distribution of the first text information on a preset vocabulary set;

[0031] Based on the first word distribution information, the second word distribution information and the weight information, target text information is determined.

[0032] In some implementation schemes of the present application, before extracting and fusing the first text information to obtain the target text information, the following steps are included:

[0033] Obtain a multi-source document dataset, where the multi-source document dataset includes multiple multi-source document sets;

[0034] Pre-processing the plurality of multi-source document sets respectively to obtain a plurality of fifth text information;

[0035] Performing mask processing on each fifth text information to obtain masked text information corresponding to each fifth text information, wherein the masked text information is obtained by performing mask processing on a target mask document in each fifth text information;

[0036] The preset network model is trained based on the masked text information and the target masked document to obtain a text generation model.

[0037] In a second aspect, the present application provides a text generation device, comprising:

[0038] An information acquisition unit, used for acquiring a multi-source information set;

[0039] An information processing unit, used for pre-processing the multi-source information set to obtain first text information;

[0040] The text generation unit is used to perform information extraction and fusion processing on the first text information to obtain target text information.

[0041] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0042] one or more processors;

[0043] Memory; and

[0044] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement any one of the text generation methods in the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the text generation methods in the first aspect.

[0046] The beneficial effects of the present invention are as follows: a multi-source information set is pre-processed to obtain first text information, and then the first text information is extracted and fused to obtain target text information. The target text information can be generated by combining all information in the multi-source information set, making full use of the information contained in all documents in the multi-source information set, avoiding ignoring important information, and improving the quality of the generated overview text. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flowchart of a text generation method provided by an embodiment of the present invention;

[0049] Figure 2 is a flowchart of a specific embodiment of the text generation method provided by an embodiment of the present invention;

[0050] Figure 3 It is a structural diagram of a text generation model provided by an embodiment of the present invention;

[0051] Figure 4 is a flowchart of a specific embodiment of the training process of the text generation model provided by an embodiment of the present invention;

[0052] Figure 5 is a principle block diagram of a text generation device provided by an embodiment of the present invention;

[0053] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third", "fourth", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", "third", "fourth", etc. may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0056] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0057] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0058] The single document summary task is to summarize the key information in a single document, while the multi-document summary task is to summarize the information in multiple related parallel documents into one summary. When processing the single document summary task, the target summary required by the model can be obtained by various methods such as manual annotation, while when processing the multi-document summary task, summarizing multiple related parallel documents into one summary is a difficult annotation task. In order to solve the problem that the target summary in the multi-document summary task is difficult to obtain, the existing technology often uses unsupervised multi-document summarization methods.

[0059] The inventors have found through research that there are two conventional unsupervised multi-document summarization methods: the first is to cluster the sentences in the document collection, then use multiple clustering clusters to generate multiple sub-summaries, and finally merge the multiple sub-summaries to generate a general summary; the second is to select a representative document in the multi-source document collection to summarize it, and use the summary of the single document as the general summary of the multi-source document collection. The first clustering-based method can only retain a large amount of basic information, and often ignores some important information, such as the details of an exclusive report by a certain media, resulting in low quality of the summary text; and the second method is actually a single document summary method, which does not reasonably utilize the information contained in the remaining documents in the multi-source document collection, so the summary obtained is often one-sided.

[0060] Based on this, in an embodiment of the present application, a multi-source information set is obtained; the multi-source information set is pre-processed to obtain first text information; information extraction and fusion processing are performed on the first text information to obtain target text information. An overview text can be generated by combining all the information in the multi-source information set, making full use of the information contained in all documents in the multi-source information set, avoiding ignoring important information, and improving the quality of the generated overview text.

[0061] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0062] This embodiment provides a text generation method, such as Figure 1 As shown in , the method includes:

[0063] Step S10: Acquire a multi-source information set.

[0064] The multi-source information set is a multi-source document dataset to be processed. The multi-source information set contains multiple documents with the same topic but different sources. For example, for a hot event, multiple reports in different languages ​​and different media at home and abroad can be collected. The collection of these reports is the multi-source information set corresponding to the hot event. The multi-source information set contains a lot of information. In order to save users' time and energy in reading the multi-source information set, it is necessary to summarize the information in the multi-source information set into an overview text, namely the target text information.

[0065] Step S20: pre-process the multi-source information set to obtain first text information.

[0066] The first text information is the first text information obtained after pre-processing the multi-source information set. The pre-processing includes but is not limited to translation processing, text filtering processing, title filtering processing, etc. of the multi-source information set. By pre-processing the multi-source information set, the quality of the target text information generated subsequently can be improved.

[0067] In a specific implementation, the pre-processing includes translation processing and information filtering processing. Accordingly, Figure 2 As shown, step S20 includes:

[0068] Step S21, translating the multi-source information set to obtain second text information;

[0069] Step S22: Filter the second text information to obtain the first text information.

[0070] Considering that a multi-source information set may contain information in different languages, for example, when the multi-source information set is a collection of multiple reports in different languages ​​and different media at home and abroad, the multi-source information set may contain information in different languages ​​such as Chinese, English, and Japanese. In order to facilitate the subsequent generation of target text information, this embodiment first translates the multi-source information set, and translates the information in the multi-source information set into information in the same target language, thereby obtaining the second text information. For example, all the information in the multi-source information set is translated into information described in Chinese.

[0071] Considering that the second text information contains a large amount of information, in order to improve the quality and efficiency of generating text, after obtaining the second text information, this embodiment further performs information filtering processing on the second text information to obtain the first text information.

[0072] In a specific implementation, step S22 includes:

[0073] Step S221: performing a first filtering process on the second text information to obtain third text information, wherein the first filtering process is to filter out documents in the second text information whose number of characters is less than a preset character threshold;

[0074] Step S222: performing a second filtering process on the third text information to obtain fourth text information, wherein the second filtering process is to filter out duplicate documents in the third text information;

[0075] Step S223: Perform a third filtering process on the fourth text information to obtain the first text information. The third filtering process is to filter out repeated titles in the fourth text information.

[0076] The first filtering process is used to filter out documents whose characters in the second text information are less than a preset character threshold, that is, the third text information is the text information obtained after filtering out documents whose characters in the second text information are less than the preset character threshold. For example, the character threshold is 100, and the first filtering process is to filter out documents whose characters in the second text information are less than 100 characters.

[0077] The second filtering process is used to filter out duplicate documents in the third text information, that is, the fourth text information is the text information obtained after filtering out duplicate documents in the third text information. This embodiment filters out documents in the second text information whose number of characters is less than a preset character threshold. After obtaining the third text information, the duplicate documents in the third text information are further filtered out to obtain the fourth text information.

[0078] The third filtering process is to filter out repeated titles in the fourth text information, that is, the first text information is the text information obtained after filtering out repeated titles in the fourth text information. After obtaining the fourth text information, this embodiment further filters out repeated titles in the fourth text information to obtain the first text information.

[0079] Step S30: extract and fuse the first text information to obtain target text information.

[0080] The target text information is an overview text of a multi-source information set. After obtaining the first text information, this embodiment performs information extraction and fusion processing on the first text information to obtain the target text information. The overview text can be generated by combining all the information in the multi-source information set, making full use of the information contained in all documents in the multi-source information set, avoiding ignoring important information, and improving the quality of the generated overview text.

[0081] In a specific implementation, continue to refer to Figure 2 As shown, step S30 includes:

[0082] Step S31, extracting attention distribution from the first text information to obtain first attention distribution information;

[0083] Step S32, determining target word distribution information based on the first attention distribution information;

[0084] Step S33: Determine target text information based on target word distribution information.

[0085] The first attention distribution information is used to characterize the importance of each word in the first text information. After obtaining the first text information, this embodiment performs attention distribution extraction on the first text information to obtain the first attention distribution information, and generates an overview text based on the first attention distribution information. When generating the overview text, more attention can be paid to important information in the first text information, thereby improving the quality of the generated overview text.

[0086] The target word distribution information is used to characterize the probability distribution of each word in the first text information in a preset vocabulary set. The target word distribution information includes the probabilities of multiple words in the vocabulary set at multiple positions. The vocabulary set is a set of many pre-set commonly used words. The vocabulary set includes not only the words that appear in the first text information, but also the words that do not appear in the first text information.

[0087] After obtaining the first attention distribution information, this embodiment determines the target word distribution information corresponding to the first text information based on the first attention distribution information, and generates the target text information based on the target word distribution information. It can generate an overview text in combination with all the texts in the first text information, make full use of the information contained in all documents in the first text information, avoid ignoring important information, and improve the quality of the generated overview text.

[0088] The target word distribution information includes the probability of multiple words in multiple positions in the vocabulary. The word with the highest probability corresponding to each position is used as the summary word of the position to obtain the target text information. For example, position x1, x2, ..., x n The corresponding words with the highest probability are A1, A2, ..., A n , then the target text information is {A1,A2,…,A n Generating target text information of a multi-source information set based on target word distribution information can combine all information in the multi-source information set to generate target text information, make full use of information contained in all documents in the multi-source information set, avoid ignoring important information, and improve the quality of the generated overview text.

[0089] In a specific implementation, the text generation method is applied to a text generation model, the text generation model includes a decoder, and step S31 includes:

[0090] Step S311, encoding the first text information to obtain first encoding information and second encoding information;

[0091] Step S312: determining second attention distribution information based on the first coding information;

[0092] Step S313: determining first similarity information based on the second encoding information and the output information of the decoder;

[0093] Step S314: determining third similarity information between each sentence in the first text information and other sentences based on the second coding information;

[0094] Step S315: normalize the third similarity information, and determine the second similarity information based on the normalized third similarity information;

[0095] Step S316: determining boundary related information based on the first similarity information and the second similarity information;

[0096] Step S317: Determine the first attention distribution information based on the boundary-related information and the second attention distribution information.

[0097] The first encoding information is the word encoding information of all words in the first text information obtained by performing word-level encoding on the first text information, that is, the first encoding information contains the word encoding information of each word in each sentence in the first text information. The second encoding information is the sentence-level embedding of all sentences in the first text information obtained by performing sentence-level encoding on the word encoding information of each sentence in the first text information. For example, the second encoding information can be expressed as: in, Represents the sentence-level embedding of the nth sentence in the first text information.

[0098] The second attention distribution information is the importance of each word in the first text information determined based on the first encoding information. In a specific implementation, the text generation method is applied to a text generation model, and the text generation model includes a decoder. Based on the first encoding information, the step of determining the second attention distribution information of the first text information can specifically include: determining the fourth encoding information based on the first encoding information and the hidden state information of the decoder, and applying the fourth encoding information to a preset activation function to obtain the second attention distribution information. For example, when the preset activation function is a softmax function, the determination process of the fourth encoding information can be expressed as: The process of determining the second attention distribution information can be expressed as: t =softmax(e t ), Indicates the fourth coded information, a t represents the second attention distribution information, h i represents the first coded information, d t Represents the hidden state information of the decoder, v, W h , W s and b attn represents learnable parameters.

[0099] In order to weigh the relevance and diversity of the information in the first text information, after determining the second attention distribution information of the first text information, this embodiment further determines the first attention distribution information of the first text information based on the second coding information and the second attention distribution information. That is, the first attention distribution information is the importance of each word in the first text information determined based on the second coding information and the second attention distribution information.

[0100] In a specific implementation, the step of determining the first attention distribution information of the first text information based on the second coding information and the second attention distribution information specifically includes: determining the first similarity information and the second similarity information of the first text information based on the second coding information; determining the boundary-related information of the first text information based on the first similarity information and the second similarity information; determining the first attention distribution information of the first text information based on the boundary-related information and the second attention distribution information.

[0101] The first similarity information is used to characterize the similarity between each sentence in the first text information and the output of the decoder, and the first similarity information is determined based on the embedded representation of each sentence in the first text information and the output information of the last step of the decoder. The determination process of the first similarity information can be expressed as: Sim i represents the first similarity information of the i-th sentence in the first text information, represents the embedding representation of the i-th sentence in the first text information, s sum Represents the output information of the last step of the decoder, W Sim are the parameters for learning.

[0102] Furthermore, the second similarity information is used to characterize the similarity between each sentence in the first text information and other sentences. The process of determining the second similarity information may specifically include: determining the third similarity information between each sentence in the first text information and other sentences based on the second coding information, normalizing the third similarity information to obtain the normalized third similarity information, and determining the second similarity information based on the normalized third similarity information. The calculation formula of the third similarity information is: The process of normalizing the third similarity information can be expressed as: The calculation formula for the second similarity information is: score i =max j (β i,j ), score i represents the second similarity information of the i-th sentence in the first text information, v i,j represents the third similarity information between the i-th sentence and the j-th sentence in the first text information, β i,j represents the normalized third similarity information, represents the embedding representation of the i-th sentence in the first text information, represents the embedding representation of the jth sentence in the first text information, W self represents learnable parameters.

[0103] Specifically, the boundary-related information is a maximum marginal relevance (MMR) score of the first text information determined based on the first similarity information and the second similarity information. In a specific implementation, the text generation model includes an encoder, a decoder, and a related information extraction module, such as Figure 3 As shown, all words {w1, ..., w k-m ,…,w N} Input the encoder, and output the second encoded information through the encoder At the same time, all the words {w1, ..., w k-m ,…,w N} Input decoder, output s through encoder sum , and then the second coded information and sum The relevant information extraction module is input, and the boundary relevant information of the first text information is output through the relevant information extraction module.

[0104] The calculation formula for boundary-related information is: MMR i =λSim i -(1-λ)score i , MMR i represents the boundary related information of the ith sentence in the first text information, Sim i Indicates the first similarity information of the i-th sentence in the first text information, score i represents the second similarity information of the i-th sentence in the first text information, and λ represents the adjustment factor.

[0105] After obtaining the boundary-related information, the first attention distribution information can be determined based on the boundary-related information and the second attention distribution information. Specifically, the step of determining the first attention distribution information based on the boundary-related information and the second attention distribution information can specifically include: normalizing the boundary-related information to obtain the normalized boundary-related information, and determining the first attention distribution information based on the normalized boundary-related information and the second attention distribution information. The normalization process of the boundary-related information can be expressed as: The calculation formula of the first attention distribution information is: represents the first attention distribution information, a t represents the second attention distribution information, represents the normalized boundary-related information of the i-th sentence in the first text information, MMR i Represents the boundary-related information of the i-th sentence in the first text information.

[0106] In a specific implementation, step S311 includes:

[0107] Step S3111: Encode the acquired first text information to obtain first encoding information, where the first encoding information includes word encoding information of each word in the first text information;

[0108] Step S3112: Obtain target word encoding information from the first encoding information, where the target word encoding information is the word encoding information of the last word of each sentence in the first text information;

[0109] Step S3113, determining third encoding information based on the target word encoding information;

[0110] Step S3114: encode the third coded information to obtain second coded information.

[0111] The first coding information mentioned in the above steps is the word coding information of each word in the first text information obtained by performing word-level coding on the first text information. When determining the first coding information in this embodiment, the first text information can be concatenated into a sentence set, and then each sentence in the sentence set is word-level coded to obtain the first coding information. For example, the first text information is concatenated into a sentence set D = {s1, s2, ..., s n}, and then use the Long Short-Term Memory (LSTM) network to train each sentence s in the sentence set D i ={w k-m , w k-m+1 ,…,w k}Perform word-level encoding to obtain the first encoding information.

[0112] The target word encoding information is the word encoding information of the last word of each sentence in the first text information, and the third encoding information is the word-level sentence embedding of the first text information determined based on the target word encoding information. For example, the third encoding information can be expressed as: Indicates the nth sentence s in the first text message n word-level sentence embeddings, Indicates the nth sentence s in the first text message n The word encoding information of the last word.

[0113] Considering that each sentence in the first text information includes multiple words, directly encoding all the word encoding information of each sentence will result in a large amount of calculation in the encoding process. After obtaining the first encoding information, this embodiment obtains the word encoding information of the last word of each sentence in the first text information from the first encoding information, and determines the word encoding information of the last word of each sentence in the first text information as the word-level sentence embedding of the sentence, thereby obtaining the third encoding information.

[0114] The second coded information is the coded information obtained by performing sentence-level coding processing on the third coded information. After obtaining the third coded information in this embodiment, the third coded information can be subjected to sentence-level coding processing to obtain the second coded information. Input Long Short-Term Memory (LSTM) network, LSTM network embeds each word-level sentence in the third encoding information Can output its hidden state Thus, the second coded information is obtained

[0115] In a specific implementation, step S32 includes:

[0116] Step S321: determining context information and first word distribution information based on the first attention distribution information, where the first word distribution information is used to characterize the probability of a text generation model copying each word in the first text information;

[0117] Step S322: determining second word distribution information and weight information based on the context information and the hidden state information of the decoder, wherein the second word distribution information is used to characterize the probability distribution of the first text information on a preset vocabulary set;

[0118] Step S323: Determine the target text information based on the first word distribution information, the second word distribution information and the weight information.

[0119] The context information represents the context vector of the average semantics of the relevant document content in the first text information in the decoding step. When the context information is determined based on the first attention distribution information, the context information is determined based on the first attention distribution information and the first encoding information. The calculation formula of the context information is: represents the context information of the t-th decoding step, represents the first attention distribution information, Represents the first coded information.

[0120] The first word distribution information is used to characterize the probability of the text generation model copying each word in the first text information. The first word distribution information contains the probability of the text generation model copying each word in the first text information. The probability of the text generation model copying each word in the first text information is the sum of the attention weights of the word in the first text information, and the attention weight of each word in the first text information can be obtained from the first attention distribution information. The calculation formula for the probability of copying each word in the first text information is: P copy represents the first word distribution information, represents the attention weight of the i-th word in the first text information, w i represents the i-th word in the first text information. For example, the word "today" appears three times in the first text information and the attention weights are 0.1, 0.15, and 0.12 respectively, then the copy probability of the word P copy =0.1+0.15+0.12.

[0121] The second word distribution information is used to characterize the probability distribution of the first text information on a preset vocabulary set. The vocabulary set mentioned in the above steps is a set of many commonly used words that are preset. The vocabulary set not only includes words that appear in the first text information, but also includes words that do not appear in the first text information. After obtaining the context information, this embodiment inputs the context information and the hidden state information of the decoder into two linear layers to establish the second word distribution information. Among them, the calculation formula of the second word distribution information is: d t Indicates hidden status information. represents the context information, V′, V, b, and b′ are learnable parameters.

[0122] The weight information indicates the probability of sampling and generating a word from a preset vocabulary set or copying a word from the first text information when the text generation model generates the target text information, wherein the calculation formula of the weight information is: P gen represents weight information, d t Indicates hidden status information. represents context information, x t represents the input of the decoder, and b ptr represents learnable parameters.

[0123] After determining the first word distribution information, the second word distribution information and the weight information, the target word distribution information corresponding to the first text information can be determined based on the first word distribution information, the second word distribution information and the weight information. The calculation formula of the target word distribution information is: P(w) = P gen Pvocab (w)+(1-P gen ) copy (w), P(w) represents the target word distribution information, P vocab (w) represents the first word distribution information, P copy (w) represents the second word distribution information, P gen Represents weight information.

[0124] In a specific implementation, Figure 4 As shown, before step S30, the following steps are included:

[0125] Step S41, obtaining a multi-source document dataset, where the multi-source document dataset includes a plurality of multi-source document sets;

[0126] Step S42, pre-processing the plurality of multi-source document sets respectively to obtain a plurality of fifth text information;

[0127] Step S43: performing mask processing on each fifth text information to obtain masked text information corresponding to each fifth text information, wherein the masked text information is obtained by performing mask processing on the target mask document in each fifth text information;

[0128] Step S44: training a preset network model based on the masked text information and the target masked document to obtain a text generation model.

[0129] The multi-source document dataset includes multiple multi-source document sets, each of which contains multiple documents on the same topic but from different sources. For example, the multi-source document dataset C = {C1, C2, ..., C N}, each multi-source document set C i (i=1,2,…,N) contains multiple documents C with the same topic but from different sources il (l=1,2,…,K).

[0130] The fifth text information is a processed multi-source document set obtained by pre-processing the multiple multi-source document sets respectively. The pre-processing process of the multiple multi-source document sets is the same as the process of pre-processing the multi-source information set in step S20. For details, please refer to the aforementioned step S20.

[0131] The target mask document is a document that needs to be masked from multiple documents of the fifth text information, and the masked text information is the masked fifth text information obtained by masking the target mask document in the fifth text information. ik , the mask text information is for the fifth text information C i Document C in ik Perform masking to obtain the fifth text information C after maskingi .

[0132] The text generation model is obtained by training the preset network model based on the mask text information and the target mask document. The structure of the text generation model is the same as that of the preset network model. The difference between the text generation model and the preset network model is that the model parameters of the preset network model are the initial model parameters, and the model parameters of the text generation model are the trained model parameters. Among them, the preset network model can adopt a deep learning model or a machine learning model, and the preset network model includes an encoder and a decoder.

[0133] The aforementioned steps mentioned that the text generation method is applied to the text generation model, therefore, the text generation model needs to be obtained before step S10. In order to obtain the text generation model, the present embodiment obtains a multi-source document data set, and then pre-processes the multiple multi-source document sets respectively to obtain multiple fifth text information, and then performs mask processing on each fifth text information to obtain the masked text information corresponding to each fifth text information, and finally trains the preset network model based on the masked text information and the target masked document to obtain the text generation model. The preset network model is trained using the masked text information and the target masked document, and the target text information generated by the obtained text generation model not only contains the central information of the document set, but also contains the unique information in other source news in the same topic set.

[0134] In a specific implementation, a preset network model is trained based on masked text information and a target masked document, and the step of obtaining a text generation model may specifically include: inputting the masked text information into the preset network model, performing attention distribution extraction on the masked text information through the preset network model, and obtaining third attention distribution information of the masked text information; based on the third attention distribution information, determining the predicted word distribution information corresponding to the masked text information; determining the predicted probability of each word in the target masked document based on the predicted word distribution information; determining the loss value based on the predicted probability of each word in the target masked document and the loss function of the preset network model; when the loss value does not meet the preset conditions, correcting the model parameters of the preset network model according to the preset parameter learning rate; and continuing to perform the steps of inputting the masked text information into the preset network model, performing attention distribution extraction on the masked text information through the preset network model, and obtaining the third attention distribution information of the masked text information, until the loss value meets the preset conditions.

[0135] The loss value satisfies the preset condition that the loss value is less than a preset first threshold, or the difference between the loss values ​​obtained twice is less than a preset second threshold. The loss value is calculated as follows: Loss represents the loss value. Represents the word in the target mask document The predicted probability of , T represents the number of words in the target mask document.

[0136] It should be noted that the step of performing attention distribution extraction on the masked text information to obtain the third attention distribution information of the masked text information is the same as the aforementioned step S31, and the step of determining the predicted word distribution information corresponding to the masked text information based on the third attention distribution information is the same as the aforementioned step S32. For details, please refer to the aforementioned steps S31 and S32, and this application will not repeat them here.

[0137] In a specific implementation, each fifth text message includes multiple documents, and further referring to Figure 4 As shown, step S43 includes:

[0138] Step S431, determining vocabulary overlap information of each document in the plurality of documents, where the vocabulary overlap information is used to characterize the degree of vocabulary overlap between each document and other documents in the plurality of documents;

[0139] Step S432, determining a target mask document from multiple documents based on vocabulary overlap information;

[0140] Step S433: mask each fifth text information based on the target mask document to obtain mask text information corresponding to each fifth text information.

[0141] Each fifth text information includes multiple documents, and the vocabulary overlap information is used to characterize the degree of vocabulary overlap between each document in the fifth text information and other documents in the fifth text information. Considering that the fifth text information itself may contain noise, randomly selecting the target mask document from the fifth text information may easily use the noisy document as the target mask document, resulting in the summary text generated by the text generation model being of low quality.

[0142] In order to improve the training effect of the text generation model, this embodiment determines the vocabulary overlap information of each document in multiple documents, and then selects the document with the highest vocabulary overlap from the multiple documents as the target mask document of the fifth text information, and then performs mask processing on the fifth text information based on the target mask document, so as to obtain the mask text information corresponding to the fifth text information. For example, the fifth text information contains document A, document B, document C and document D, and the vocabulary overlap information is 0.6, 0.3, 0.5, and 0.8 respectively, then document D is used as the target mask document of the fifth text information, and document D in the fifth text information is masked.

[0143] In order to better implement the text generation method in the embodiment of the present application, based on the text generation method, the embodiment of the present application also provides a text generation device, such as Figure 5 As shown, the text generation device includes:

[0144] An information acquisition unit 610 is used to acquire a multi-source information set;

[0145] The information processing unit 620 is used to pre-process the multi-source information set to obtain the first text information;

[0146] The text generation unit 630 is used to perform information extraction and fusion processing on the first text information to obtain target text information.

[0147] In an embodiment of the present application, a multi-source information set is pre-processed to obtain first text information, and then the first text information is subjected to information extraction and fusion processing to obtain target text information. The target text information can be generated by combining all information in the multi-source information set, making full use of the information contained in all documents in the multi-source information set, avoiding ignoring important information, and improving the quality of the generated overview text.

[0148] In some embodiments of the present application, the information processing unit 620 is specifically used to:

[0149] Translating the multi-source information set to obtain second text information;

[0150] The second text information is subjected to information filtering processing to obtain the first text information.

[0151] In some embodiments of the present application, the information processing unit 620 is further configured to:

[0152] Performing a first filtering process on the second text information to obtain third text information, wherein the first filtering process is to filter out documents in the second text information whose number of characters is less than a preset character threshold;

[0153] Performing a second filtering process on the third text information to obtain fourth text information, wherein the second filtering process is to filter out duplicate documents in the third text information;

[0154] The fourth text information is subjected to a third filtering process to obtain the first text information, wherein the third filtering process is to filter out repeated titles in the fourth text information.

[0155] In some embodiments of the present application, the text generation unit 630 is specifically used to:

[0156] Performing attention distribution extraction on the first text information to obtain first attention distribution information;

[0157] Determining target word distribution information based on the first attention distribution information;

[0158] Based on the target word distribution information, the target text information is determined.

[0159] In some embodiments of the present application, the text generation unit 630 is further configured to:

[0160] Encoding the first text information to obtain first coded information and second coded information;

[0161] Determining second attention distribution information based on the first encoding information;

[0162] Determining first similarity information based on the second encoding information and output information of the decoder;

[0163] Based on the second encoding information, determining third similarity information between each sentence in the first text information and other sentences;

[0164] normalizing the third similarity information, and determining the second similarity information based on the normalized third similarity information;

[0165] Determining boundary related information based on the first similarity information and the second similarity information;

[0166] Based on the boundary-related information and the second attention distribution information, the first attention distribution information is determined.

[0167] In some embodiments of the present application, the text generation method is applied to a text generation model, the text generation model includes a decoder, and the text generation unit 630 is specifically further used to:

[0168] Based on the first attention distribution information, determining context information and first word distribution information, where the first word distribution information is used to characterize the probability of the text generation model copying each word in the first text information;

[0169] Determining second word distribution information and weight information based on the context information and the hidden state information of the decoder, wherein the second word distribution information is used to characterize the probability distribution of the first text information on a preset vocabulary set;

[0170] Based on the first word distribution information, the second word distribution information and the weight information, target word distribution information is determined.

[0171] In some embodiments of the present application, the text generation device further includes:

[0172] A data acquisition unit, used to acquire a multi-source document data set, wherein the multi-source document data set includes a plurality of multi-source document sets;

[0173] A first processing unit is used to pre-process the multiple multi-source document sets respectively to obtain multiple fifth text information;

[0174] A second processing unit is used to perform mask processing on each fifth text information to obtain mask text information corresponding to each fifth text information, where the mask text information is obtained by performing mask processing on a target mask document in each fifth text information;

[0175] The model training unit is used to train a preset network model based on masked text information and a target masked document to obtain a text generation model.

[0176] The embodiment of the present application further provides a computer device, which integrates any text generation device provided in the embodiment of the present application, and the computer device includes:

[0177] one or more processors;

[0178] Memory; and

[0179] One or more applications, wherein the one or more applications are stored in a memory and are configured to execute, by a processor, the steps of the text generation method in any of the above-mentioned text generation method embodiments.

[0180] The present application also provides a computer device that integrates any of the text generation devices provided in the present application. Figure 6 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0181] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0182] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0183] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0184] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0185] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0186] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0187] Obtain multi-source information collection;

[0188] Pre-processing the multi-source information set to obtain first text information;

[0189] The first text information is subjected to information extraction and fusion processing to obtain target text information.

[0190] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0191] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any text generation method provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:

[0192] Obtain multi-source information collection;

[0193] Pre-processing the multi-source information set to obtain first text information;

[0194] The first text information is subjected to information extraction and fusion processing to obtain target text information.

[0195] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0196] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0197] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0198] The above is a detailed introduction to a text generation method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text generation method, characterized in that: include: Obtain multi-source information collection; Pre-processing the multi-source information set to obtain first text information; The first text information is subjected to information extraction and fusion processing to obtain target text information.

2. The method according to claim 1, characterized in that The pre-processing of the multi-source information set to obtain the first text information includes: Translating the multi-source information set to obtain second text information; The second text information is subjected to information filtering processing to obtain first text information.

3. The method according to claim 2, characterized in that The filtering the second text information to obtain the first text information includes: Performing a first filtering process on the second text information to obtain third text information, wherein the first filtering process is filtering out documents in the second text information whose number of characters is less than a preset character threshold; Performing a second filtering process on the third text information to obtain fourth text information, wherein the second filtering process is filtering out duplicate documents in the third text information; A third filtering process is performed on the fourth text information to obtain the first text information, wherein the third filtering process is to filter out repeated titles in the fourth text information.

4. The method according to claim 1, characterized in that: The extracting and fusing the first text information to obtain target text information includes: Performing attention distribution extraction on the first text information to obtain first attention distribution information; Determining target word distribution information based on the first attention distribution information; Based on the target word distribution information, target text information is determined.

5. The method according to claim 4, characterized in that The text generation method is applied to a text generation model, the text generation model includes a decoder, and the attention distribution extraction of the first text information to obtain the first attention distribution information includes: Encoding the first text information to obtain first coded information and second coded information; Determining second attention distribution information based on the first encoding information; Determining first similarity information based on the second encoding information and output information of the decoder; Based on the second encoding information, determining third similarity information between each sentence in the first text information and other sentences; normalizing the third similarity information, and determining the second similarity information based on the normalized third similarity information; Determine boundary related information based on the first similarity information and the second similarity information; Based on the boundary-related information and the second attention distribution information, first attention distribution information is determined.

6. The method according to claim 5, characterized in that The determining target word distribution information based on the first attention distribution information includes: Based on the first attention distribution information, determining context information and first word distribution information, where the first word distribution information is used to characterize the probability of the text generation model copying each word in the first text information; Determining second word distribution information and weight information based on the context information and the hidden state information of the decoder, wherein the second word distribution information is used to characterize the probability distribution of the first text information on a preset vocabulary set; Based on the first word distribution information, the second word distribution information and the weight information, target word distribution information is determined.

7. The method according to claim 1, characterized in that Before extracting and fusing the first text information to obtain the target text information, the method includes: Acquire a multi-source document dataset, wherein the multi-source document dataset includes a plurality of multi-source document sets; Pre-processing the plurality of multi-source document sets respectively to obtain a plurality of fifth text information; Performing mask processing on each of the fifth text information to obtain masked text information corresponding to each of the fifth text information, wherein the masked text information is obtained by performing mask processing on a target mask document in each of the fifth text information; A preset network model is trained based on the masked text information and the target masked document to obtain a text generation model.

8. A text generation device, characterized in that: include: An information acquisition unit, used for acquiring a multi-source information set; An information processing unit, configured to pre-process the multi-source information set to obtain first text information; The text generation unit is used to perform information extraction and fusion processing on the first text information to obtain target text information.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the text generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the text generation method according to any one of claims 1 to 7.

Images