A text generation method, device, equipment and storage medium

Abstract

The present application provides a text generation method, device, equipment and storage medium, wherein the method comprises: obtaining a target text in a specified field; using a pre-established text generation model to generate auxiliary reading sentences for each sentence in the target text, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field; and generating a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentences for each sentence in the target text. The text generation method provided by the present application can generate auxiliary reading text for a target text in a specified field, and the generated auxiliary reading text can assist readers in reading, thereby greatly reducing the difficulty of readers in reading the target text, and providing a good user experience.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text generation method, apparatus, device and storage medium. Background Art

[0002] Texts in certain fields are too domain-specific, which makes them difficult for readers to read. For example, medical documents in the medical field are too domain-specific, which makes them difficult for ordinary patients and their families to read. This difficulty causes patients and their families to consult medical staff repeatedly, increasing the workload of medical staff, and patients and their families do not have enough understanding of their conditions, which makes subsequent communication between doctors and patients difficult. Summary of the invention

[0003] In view of this, the present application provides a text generation method, device, equipment and storage medium for generating auxiliary reading text for a target text in a specified field, thereby solving the problem of readers having difficulty reading due to excessive domain specificity. The technical solution is as follows:

[0004] A text generation method, comprising:

[0005] Get the target text of the specified field;

[0006] Generate an auxiliary reading sentence for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field;

[0007] According to the auxiliary reading sentences of each sentence in the target text, a chapter-level auxiliary reading text of the target text is generated.

[0008] Optionally, the training objectives of the text generation model include:

[0009] Make the auxiliary reading sentence generated by the text generation model for the training sentence consistent with the expression of the real auxiliary reading sentence of the training sentence;

[0010] as well as,

[0011] Make each word in the training sentence appear in the auxiliary reading sentence generated by the text generation model for the training sentence, and / or make the auxiliary reading sentence generated by the text generation model for the training sentence consistent with the subject of the retrieval text corresponding to the training sentence, wherein the retrieval text corresponding to the training sentence consists of a preset number of text fragments in the retrieval database that are most relevant to the training sentence.

[0012] Optionally, the real auxiliary reading sentences annotated with the training sentences are obtained by inserting explanatory segments and / or sentence fluency auxiliary segments into the training sentences.

[0013] Optionally, the using a pre-established text generation model to generate an auxiliary reading sentence for each sentence in the target text includes:

[0014] For each target sentence to be generated as an auxiliary reading sentence in the target text:

[0015] Retrieving a preset number of text segments most relevant to the target sentence from a retrieval database, and processing the preset number of text segments most relevant to the target sentence into a long sentence as a retrieval text corresponding to the target sentence;

[0016] An auxiliary reading sentence of the target sentence is generated by using the text generation model, the target sentence and the search text corresponding to the target sentence.

[0017] Optionally, the step of generating an auxiliary reading sentence of the target sentence by using the text generation model, the target sentence and a search text corresponding to the target sentence includes:

[0018] The text generation model is used to generate target words for forming an auxiliary reading sentence of the target sentence from a dictionary, the target sentence and a search text corresponding to the target sentence, so as to obtain an auxiliary reading sentence of the target sentence.

[0019] Optionally, the step of generating target words for forming the auxiliary reading sentence of the target sentence from the dictionary, the target sentence and the search text corresponding to the target sentence one by one includes:

[0020] respectively encoding the target sentence and the search text corresponding to the target sentence;

[0021] At each decoding moment, a feature vector required for decoding at the current decoding moment is determined as a second vector according to the first vector and the encoding result of the target sentence, and a feature vector required for decoding at the current decoding moment is determined as a third vector according to the encoding result of the first vector and the search text corresponding to the target sentence, wherein the first vector is a state vector representing decoded result information output at the current decoding moment;

[0022] According to the first vector, the second vector and the third vector, predict the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively;

[0023] The target word at the current decoding moment is generated according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence.

[0024] Optionally, generating the target word generated at the current decoding moment according to the probabilities that the target word at the current decoding moment is generated in the dictionary, the target sentence, and the search text corresponding to the target sentence, comprises:

[0025] Predicting the probability of each word in the dictionary being generated based on the first vector and the text generated for the target sentence, predicting the probability of each word in the target sentence being generated based on the encoding result of the target sentence and the first vector, and predicting the probability of each word in the search text corresponding to the target sentence being generated based on the encoding result of the search text corresponding to the target sentence and the first vector;

[0026] The target word at the current decoding moment is generated according to the probability of each word in the dictionary being generated, the probability of each word in the target sentence being generated, the probability of each word in the search text corresponding to the target sentence being generated, and the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively.

[0027] Optionally, the training process of the text generation model includes:

[0028] Obtaining a search text corresponding to the training sentence, wherein the search text corresponding to the training sentence is composed of a preset number of text segments in a search database that are most relevant to the training sentence;

[0029] Generate an auxiliary reading sentence of the training sentence by using a text generation model, the training sentence and a search text corresponding to the training sentence;

[0030] Determine a first prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence and the real auxiliary reading sentence annotated with the training sentence;

[0031] Determine a second prediction loss that can reflect the coverage of each word in the training sentence by the auxiliary reading text of the training sentence, and / or determine a third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the search text corresponding to the training sentence;

[0032] Based on the determined prediction loss, the parameters of the text generation model are updated.

[0033] Optionally, generating a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text includes:

[0034] splicing the auxiliary reading sentences of each sentence in the target text according to the order of each sentence in the target text to obtain a text to be edited;

[0035] The text to be edited is edited using a pre-established chapter editing model, and the edited text is used as the chapter-level auxiliary reading text of the target text, wherein the chapter editing model is trained using training text annotated with real editing results.

[0036] Optionally, the editing of the text to be edited using a pre-established chapter editing model includes:

[0037] The redundant words and / or incoherent words in the text to be edited are deleted using a pre-established paragraph editing model.

[0038] A text generation device, comprising: a text acquisition module, an auxiliary reading sentence generation module and an auxiliary reading text generation module;

[0039] The text acquisition module is used to acquire the target text in a specified field;

[0040] The auxiliary reading sentence generation module is used to generate auxiliary reading sentences for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field;

[0041] The auxiliary reading text generation module is used to generate a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentences of each sentence in the target text.

[0042] Optionally, the auxiliary reading text generation module includes: a sentence splicing module and a text editing module;

[0043] The sentence splicing module is used to splice the auxiliary reading sentences of each sentence in the target text according to the order of each sentence in the target text to obtain the text to be edited;

[0044] The text editing module is used to edit the text to be edited using a pre-established chapter editing model, and the edited text serves as the chapter-level auxiliary reading text of the target text, wherein the chapter editing model is trained using training text annotated with real editing results.

[0045] A text generation device, comprising: a memory and a processor;

[0046] The memory is used to store programs;

[0047] The processor is used to execute the program to implement each step of the text generation method described in any one of the above items.

[0048] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements each step of any of the above-mentioned text generation methods.

[0049] Through the above scheme, it can be known that the text generation method, device, equipment and storage medium provided by the present application, after obtaining the target text in the specified field, first use the pre-established text generation model to generate auxiliary reading sentences for each sentence in the target text, and then generate the chapter-level auxiliary reading text of the target text based on the auxiliary reading sentences for each sentence in the target text. The text generation method provided by the present application can generate auxiliary reading text for the target text in the specified field, and the generated auxiliary reading text can assist the reader in reading, thereby greatly reducing the difficulty of the reader in reading the target text, and the user experience is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 A flowchart of a text generation method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of the training process of the text generation model provided in the embodiment of the present application;

[0053] Figure 3 A schematic diagram of using a text generation model to generate auxiliary reading sentences provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a text generation device provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a text generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In order to solve the problem that texts in certain fields are difficult for readers to read due to their strong domain specificity, the applicant thought of using a text generation strategy to generate auxiliary reading texts for texts in specified fields to assist readers in reading, thereby reducing readers' reading difficulties. Following this idea, the applicant conducted research and, through continuous research, finally proposed a text generation method with better results. This text generation method can generate high-quality auxiliary reading texts for texts in specified fields.

[0058] The text generation method proposed in this application can be applied to electronic devices with data processing capabilities, which can be servers on the network side (can be a single server, or multiple servers or server clusters), or terminals used on the user side, such as smart phones, PCs, notebooks, PADs, etc. The server or terminal can generate auxiliary reading text for texts in a specified field according to the text generation method provided in this application. Next, the text generation method provided in this application is introduced through the following embodiments.

[0059] First embodiment

[0060] See also Figure 1 , shows a schematic diagram of the process of the text generation method provided in an embodiment of the present application, which may include:

[0061] Step S101: Obtain target text in a specified field.

[0062] Optionally, the designated field may be a medical field, and accordingly, the target text may be a medical text, such as an imaging report, an operation report, an electronic medical record, etc. Of course, this embodiment does not limit the designated field to the medical field, and the designated field in this application may be any highly professional field. The target text in this embodiment is a chapter-level text, which may include one sentence or multiple sentences.

[0063] Step S102: Generate auxiliary reading sentences for each sentence in the target text using the pre-established text generation model.

[0064] Specifically, the target text is firstly processed by sentence segmentation to obtain each sentence in the target text, and then the pre-established text generation model is used to generate auxiliary reading sentences for each sentence in the target text.

[0065] Among them, the auxiliary reading sentence of each sentence in the target text can be an explanatory sentence of each sentence in the target text. For example, a sentence in a medical text is "the posterior cruciate ligament is blurred", and the auxiliary reading sentence of this sentence is "the posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament damage causing the blurred display."

[0066] The text generation model in this embodiment is trained using training sentences annotated with real auxiliary reading sentences in a specified field.

[0067] In one possible implementation, a text generation model for generating auxiliary reading sentences can be obtained by directly using training sentences annotated with real auxiliary reading sentences; in another possible implementation, an initial text generation model for a specified field can be trained based on the pre-trained model, and then the initial text generation model can be trained using training sentences annotated with real auxiliary reading sentences in the specified field to obtain a final text generation model for generating auxiliary reading sentences. Considering that the second implementation can accelerate the convergence speed of model training and improve the performance of the model, the present application preferably adopts the second implementation to establish a text generation model.

[0068] It is understandable that in order to ensure the generation effect of the text generation model, it is necessary to use annotated training data for training. The quality of the training data is the key to whether the model can generate reliable text. In order to ensure the quality of the training data, the applicant thought that the quality of the training data can be ensured by limiting the annotation operation of the annotator. Along this line of thought, the applicant thought that when annotating the training sentences, only the training sentences can be inserted. When inserting, insert explanatory fragments and / or sentence fluency auxiliary fragments, that is, the real auxiliary reading sentences annotated by the above training sentences are obtained by inserting explanatory fragments and / or sentence fluency auxiliary fragments into the training sentences. Exemplarily, the training sentence is "The posterior cruciate ligament is blurred", and the auxiliary reading sentence annotated by the training sentence is "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. Considering that there is ligament damage, the display is blurred", where "considered to be" is a sentence fluency auxiliary fragment, "located inside the joint and outside the synovial capsule" and "there is ligament damage that causes display" are explanatory fragments.

[0069] When training the text generation model with training sentences annotated with real auxiliary reading sentences, the goal of the text generation model is to make the auxiliary reading sentences generated by the text generation model for the training sentences consistent with the expressions of the real auxiliary reading sentences annotated with the training sentences. Preferably, in order to make the text generation model generate auxiliary reading sentences with better quality, the following training goals can also be set: make each word in the training sentence appear in the auxiliary reading sentence generated by the text generation model for the training sentence, and / or make the auxiliary reading sentence generated by the text generation model for the training sentence consistent with the subject of the search text corresponding to the training sentence, wherein the search text corresponding to the training sentence consists of a preset number of text fragments in the search database (the search database includes a number of text fragments in a specified field) that are most relevant to the training sentence.

[0070] Step S103: Generate a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text.

[0071] There are several ways to generate the chapter-level auxiliary reading text of the target text according to the auxiliary reading sentences of each sentence in the target text:

[0072] In a possible implementation, the auxiliary reading sentences of each sentence in the target text may be spliced ​​according to the order of each sentence in the target text, and the text obtained after splicing is used as the chapter-level auxiliary reading text of the target text.

[0073] Considering that when generating the reading sentences of each sentence in the target text, there is no interaction between the sentences in the target text, which may lead to information redundancy and / or incoherence in the text obtained by splicing the auxiliary reading sentences of each sentence in the target text. In order to improve the quality of the chapter-level auxiliary reading text finally generated, this embodiment provides another preferred implementation method:

[0074] Step S1031 , according to the order of each sentence in the target text, the auxiliary reading sentences of each sentence in the target text are spliced ​​to obtain the text to be edited.

[0075] Exemplarily, the target text includes three sentences, namely s1, s2, and s3, among which the auxiliary reading sentence of s1 is "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament damage, resulting in blurred display.", the auxiliary reading sentence of s2 is "The injury needs to be investigated.", and the auxiliary reading sentence of s3 is "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament rupture." Then, according to the order of each sentence in the target text, the auxiliary reading sentences of each sentence in the target text are spliced ​​to obtain: "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament damage, resulting in blurred display. [seq] The injury needs to be investigated. [seq] The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament rupture." The text obtained after splicing is used as the text to be edited.

[0076] Step S1032: edit the text to be edited using the pre-established chapter editing model, and use the edited text as the chapter-level auxiliary reading text of the target text.

[0077] The text to be edited is input into a pre-established chapter editing model for editing, and the text output by the chapter editing model is used as the chapter-level auxiliary reading text of the target text.

[0078] The chapter editing model in this embodiment is trained using training texts in a pre-built training text set, wherein the training texts in the training text set are annotated with real editing results. Optionally, the chapter editor can be, but is not limited to, a 12-layer Transformer structure plus a linear prediction layer.

[0079] It should be noted that the training texts in the training text set are obtained based on the original text set (texts collected from specified fields) and the above-mentioned text generation model. Specifically, for each original text in the original text set, the text generation model is used to generate auxiliary reading sentences for each sentence in the original text. According to the order of the sentences in the original text, the auxiliary reading sentences of each sentence in the original text are spliced, and the spliced ​​text is used as the training text for the chapter editing model.

[0080] Considering that there may be information redundancy and / or incoherence in the text to be edited, when the pre-established paragraph editing model is used to edit the text to be edited, the paragraph editing model is used to delete redundant words and / or incoherent words in the text to be edited. For the above-mentioned text to be edited "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament damage, which leads to blurred display. [seq] The damage needs to be checked. [seq] The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament rupture.", after it is input into the paragraph editing model, the paragraph editing model edits it and outputs "The posterior cruciate ligament is located inside the joint and outside the synovial capsule. It is considered that there is ligament damage, which leads to blurred display. [seq] The damage needs to be checked. [seq] The posterior cruciate ligament is considered to have ligament rupture." It can be seen that "The posterior cruciate ligament is located inside the joint and outside the synovial capsule" in the text to be edited is deleted.

[0081] The text generation method provided in the embodiment of the present application, after obtaining the target text in the specified field, first uses the pre-established text generation model to generate auxiliary reading sentences for each sentence in the target text, and then generates the chapter-level auxiliary reading text of the target text based on the auxiliary reading sentences for each sentence in the target text. The text generation method provided in the embodiment of the present application can generate auxiliary reading text for the target text in the specified field, and the generated auxiliary reading text can assist the reader in reading, thereby greatly reducing the reader's difficulty in reading the target text, and providing a good user experience.

[0082] Second embodiment

[0083] The above embodiment mentioned that "a pre-established text generation model can be used to generate auxiliary reading sentences for each sentence in the target text". The above embodiment also mentioned that the text generation model is trained using training sentences that are labeled with real auxiliary reading sentences. This embodiment focuses on introducing the training process of the text generation model.

[0084] Refer to 2, which shows a flowchart of the training process of the text generation model, which may include:

[0085] Step S201: Acquire training sentences from a training sentence set.

[0086] The training sentence set includes a plurality of training sentences in a specified field, and each training sentence in the training sentence set is annotated with a real auxiliary reading sentence.

[0087] Step S202: Obtain the search text corresponding to the training sentence.

[0088] The search text corresponding to the training sentence is composed of the preset text segments most relevant to the training sentence in the search database. It should be noted that the search text corresponding to the training sentence is obtained in order to introduce external knowledge to better generate auxiliary reading sentences.

[0089] The search database includes multiple text fragments in the specified field. Optionally, the text fragment in the search database can be a natural paragraph of the text content in the specified field separated by line breaks. The text fragment in the search database can be crawled from an encyclopedia website in the specified field. For example, if the specified field is the medical field, text fragments can be crawled from a medical encyclopedia website to construct the search database.

[0090] Specifically, the process of obtaining the retrieval text corresponding to the training sentence may include: retrieving a preset number of text fragments that are most relevant to the training sentence (for example, the 10 text fragments that are most relevant to the training sentence) from the retrieval database, and processing the preset number of text fragments that are most relevant to the training sentence into a long sentence as the retrieval text corresponding to the target sentence.

[0091] The process of processing the preset text segments most relevant to the training sentence into a long sentence may include: deleting the periods of the preset text segments and then splicing them into a long sentence. Optionally, in order to avoid the retrieval text being too long and affecting the calculation efficiency, the stop words in the spliced ​​long sentence may be removed based on a stop word list and truncated according to 50 words.

[0092] Step S203: Generate auxiliary reading sentences for the training sentences by using the text generation model, the training sentences and the search texts corresponding to the training sentences.

[0093] Specifically, the process of generating auxiliary reading sentences for training sentences using a text generation model, training sentences, and retrieval texts corresponding to the training sentences may include: using the text generation model to generate target words for forming auxiliary reading sentences for training sentences from a dictionary, training sentences, and retrieval texts corresponding to the training sentences one by one, so as to obtain auxiliary reading sentences for the training sentences.

[0094] More specifically, the process of using the text generation model to generate target words for forming auxiliary reading sentences of the training sentences from the dictionary, the training sentences, and the search texts corresponding to the training sentences may include:

[0095] Step a1: Use the text generation model to encode the training sentences and the retrieval texts corresponding to the training sentences respectively.

[0096] Optional, such as Figure 3 As shown, the text generation model may include a first encoding module 301, a second encoding module 302 and a decoding module 303. The first encoding module 301 of the text generation model may be used to encode the training sentence to obtain the encoding result of the training sentence, and the second encoding module 302 of the text generation model may be used to encode the search text corresponding to the training sentence to obtain the encoding result of the search text corresponding to the training sentence. Optionally, the first encoding module 301 and the second encoding module 302 may adopt a 12-layer Transformer structure, and the decoding module 303 may adopt a layer of LSTM.

[0097] Step a2: Use the decoding module of the text generation model to perform the following at each decoding moment:

[0098] Step a21, determine the feature vector required for decoding at the current decoding moment as the second vector based on the encoding results of the first vector and the training sentence, and determine the feature vector required for decoding at the current decoding moment as the third vector based on the encoding results of the first vector and the search text corresponding to the training sentence.

[0099] The first vector is a state vector output at the current decoding moment and represents decoded result information.

[0100] The process of determining the feature vector required for decoding at the current decoding moment as the second vector according to the encoding results of the first vector and the training sentence includes: calculating the encoding results of the first vector and the training sentence based on the attention mechanism, and obtaining the context vector corresponding to the training sentence at the current decoding moment as the second vector.

[0101] Similarly, determining the feature vector required for decoding at the current decoding moment as the third vector based on the encoding results of the first vector and the retrieval text corresponding to the training sentence includes: calculating the encoding results of the first vector and the retrieval text corresponding to the training sentence based on the attention mechanism, and obtaining the context vector corresponding to the retrieval text corresponding to the training sentence at the current decoding moment as the third vector.

[0102] Step a22: predict the probability that the target word at the current decoding moment is generated from the dictionary, the training sentence, and the retrieval text corresponding to the training sentence, respectively, based on the first vector, the second vector, and the third vector.

[0103] Specifically, the probability p of the target word at the current decoding time t being generated from the dictionary, the training sentence, and the retrieval text corresponding to the training sentence can be determined according to the following formula: gen ( Figure 3 Gate part of the operation):

[0104] p gen =softmax(W1h t +W2*s t +W3*m t +b ptr ) (1)

[0105] Among them, s t represents the state vector representing the decoded result information output at the current decoding time t, i.e., the first vector mentioned above, h t represents the context vector corresponding to the training sentence at the current decoding time t, that is, the second vector mentioned above, m t Represents the context vector corresponding to the retrieval text corresponding to the training sentence at the current decoding time t, that is, the third vector mentioned above, W1, W2, W3, b ptr Train the parameters for the model.

[0106] It should be noted that p gen The dimension is 3, that is, p gen It includes three probabilities, namely the probability p of the target word generated in the dictionary at the current decoding time t gen，0 , the probability p of the target word at the current decoding time t being generated from the training sentence gen，1 , and the probability p of the target word at the current decoding time t being generated from the retrieval text corresponding to the training sentence gen，2 , where p gen，0 +p gen，1 +p gen，2 =1.

[0107] Step a23, generating the target word at the current decoding moment according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the training sentence and the search text corresponding to the training sentence.

[0108] Specifically, according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the training sentence, and the search text corresponding to the training sentence, the process of generating the target word generated at the current decoding moment may include:

[0109] Step a231: predict the probability of each word in the dictionary being generated based on the first vector and the annotation information before the current decoding moment; predict the probability of each word in the training sentence being generated based on the encoding result of the training sentence and the first vector; predict the probability of each word in the retrieval text corresponding to the training sentence being generated based on the encoding result of the retrieval text corresponding to the training sentence and the first vector.

[0110] In this step, the "annotation information before the current decoding time" refers to the part of the real auxiliary reading sentence annotated by the training sentence before the current decoding time. For example, if the current decoding time is t, then the "annotation information before the current decoding time" in this step refers to the part before the tth word in the real auxiliary reading sentence annotated by the training sentence.

[0111] Among them, according to the encoding result and the first vector of the training sentence, the process of predicting the probability of each word in the training sentence being generated includes: for each word in the training sentence, according to the encoding result and the first vector of the word in the encoding result of the training sentence, determining the attention score of the word, normalizing the attention score of the word, and using the normalized score as the probability of the word being generated. It should be noted that the number of probabilities obtained in the end is the same as the number of words included in the training sentence, regardless of whether there are repeated words in the sentence. For example, the training sentence is expressed as ABCAD (each letter represents a word), and 5 probabilities will be obtained in the end, that is, the probability of the first word A being generated, the probability of word B being generated, the probability of word C being generated, the probability of the second word A being generated, and the probability of word D being generated.

[0112] Specifically, the attention score of the jth word in the training sentence It can be calculated by the following formula:

[0113]

[0114] Among them, h j represents the encoding result of the jth word in the training sentence, W n , W s_1 、b h_atten Train the parameters for the model.

[0115] After obtaining the attention score of each word in the training sentence, it can be normalized as follows:

[0116] a t =softmax(e t ) (3)

[0117] Similarly, the process of predicting the probability of each word being generated in the retrieval text corresponding to the training sentence based on the encoding result and the first vector of the retrieval text corresponding to the training sentence includes: for each word in the retrieval text corresponding to the training sentence, determining the attention score of the word based on the encoding result and the first vector of the word in the encoding result of the retrieval text corresponding to the training sentence, normalizing the attention score of the word, and using the normalized score as the probability of the word being generated.

[0118] Specifically, the attention score of the jth word in the retrieval text corresponding to the training sentence It can be calculated by the following formula:

[0119]

[0120] Among them, h j represents the encoding result of the jth word in the retrieval text corresponding to the training sentence, W m , W s_2 、b m_atten Train the parameters for the model.

[0121] After obtaining the attention score of each word in the retrieval text corresponding to the training sentence, it can be normalized as follows:

[0122] d t =softmax(r t ) (5)

[0123] Step a232, generate the target word at the current decoding moment according to the probability of each word being generated in the dictionary, the probability of each word being generated in the training sentence, the probability of each word being generated in the search text corresponding to the training sentence, and the probability that the target word at the current decoding moment is generated from the dictionary, the training sentence and the search text corresponding to the training sentence respectively.

[0124] Specifically, the implementation process of step a232 may include: determining the probability of each word in the target vocabulary corresponding to the training sentence being generated at the current decoding moment according to the probability of each word in the dictionary being generated, the probability of each word in the training sentence being generated, the probability of each word in the search text corresponding to the training sentence being generated, and the probability that the target word at the current decoding moment is generated from the dictionary, the training sentence, and the search text corresponding to the training sentence, respectively; generating the target word at the current decoding moment according to the probability of each word in the target vocabulary corresponding to the training sentence being generated at the current decoding moment, for example, the word with the highest probability of being generated may be used as the target word at the current decoding moment. It should be noted that the target vocabulary corresponding to the training sentence includes each word in the dictionary, the words in the training sentence that do not appear in the dictionary, and the words in the search text corresponding to the training sentence that do not appear in the dictionary and the training sentence.

[0125] More specifically, the probability of a word w in the target vocabulary corresponding to the training sentence being generated at the current decoding time can be determined according to the following formula:

[0126]

[0127] Among them, P vocab (w) represents the probability of word w in the dictionary being generated, represents the probability and probability of word w in the training sentence being generated (word w may appear multiple times in the training sentence, so here is the probability and), It represents the probability and value of word w being generated in the search text corresponding to the training sentence (word w may appear multiple times in the search text corresponding to the training sentence, so it is the probability and value here).

[0128] It should be noted that if word w only appears in the dictionary but not in the training sentence and the retrieval text corresponding to the training sentence, then and If word w only appears in the training sentence but not in the dictionary or the retrieval text corresponding to the training sentence, then P vocab (w) and are all 0. If word w only appears in the retrieval text corresponding to the training sentence, but not in the dictionary and the training sentence, then P in the above formula vocab (w) and are all 0. If word w appears in the dictionary and training sentences, but does not appear in the retrieval text corresponding to the training sentences, then The same is true for other situations.

[0129] Step S204: Determine the prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence (the "auxiliary reading sentence of the training sentence" mentioned here and later refers to the auxiliary reading sentence generated for the training sentence) and the real auxiliary reading sentence annotated with the training sentence.

[0130] Specifically, according to the probability of each word being generated in the auxiliary reading text of the training sentence (i.e., the above-mentioned P(w)) and the real auxiliary reading text annotated with the training sentence, a prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence and the real auxiliary reading sentence annotated with the training sentence is determined. Optionally, the prediction loss can be a cross-entropy loss.

[0131] In one possible implementation, after the prediction loss is determined in step S204, the parameters of the text generation model can be directly updated according to the prediction loss determined in step S204. In order to improve the model performance, in another possible implementation, the prediction loss determined in step S204 can be used as the first prediction loss loss1, and then the second prediction loss loss2 and the third prediction loss loss3 are determined by the following steps S205 and S206:

[0132] Step S205: Determine a second prediction loss that can reflect the coverage of each word in the training sentence by the auxiliary reading text of the training sentence.

[0133] Specifically, the probability p of each word in the auxiliary reading text of the training sentence being generated in the training sentence can be calculated. gen，1And the generation probability of each word in the training sentence (the generation probability of a word in the training sentence refers to the probability of the word being generated in the training sentence), determine the second prediction loss that can reflect the coverage of the auxiliary reading text of the training sentence for each word in the training sentence.

[0134] In order to enable the text generation model to generate more accurate auxiliary reading text, this application introduces coverage constraints, the purpose of which is to make each word in the training sentence appear in the generated auxiliary reading text as much as possible to ensure that the information of the training sentence is not missed, thereby ensuring the information integrity of the generated auxiliary reading text. In order to make each word in the training sentence be completely included in the generated text, the sum of the probabilities of each word in the training sentence being generated must be greater than 1.

[0135] Specifically, the second prediction loss loss2 can be calculated according to the calculation method shown in the following formula:

[0136] loss2=∑ W c w / |W| (7)

[0137]

[0138] Among them, |W| represents the total number of words contained in the training sentence. It represents the probability and probability of word w in the training sentence being generated before the current decoding time t. The larger the probability, the greater the possibility of the word appearing in the generated text.

[0139] Step S206: Determine a third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the retrieval text corresponding to the training sentence.

[0140] Specifically, the auxiliary reading sentence of the training sentence is taken as a positive sample, and multiple negative samples are obtained, and a third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the retrieval text corresponding to the training sentence is determined according to the distance between the positive sample and the retrieval text corresponding to the training sentence, and the distance between each negative sample and the retrieval text corresponding to the training sentence. The multiple negative samples are multiple auxiliary reading sentences generated before the auxiliary reading sentence of the training sentence.

[0141] Considering that the auxiliary reading sentences generated for the training sentences and the retrieval texts corresponding to the training sentences should have shared information, that is, have a high similarity, to ensure the interpretability of the auxiliary reading texts. In one possible implementation, semantic similarity can be used to constrain the topic consistency between the auxiliary reading sentences generated for the training sentences and the retrieval texts corresponding to the training sentences. However, when the quality of the retrieval texts is poor and there is noise, it is difficult to train by directly maximizing the semantic similarity. For this reason, the present application proposes that a looser constraint can be set by using contrast loss. The constraint considers the relative value of the similarity between the auxiliary reading sentences generated for the training sentences and the retrieval texts corresponding to the training sentences, takes the retrieval text as the anchor sample, and takes the generated auxiliary reading text as the positive sample, so as to shorten the distance between the positive sample and the anchor sample. In view of this, the third prediction loss loss3 can be calculated according to the calculation method shown in the following formula:

[0142]

[0143] Among them, i represents the encoding result of the retrieval text corresponding to the training sentence, i + Represents the encoding result of the positive sample, i - Represents the encoding result of the negative sample, N i Represents the negative sample set. Optionally, the size n of the negative sample set (the number of negative samples in the negative sample set) can be set to 10. τ is a temperature parameter. The larger the value, the sharper the contrast loss distribution. It can be adjusted according to experimental results and can generally be set to 1.

[0144] Optionally, in the process of training the text generation model, a negative sample queue can be maintained, and the auxiliary reading sentences generated during the training process can be added to the queue. When the number of sentences generated exceeds the number of sentences that the queue can accommodate, the first-in-first-out principle can be adopted to make the earliest generated auxiliary reading sentence out of the queue and the latest generated auxiliary reading sentence enter the queue. When it is necessary to calculate the above loss3, n negative samples are sampled from the queue to form the above negative sample set. It should be noted that in the early stage of training, that is, when the number of negative samples in the queue is less than n, loss3 can be calculated first. When the number of negative samples in the queue reaches n, loss3 can be calculated.

[0145] Step S207: updating parameters of the text generation model according to the first prediction loss, the second prediction loss and the third prediction loss.

[0146] In one possible implementation, the first prediction loss, the second prediction loss, and the third prediction loss may be directly summed, and the parameters of the text generation model may be updated according to the summed loss. In another possible implementation, the second prediction loss and the third prediction loss may be weighted, and the first prediction loss, the weighted second prediction loss, and the weighted third prediction loss may be summed, and the parameters of the text generation model may be updated according to the summed loss, that is, the parameters of the text generation model may be updated according to the Loss obtained by the following formula:

[0147] Loss=loss1+βloss2+γloss3 (10)

[0148] Among them, β is the weight corresponding to the second prediction loss, γ is the weight corresponding to the third prediction loss, and both β and γ are model training hyperparameters. It should be noted that loss1 is the main training loss, and loss2 and loss3 are auxiliary training losses. The influence of loss2 and loss3 on training can be controlled by adjusting β and γ.

[0149] In addition, it should be noted that in other methods, the parameters of the text generation model can be updated according to the first prediction loss and the second prediction loss (that is, the third prediction loss may not be determined), or the parameters of the text generation model can be updated according to the first prediction loss and the third prediction loss (that is, the second prediction loss may not be determined).

[0150] The text generation model is iterated multiple times according to the above training method until the training end condition is met. The model obtained after the training is completed is the final text generation model.

[0151] Third embodiment

[0152] The text generation model can be established through the implementation method provided in the second embodiment. After the text generation model is established, the established text generation model can be used to generate auxiliary reading sentences for each sentence in the target text. This embodiment focuses on the implementation process of using the established text generation model to generate auxiliary reading sentences for each sentence in the target text.

[0153] The implementation process of generating the auxiliary reading sentence for each sentence in the target text by using the established text generation model may include: executing, for each target sentence in the target text to be generated with the auxiliary reading sentence:

[0154] Step b1: Retrieve a preset number of text segments that are most relevant to the target sentence from the retrieval database, and process the preset number of text segments that are most relevant to the target sentence into a long sentence as the retrieval text corresponding to the target sentence.

[0155] The specific method of processing the preset text segments most relevant to the target sentence into one long sentence is the same as the specific method of processing the preset text segments most relevant to the training sentence into one long sentence. For details, please refer to the above embodiment, and this embodiment will not be repeated here.

[0156] Step b2: Generate an auxiliary reading sentence for the target sentence by using the text generation model, the target sentence and the retrieval text corresponding to the target sentence.

[0157] Specifically, the process of generating an auxiliary reading sentence for a target sentence using a text generation model, a target sentence, and a search text corresponding to the target sentence may include: using the text generation model to generate target words for constituting the auxiliary reading sentence for the target sentence one by one from a dictionary, the target sentence, and the search text corresponding to the target sentence, so as to obtain the auxiliary reading sentence for the target sentence.

[0158] The process of using the text generation model to generate target words for forming the auxiliary reading sentence of the target sentence from the dictionary, the target sentence and the search text corresponding to the target sentence may include:

[0159] Step c1: Encode the target sentence and the search text corresponding to the target sentence respectively using the text generation model to obtain the encoding result of the target sentence and the encoding result of the search text corresponding to the target sentence.

[0160] Step c2: Use the text generation model to perform the following operations at each decoding moment:

[0161] Step c21, determine the feature vector required for decoding at the current decoding moment as the second vector based on the encoding result of the first vector and the target sentence, and determine the feature vector required for decoding at the current decoding moment as the third vector based on the encoding result of the first vector and the search text corresponding to the target sentence.

[0162] The first vector is a state vector output at the current decoding moment and represents decoded result information.

[0163] Determining the feature vector required for decoding at the current decoding moment as the second vector according to the encoding results of the first vector and the target sentence includes: calculating the encoding results of the first vector and the target sentence based on the attention mechanism, and obtaining the context vector corresponding to the target sentence at the current decoding moment as the second vector.

[0164] Determining the feature vector required for decoding at the current decoding moment as the third vector based on the encoding result of the first vector and the retrieval text corresponding to the target sentence includes: calculating the encoding result of the first vector and the retrieval text corresponding to the target sentence based on the attention mechanism, and obtaining the context vector corresponding to the retrieval text corresponding to the target sentence at the current decoding moment as the third vector.

[0165] Step c22: predict the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence, and the search text corresponding to the target sentence, respectively, based on the first vector, the second vector, and the third vector.

[0166] The specific implementation process of this step is the same as the implementation process of "predicting the probability that the target word at the current decoding moment is generated from the dictionary, training sentences and the retrieval text corresponding to the training sentences respectively based on the first vector, the second vector and the third vector" in the above embodiment. Please refer to the above embodiment for details, and this embodiment will not be repeated here.

[0167] Step c23: Generate the target word at the current decoding moment according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the target sentence, and the search text corresponding to the target sentence.

[0168] Specifically, the target word at the current decoding moment is generated according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the target sentence, and the search text corresponding to the target sentence, including:

[0169] Step c231: predict the probability of each word in the dictionary being generated based on the first vector and the text generated for the target sentence; predict the probability of each word in the target sentence being generated based on the encoding result of the target sentence and the first vector; predict the probability of each word in the search text corresponding to the target sentence being generated based on the encoding result of the search text corresponding to the target sentence and the first vector.

[0170] The implementation method of "predicting the probability of each word in the target sentence being generated based on the encoding result and the first vector of the target sentence" is the same as the implementation method of "predicting the probability of each word in the training sentence being generated based on the encoding result and the first vector of the training sentence" in the above embodiment. The implementation method of "predicting the probability of each word in the retrieval text corresponding to the target sentence being generated based on the encoding result and the first vector of the retrieval text corresponding to the training sentence" is the same as the implementation method of "predicting the probability of each word in the retrieval text corresponding to the training sentence being generated based on the encoding result and the first vector of the retrieval text corresponding to the training sentence" in the above embodiment. For details, please refer to the relevant parts of the above embodiment, which will not be repeated in this embodiment.

[0171] Step c232, generate the target word at the current decoding moment according to the probability of each word being generated in the dictionary, the probability of each word being generated in the target sentence, the probability of each word being generated in the search text corresponding to the target sentence, and the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively.

[0172] Specifically, the probability of each word in the target word list corresponding to the target sentence being generated at the current decoding time can be determined according to the calculation method shown in formula (6) in the previous embodiment, and then the target word at the current decoding time can be determined according to the probability of each word in the target word list corresponding to the target sentence being generated at the current decoding time. For example, the word with the highest probability of being generated can be determined as the target word at the current decoding time. The target word list corresponding to the target sentence includes each word in the dictionary, words in the target sentence that do not appear in the dictionary, and words in the search text corresponding to the target sentence that do not appear in the dictionary and the target sentence.

[0173] The text generation methods provided in the first to third embodiments above have the following advantages:

[0174] First, the present application does not directly generate the chapter-level auxiliary reading text of the target text (i.e., it does not directly process the entire target text), but first processes the sentences in the target text to generate auxiliary reading sentences of the sentences, and then further generates the chapter-level auxiliary reading text based on the auxiliary reading sentences of the sentences. Compared with processing the target text at the chapter level, the sentences have a small amount of information and a low degree of data divergence, so the processing is simpler. At the same time, for the training stage of the text generation model, since the sentences are annotated instead of the chapter-level text, the annotation is simpler. Since the sentences are processed, the training of the model is also simpler. It can be understood that directly generating the chapter-level auxiliary reading text of the target text can easily cause the generated text to deviate from the target text, while first processing the sentences with a small amount of information and a low degree of data divergence can generate auxiliary reading sentences with better quality (i.e., the generated auxiliary reading sentences will not deviate from the original sentences), and a chapter-level auxiliary reading text with better quality can be generated based on the auxiliary reading sentences with better quality. In addition, this application introduces external knowledge, namely, search text, when generating auxiliary reading text for sentences. The introduction of search text can improve the quality of text generation. Furthermore, this application introduces coverage constraints and topic consistency constraints in the training process of the text generation model. The introduction of coverage constraints and topic consistency constraints enables the trained text generation model to generate auxiliary reading sentences with higher accuracy and stronger interpretability.

[0175] Secondly, considering that some scenarios have lower requirements for fluency and higher requirements for accuracy, this application defines the text generation task as an explanatory task. When generating text, only retention and insertion operations are considered. This generation strategy greatly reduces the difficulty of the task.

[0176] Third, this application adopts a domain migration strategy, training an initial text generation model in a specified domain based on the pre-trained model, and using a supervised training method to train the initial text generation model. This strategy can accelerate the convergence speed of model training and improve model performance.

[0177] Fourth embodiment

[0178] The embodiment of the present application also provides a text generation device. The text generation device provided in the embodiment of the present application is described below. The text generation device described below and the text generation method described above can be referenced to each other.

[0179] See also Figure 4 , shows a schematic diagram of the structure of the text generation device provided in an embodiment of the present application, which may include: a text acquisition module 401, an auxiliary reading sentence generation module 402 and an auxiliary reading text generation module 403.

[0180] A text acquisition module 401 is used to acquire a target text in a specified field;

[0181] The auxiliary reading sentence generation module 402 is used to generate an auxiliary reading sentence for each sentence in the target text by using a pre-established text generation model.

[0182] The text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field.

[0183] The auxiliary reading text generating module 403 is used to generate the chapter-level auxiliary reading text of the target text according to the auxiliary reading sentences of each sentence in the target text.

[0184] Optionally, the training objectives of the text generation model include:

[0185] Make the auxiliary reading sentence generated by the text generation model for the training sentence consistent with the expression of the real auxiliary reading sentence of the training sentence;

[0186] as well as,

[0187] Make each word in the training sentence appear in the auxiliary reading sentence generated by the text generation model for the training sentence, and / or make the auxiliary reading sentence generated by the text generation model for the training sentence consistent with the subject of the retrieval text corresponding to the training sentence, wherein the retrieval text corresponding to the training sentence consists of a preset number of text fragments in the retrieval database that are most relevant to the training sentence.

[0188] Optionally, the real auxiliary reading sentences annotated with the training sentences are obtained by inserting explanatory segments and / or sentence fluency auxiliary segments into the training sentences.

[0189] Optionally, when the auxiliary reading sentence generating module 402 generates the auxiliary reading sentence for each sentence in the target text by using the pre-established text generation model, it is specifically used to:

[0190] For each target sentence to be generated as an auxiliary reading sentence in the target text:

[0191] Retrieving a preset number of text segments most relevant to the target sentence from a retrieval database, and processing the preset number of text segments most relevant to the target sentence into a long sentence as a retrieval text corresponding to the target sentence;

[0192] An auxiliary reading sentence of the target sentence is generated by using the text generation model, the target sentence and the search text corresponding to the target sentence.

[0193] Optionally, when the auxiliary reading sentence generating module 402 generates the auxiliary reading sentence of the target sentence by using the text generation model, the target sentence and the search text corresponding to the target sentence, it is specifically used to:

[0194] The text generation model is used to generate target words for forming an auxiliary reading sentence of the target sentence from a dictionary, the target sentence and a search text corresponding to the target sentence, so as to obtain an auxiliary reading sentence of the target sentence.

[0195] Optionally, the text generation model includes: an encoding module and a decoding module.

[0196] The encoding module encodes the target sentence and the search text corresponding to the target sentence;

[0197] At each decoding moment, the decoding module executes:

[0198] Determine the feature vector required for decoding at the current decoding moment as the second vector according to the first vector and the encoding result of the target sentence, and determine the feature vector required for decoding at the current decoding moment as the third vector according to the encoding result of the first vector and the search text corresponding to the target sentence, wherein the first vector is a state vector representing the decoded result information output at the current decoding moment;

[0199] According to the first vector, the second vector and the third vector, predict the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively;

[0200] The target word at the current decoding moment is generated according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence.

[0201] Optionally, the decoding module generates the target word generated at the current decoding moment according to the probability that the target word at the current decoding moment is generated in the dictionary, the target sentence and the search text corresponding to the target sentence, specifically for:

[0202] Predicting the probability of each word in the dictionary being generated based on the first vector and the text generated for the target sentence, predicting the probability of each word in the target sentence being generated based on the encoding result of the target sentence and the first vector, and predicting the probability of each word in the search text corresponding to the target sentence being generated based on the encoding result of the search text corresponding to the target sentence and the first vector;

[0203] The target word at the current decoding moment is generated according to the probability of each word in the dictionary being generated, the probability of each word in the target sentence being generated, the probability of each word in the search text corresponding to the target sentence being generated, and the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively.

[0204] Optionally, the text generation device provided in the embodiment of the present application may further include: a model training module. The model training module is used to:

[0205] Obtaining a search text corresponding to the training sentence, wherein the search text corresponding to the training sentence is composed of a preset number of text segments in a search database that are most relevant to the training sentence;

[0206] Generate an auxiliary reading sentence of the training sentence by using a text generation model, the training sentence and a search text corresponding to the training sentence;

[0207] Determine a first prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence and the real auxiliary reading sentence annotated with the training sentence;

[0208] Determine a second prediction loss that can reflect the coverage of each word in the training sentence by the auxiliary reading text of the training sentence, and / or determine a third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the search text corresponding to the training sentence;

[0209] Based on the determined prediction loss, the parameters of the text generation model are updated.

[0210] Optionally, when determining the first prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence and the real auxiliary reading sentence annotated with the training sentence, the model training module is specifically used to:

[0211] Determining the first prediction loss according to the probability of each word being generated in the auxiliary reading text of the training sentence and the real auxiliary reading text annotated by the training sentence;

[0212] Optionally, when determining the second prediction loss that can reflect the coverage of each word in the training sentence by the auxiliary reading text of the training sentence, the model training module is specifically used to:

[0213] Determine the second prediction loss according to the probability that each word in the auxiliary reading text of the training sentence is generated in the training sentence and the generation probability of each word in the training sentence, wherein the generation probability of a word in the training sentence is the probability that the word is generated in the training sentence;

[0214] Optionally, when determining the third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the search text corresponding to the training sentence, the model training module is specifically used to:

[0215] Taking the auxiliary reading sentence of the training sentence as a positive sample, and obtaining a plurality of negative samples, wherein the plurality of negative samples are a plurality of auxiliary reading sentences generated before the auxiliary reading sentence of the training sentence;

[0216] The third prediction loss is determined according to the distance between the positive sample and the retrieval text corresponding to the training sentence, and the distance between each negative sample and the retrieval text corresponding to the training sentence.

[0217] Optionally, the auxiliary reading text generation module 403 includes: a sentence splicing module and a text editing module;

[0218] The sentence splicing module is used to splice the auxiliary reading sentences of each sentence in the target text according to the order of each sentence in the target text to obtain the text to be edited;

[0219] The text editing module is used to edit the text to be edited using a pre-established chapter editing model, and the edited text serves as the chapter-level auxiliary reading text of the target text, wherein the chapter editing model is trained using training text annotated with real editing results.

[0220] Optionally, when the text editing module uses the pre-established chapter editing model to edit the text to be edited, it is specifically used to:

[0221] The redundant words and / or incoherent words in the text to be edited are deleted using a pre-established paragraph editing model.

[0222] The text generation device provided in the embodiment of the present application, after obtaining the target text in the specified field, first uses the pre-established text generation model to generate auxiliary reading sentences for each sentence in the target text, and then generates the chapter-level auxiliary reading text of the target text based on the auxiliary reading sentences for each sentence in the target text. The text generation device provided in the embodiment of the present application can generate auxiliary reading text for the target text in the specified field, and the generated auxiliary reading text can assist the reader in reading, thereby greatly reducing the difficulty of the reader in reading the target text, and providing a good user experience.

[0223] Fifth embodiment

[0224] The present application also provides a text generation device. Figure 5 , shows a schematic diagram of the structure of the text generation device, the text generation device may include: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one communication bus 504;

[0225] In the embodiment of the present application, the number of the processor 501, the communication interface 502, the memory 503, and the communication bus 504 is at least one, and the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0226] The processor 501 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0227] The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0228] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0229] Get the target text of the specified field;

[0230] Generate an auxiliary reading sentence for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field;

[0231] According to the auxiliary reading sentences of each sentence in the target text, a chapter-level auxiliary reading text of the target text is generated.

[0232] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0233] Sixth embodiment

[0234] The embodiment of the present application further provides a computer-readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0235] Get the target text of the specified field;

[0236] Generate an auxiliary reading sentence for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field;

[0237] According to the auxiliary reading sentences of each sentence in the target text, a chapter-level auxiliary reading text of the target text is generated.

[0238] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0239] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0240] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0241] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text generation method, characterized in that: include: Get the target text of the specified field; Generate an auxiliary reading sentence for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field; Generating a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text; The method of using a pre-established text generation model to generate an auxiliary reading sentence for each sentence in the target text includes: For each target sentence in the target text to be generated as an auxiliary reading sentence: retrieve a preset number of text segments most relevant to the target sentence from a retrieval database, and process the preset number of text segments most relevant to the target sentence into a long sentence as the retrieval text corresponding to the target sentence; generate an auxiliary reading sentence for the target sentence using the text generation model, the target sentence and the retrieval text corresponding to the target sentence; The step of generating a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text includes: According to the order of each sentence in the target text, the auxiliary reading sentences of each sentence in the target text are spliced, and the spliced ​​text is used as the chapter-level auxiliary reading text of the target text.

2. The text generation method according to claim 1, characterized in that: The real auxiliary reading sentences annotated with the training sentences are obtained by inserting explanatory segments and / or sentence fluency auxiliary segments into the training sentences.

3. The text generation method according to claim 1, characterized in that: The step of generating an auxiliary reading sentence of the target sentence by using the text generation model, the target sentence and a search text corresponding to the target sentence includes: The text generation model is used to generate target words for forming an auxiliary reading sentence of the target sentence from a dictionary, the target sentence and a search text corresponding to the target sentence, so as to obtain an auxiliary reading sentence of the target sentence.

4. The text generation method according to claim 3, characterized in that: The step of generating target words for forming the auxiliary reading sentence of the target sentence from the dictionary, the target sentence and the search text corresponding to the target sentence one by one comprises: respectively encoding the target sentence and the search text corresponding to the target sentence; At each decoding moment, a feature vector required for decoding at the current decoding moment is determined as a second vector according to the first vector and the encoding result of the target sentence, and a feature vector required for decoding at the current decoding moment is determined as a third vector according to the encoding result of the first vector and the search text corresponding to the target sentence, wherein the first vector is a state vector representing decoded result information output at the current decoding moment; According to the first vector, the second vector and the third vector, predict the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively; The target word at the current decoding moment is generated according to the probabilities that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence.

5. The text generation method according to claim 4, characterized in that: The generating the target word generated at the current decoding moment according to the probability that the target word at the current decoding moment is generated in the dictionary, the target sentence and the search text corresponding to the target sentence respectively comprises: Predicting the probability of each word in the dictionary being generated based on the first vector and the text generated for the target sentence, predicting the probability of each word in the target sentence being generated based on the encoding result of the target sentence and the first vector, and predicting the probability of each word in the search text corresponding to the target sentence being generated based on the encoding result of the search text corresponding to the target sentence and the first vector; The target word at the current decoding moment is generated according to the probability of each word in the dictionary being generated, the probability of each word in the target sentence being generated, the probability of each word in the search text corresponding to the target sentence being generated, and the probability that the target word at the current decoding moment is generated from the dictionary, the target sentence and the search text corresponding to the target sentence respectively.

6. The text generation method according to claim 1, characterized in that: The training process of the text generation model includes: Obtaining a search text corresponding to the training sentence, wherein the search text corresponding to the training sentence is composed of a preset number of text segments in a search database that are most relevant to the training sentence; Generate an auxiliary reading sentence of the training sentence by using a text generation model, the training sentence and a search text corresponding to the training sentence; Determine a first prediction loss that can reflect the expression consistency between the auxiliary reading sentence of the training sentence and the real auxiliary reading sentence annotated with the training sentence; Determine a second prediction loss that can reflect the coverage of each word in the training sentence by the auxiliary reading text of the training sentence, and / or determine a third prediction loss that can reflect the topic consistency between the auxiliary reading sentence of the training sentence and the search text corresponding to the training sentence; Based on the determined prediction loss, the parameters of the text generation model are updated.

7. The text generation method according to claim 1, characterized in that: The step of generating a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text further includes: The spliced ​​text is edited using a pre-established chapter editing model, and the edited text serves as the chapter-level auxiliary reading text of the target text, wherein the chapter editing model is trained using training text annotated with real editing results.

8. The text generation method according to claim 7, characterized in that: The editing of the spliced ​​text using a pre-established chapter editing model includes: The redundant words and / or incoherent words in the spliced ​​text are deleted using a pre-established paragraph editing model.

9. A text generation device, characterized in that: include: A text acquisition module, an auxiliary reading sentence generation module and an auxiliary reading text generation module; The text acquisition module is used to acquire the target text in a specified field; The auxiliary reading sentence generation module is used to generate auxiliary reading sentences for each sentence in the target text using a pre-established text generation model, wherein the text generation model is trained using training sentences annotated with real auxiliary reading sentences in the specified field; The auxiliary reading text generation module is used to generate a chapter-level auxiliary reading text of the target text according to the auxiliary reading sentences of each sentence in the target text; Wherein, when the auxiliary reading sentence generation module generates the auxiliary reading sentence of each sentence in the target text by using the pre-established text generation model, it is specifically used to: For each target sentence in the target text to be generated as an auxiliary reading sentence: retrieve a preset number of text segments most relevant to the target sentence from a retrieval database, and process the preset number of text segments most relevant to the target sentence into a long sentence as the retrieval text corresponding to the target sentence; generate an auxiliary reading sentence for the target sentence using the text generation model, the target sentence and the retrieval text corresponding to the target sentence; Wherein, when the auxiliary reading text generation module generates the chapter-level auxiliary reading text of the target text according to the auxiliary reading sentence of each sentence in the target text, it is specifically used to: According to the order of each sentence in the target text, the auxiliary reading sentences of each sentence in the target text are spliced, and the spliced ​​text is used as the chapter-level auxiliary reading text of the target text.

10. The text generation device according to claim 9, characterized in that: The auxiliary reading text generation module is also used for: The spliced ​​text is edited using a pre-established chapter editing model, and the edited text serves as the chapter-level auxiliary reading text of the target text, wherein the chapter editing model is trained using training text annotated with real editing results.

11. A text generation device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the text generation method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the text generation method according to any one of claims 1 to 8 is implemented.

Images