Method, device and computer readable medium for text generation

Abstract

The application discloses a text generation method, device and equipment and a computer readable medium, and relates to the technical field of artificial intelligence. A specific embodiment of the method comprises: inputting an original text into a pointer generation network; in the pointer generation network, updating the distribution probability of a decoded word in the original text according to the weight of the word in the original text; establishing a decoding word table based on the distribution probability of the decoded word, so as to generate an evolved text of the original text in the pointer generation network according to the decoding word table; and the pointer generation network outputs the evolved text of the original text. The embodiment can improve accuracy and reduce repetition for the text generated by words beyond the word table.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and computer-readable medium for text generation. Background Technology

[0002] Text generation technology has seen significant advancements in recent years. Early techniques focused on extraction methods, prioritizing importance scores for each sentence from the original text. Scoring methods primarily relied on frequency and random topic models. The advantage of extraction methods is their ability to more completely preserve original information, particularly ensuring the coherence of each sentence.

[0003] In the process of realizing this invention, the inventors discovered that the prior art has at least the following problems: out-of-vocabulary (OOV) words lead to poor accuracy and high repetition in the generated text. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, device, and computer-readable medium for text generation, which can improve accuracy and reduce repetition for text generated from words that exceed a vocabulary.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for text generation is provided, comprising:

[0006] Generate a network from the original text input pointer;

[0007] In the pointer generation network, the distribution probability of the decoded words in the original text is updated according to the weights of the words in the original text;

[0008] A decoding vocabulary is established based on the probability distribution of the decoded words, and the evolved text of the original text is generated in the pointer generation network according to the decoding vocabulary.

[0009] The pointer generation network outputs the evolved text of the original text.

[0010] The network for generating the original text input pointer includes:

[0011] The filtered original text is input into the pointer to generate a network.

[0012] The step of updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network includes:

[0013] In the pointer generation network, the pointer coefficients in the original text are adjusted according to the weights of the words in the original text to update the distribution probability of the decoded words in the original text.

[0014] The step of updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network includes:

[0015] In the pointer generation network, the weight of each word in the original text is determined based on the number of times each word appears in the original text;

[0016] Update the distribution probability of the decoded words in the original text according to the weights of the words in the original text.

[0017] In the pointer generation network, determining the weight of words in the original text based on the frequency of each word's occurrence includes:

[0018] In the pointer generation network, the weights of words in the original text are determined based on the frequency of each word in the original text and a preset sampling coefficient.

[0019] In the pointer generation network, the weights of words in the original text are determined based on the frequency of each word and a preset sampling coefficient, including:

[0020] Within a preset range, adjust the sampling coefficients and determine the preset sampling coefficients;

[0021] In the pointer generation network, the weights of words in the original text are determined based on the frequency of each word in the original text and a preset sampling coefficient.

[0022] The preset range includes an interval greater than or equal to 0 and less than or equal to 1.

[0023] The step of adjusting the sampling coefficients within a preset range and determining the preset sampling coefficients includes:

[0024] Within a preset range, the sampling coefficients are adjusted in descending order, and the preset sampling coefficients are determined.

[0025] In the pointer generation network, the weights of words in the original text are determined based on the frequency of each word and a preset sampling coefficient, including:

[0026] In the pointer generation network, the weights of words in the original text are determined by smoothing the weights of words in the original text based on the frequency of each word in the original text and a preset sampling coefficient.

[0027] The step of updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text includes:

[0028] Based on the weights of the words in the original text, the distribution probability of commonly used words decoded in the original text is reduced, and the distribution probability of uncommon words decoded in the original text is increased.

[0029] The step of establishing a decoding vocabulary based on the probability distribution of the decoded words, and generating evolved text of the original text in the pointer generation network according to the decoding vocabulary, includes:

[0030] A decoding word list is established based on the probability distribution of the decoded words;

[0031] In the pointer generation network, an evolved text of the original text is generated based on the vocabulary in the decoded vocabulary.

[0032] The decoded vocabulary includes uncommon words and common words in the original text, and the probability distribution of the uncommon words is different from that of the common words.

[0033] The pointer generation network outputs evolved text of the original text, including:

[0034] The pointer generation network outputs the evolved text of the original text in a preset document.

[0035] The evolutionary text includes summary text.

[0036] The evolutionary text includes keyword text.

[0037] According to a second aspect of the present invention, a text generation apparatus is provided, comprising:

[0038] The input module is used to generate a network from the raw text input pointer;

[0039] An update module is used to update the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network.

[0040] A generation module is used to establish a decoding vocabulary based on the probability distribution of the decoded words, so as to generate the evolved text of the original text in the pointer generation network according to the decoding vocabulary;

[0041] The output module is used by the pointer generation network to output the evolved text of the original text.

[0042] According to a third aspect of the present invention, an electronic device for text generation is provided, comprising:

[0043] One or more processors;

[0044] Storage device for storing one or more programs.

[0045] When the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0046] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0047] One embodiment of the above invention has the following advantages or beneficial effects: The original text is input into a pointer generation network; in the pointer generation network, the distribution probability of the decoded words in the original text is updated according to the weights of the words in the original text; a decoding vocabulary is established based on the distribution probability of the decoded words, and an evolved text of the original text is generated in the pointer generation network according to the decoding vocabulary; the pointer generation network outputs the evolved text of the original text. For text generated from words exceeding the vocabulary list, the distribution probability of the words is updated, thereby improving accuracy and reducing repetition.

[0048] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0049] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0050] Figure 1 This is a schematic diagram of the main flow of a text generation method according to an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating the process of updating the distribution probability of decoded words in the original text according to an embodiment of the present invention.

[0052] Figure 3 This is a flowchart illustrating the process of determining the weights of words in the original text according to an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram illustrating the relationship between the frequency of word occurrences and their probability distribution according to an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the process of generating evolved text from original text according to an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of the main structure of a text generation apparatus according to an embodiment of the present invention;

[0056] Figure 7 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0057] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0058] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0059] In recent years, directly generating text using neural networks has proven to be an effective method. With the rapid development of deep learning technology over the years, text generation methods have also continuously evolved. Artificial neural networks have been successfully applied to the field of natural language processing, including but not limited to syntactic analysis, text summarization, and dialogue systems. Among these, the encoder-decoder structure and attention mechanism for generating text are representative methods and have achieved good results.

[0060] Text generation has wide applications in Natural Language Processing (NLP), with mainstream techniques including Seq2Seq and pointer generation network structures to perform general text generation tasks. Later, the Transformer model was developed as a successor to the improved Seq2Seq model.

[0061] However, neither the Transformer model nor the algorithm combining Seq2Seq and pointer generation network proposed a clear metric for the frequency of sample words in the training samples, nor did it provide any practical considerations during the algorithm's decoding stage.

[0062] To address the technical problems of poor accuracy and high repetition in generated text, the following technical solutions from the embodiments of the present invention can be adopted.

[0063] See Figure 1 , Figure 1 This is a schematic diagram illustrating the main flow of a text generation method according to an embodiment of the present invention. A decoding vocabulary is established by updating the distribution probabilities of words, thereby generating an evolved text from the original text. Figure 1 As shown, the specific steps include:

[0064] S101, Generate a network from the original text input pointer.

[0065] In embodiments of the present invention, the text to be processed is referred to as the original text. That is, the original text is text containing multiple characters that requires pointer generation network processing.

[0066] To improve the efficiency of pointer generation networks in processing text, the original text can be filtered first, and then the filtered text can be input into the pointer generation network. This is because non-standard characters in the original text will increase the processing time. Filtering the original text removes non-standard characters, thereby improving the efficiency of text processing.

[0067] Pointer-Generator Networks are generative models based on seq2seq + attention. Compared to seq2seq + attention generative models, they can solve the OOV problem to a certain extent.

[0068] S102. In the pointer generation network, update the distribution probability of the decoded words in the original text according to the weights of the words in the original text.

[0069] In existing pointer generation networks, the final decoding of words in a time series comes not only from the weights captured by the attention mechanism, but also from the weights of each word in the original text, in order to capture some keywords in the original text that cannot be captured in the Attention stage.

[0070] It's important to note that including words in the original text presupposes that all words have the same weight. However, in practice, it's easy to find that the most frequently used words provide less information than the rarer ones. Therefore, it's unreasonable to consider all words with the same weight when constructing the decoding table.

[0071] In this embodiment of the invention, a decoding vocabulary is established by updating the distribution probability of words, and the weight of each word in the decoding vocabulary is not the same.

[0072] The following section, with reference to the accompanying diagram, details the process of updating the probability distribution of decoded words in the original text. The words in the original text need to be decoded, each word has a weight, and the probability distribution of decoded words in the original text is updated based on these weights.

[0073] In one embodiment of the present invention, considering that in the pointer generation network, the pointer coefficients in the original text are related to the weights of the words in the original text, the distribution probability of the decoded words in the original text can be updated based on the pointer coefficients in the original text.

[0074] In one embodiment of the present invention, the pointer generation network includes a Transformer model. The Transformer model is also composed of an encoder and a decoder. The Transformer model is characterized by its ability to generate new words based on a fixed vocabulary, as well as to process new words based on the original text or handle some keywords that cannot be generated by the model algorithm.

[0075] In the actual implementation, P(w) is the probability distribution of the decoded word. Because of the introduction of the pointer generation network, P(w) is modified as follows:

[0076] P(w)=P gen *P vocab (w)+(1-P gen )∑αti Formula 1

[0077] Where ∑αti represents the attention weights captured by the transformers model. gen These are pointer coefficients, which primarily control the probability distribution of the final decoded word. Part of this probability distribution comes from the attention results, and the other part comes from the original text input P. vocab (w).

[0078] Updating the probability distribution of decoded words in the original text involves the weights of words in the original text. The specific process of updating is illustrated below with an example and accompanying figures.

[0079] See Figure 2 , Figure 2 This is a flowchart illustrating the process of updating the probability distribution of decoded words in the original text according to an embodiment of the present invention, specifically including the following steps:

[0080] S201. In the pointer generation network, the weight of each word in the original text is determined based on the number of times each word appears in the original text.

[0081] The original text contains multiple words, and these words may appear more than once. Each word has a corresponding weight. The weight of each word in the original text is... j The sum equals 1, and there are j distinct words in the original text. That is:

[0082] 1 = ∑weight j Formula 2

[0083] The weight of each word in the original text can be calculated using Formula 3.

[0084]

[0085] Wherein, P(word) iP(word) is the sampling rate of the vocabulary. i The weight of a word is related to its frequency of occurrence and a preset sampling coefficient. Therefore, the weight of a word in the original text is determined based on the frequency of occurrence of each word and the preset sampling coefficient.

[0086] See Figure 3 , Figure 3 This is a flowchart illustrating the process of determining the weights of words in an original text according to an embodiment of the present invention, specifically including the following steps:

[0087] S301. Within the preset range, adjust the sampling coefficients and determine the preset sampling coefficients.

[0088] P(word i The value of P(word) is related to the frequency of word occurrence and the preset sampling coefficient. See Formula 4, where Formula 4 is P(word) i The functional relationship between the frequency of word occurrence and the preset sampling coefficient.

[0089]

[0090] Among them, Z (word i ) represents the frequency of word occurrences, and β is the preset sampling coefficient. It should be noted that Formula 4 was obtained through multiple practical experiments.

[0091] According to Formula 4, to determine the weights of words in the original text, two parameters need to be determined: the frequency of word occurrences and the preset sampling coefficient. The frequency of word occurrences can be obtained by counting each word in the original text. The preset sampling coefficient needs to be adjusted.

[0092] In this embodiment of the invention, the adjustment of the sampling coefficient is implemented within a preset range. As an example, the preset range is the interval from 0 to 1. That is, the preset range includes the interval greater than or equal to 0 and less than or equal to 1. In other words, the maximum value of the sampling coefficient is 1, and the minimum value of the sampling coefficient is 0.

[0093] See Figure 4 , Figure 4 This is a schematic diagram illustrating the relationship between the frequency of word occurrences and the probability distribution according to an embodiment of the present invention. Figure 4 The horizontal axis represents the frequency of word occurrences, and the vertical axis represents the probability distribution. Figure 4 The data includes four curves, corresponding to sampling coefficients β = 0.1, 0.01, 0.001, and 0.0001.

[0094] from Figure 4As can be seen, the smoothing level is controlled by changing β. For example, when β = 0.001 and the frequency of a word is less than or equal to 0.00089, the probability distribution is 1.0. This means that rarely sampled words will be sampled with a high probability, such as names or addresses. In the original text, the word frequency is only 0.00089, but after smoothing, the probability of it being sampled is close to 1.

[0095] As β decreases, the probability of words being sampled drops significantly. Therefore, when calculating the original text input in the pointer generation network stage, the input matrix of the original samples becomes sparse, leading to underfitting. Conversely, as β increases, the probability of words being sampled increases significantly, and the smoothing effect of sampling disappears.

[0096] Considering that the sampling coefficients need to be adjusted sequentially during the determination process, to quickly determine the sampling coefficients, they can be adjusted in descending order to establish the preset sampling coefficients. As an example, multiple trials will be conducted, and with the preset sampling coefficient set to 0.001, the sampling rate meets the requirements for text generation.

[0097] By controlling the sampling coefficient β, the smoothness of sampling can be controlled, adapting to data distributions in various scenarios and increasing the universality of pointer generation networks in the text generation process.

[0098] S302. In the pointer generation network, based on the occurrence frequency of each word in the original text and the preset sampling coefficient, the weights of the words in the original text are smoothed and then the weights of the words in the original text are determined.

[0099] In pointer generation networks, the weights of words in the original text can be smoothed by using the frequency of each word and a preset sampling coefficient. Specifically, to smooth out the weights of useless high-frequency words, the preset sampling coefficient is adjusted.

[0100] After determining the preset sampling coefficients to smooth the weights of words in the original text, the weights of words in the original text are determined according to Formulas 2, 3 and 4.

[0101]

[0102] From Formula 5, it can be seen that the original text input P can be obtained by multiplying the matrix formed by each word w in the original text with the weight of each word in the original text. vocab (w).

[0103] Further smooth the probability distribution of the vocabulary in the original text to make the probability matrix of the original text input more conform to the information structure of the original text itself, reduce the weights of some weakly informative vocabulary, increase the weights of some vocabulary containing important information, make the word probability distribution obtained in the decoding stage more reasonable, and at the same time enhance the interpretability of the word distribution.

[0104] In Figure 3 In the embodiment, by adjusting the sampling coefficient, smooth the weights of useless high-frequency words, and then determine the weights of the vocabulary in the original text.

[0105] S202. Update the distribution probability of the decoded words in the original text according to the weights of the vocabulary in the original text.

[0106] After determining the weights of the vocabulary in the original text, the distribution probability of the decoded words in the original text can be updated based on Formula 1 according to the weights of the vocabulary in the original text.

[0107] In an embodiment of the present invention, adjust the distribution probability of the vocabulary according to the weights of the vocabulary in the original text. Specifically, according to the weights of the vocabulary in the original text, reduce the distribution probability of the common words decoded in the original text and increase the distribution probability of the uncommon words decoded in the original text.

[0108] In Figure 2 In the embodiment, by adjusting the weights of the vocabulary, update the distribution probability of the vocabulary, and then increase the probability of the uncommon words being selected.

[0109] S103. Establish a decoding vocabulary based on the distribution probability of the decoded words to generate an evolved text of the original text in the pointer generation network according to the decoding vocabulary.

[0110] For the construction of the decoding vocabulary, traditional methods will select the vocabulary based on scalars such as word frequency and tfidf. For example, there are N words in the corpus. Considering training efficiency and avoiding overfitting, generally K, K < N words will be selected to construct the decoding vocabulary. However, if only select vocabulary by word frequency, then some important vocabulary will never be included in the decoding vocabulary due to too low word frequency.

[0111] See Figure 5 , Figure 5 is a schematic flowchart of generating an evolved text of the original text according to an embodiment of the present invention, specifically including:

[0112] S501. Establish a decoding vocabulary based on the distribution probability of the decoded words.

[0113] After determining the probability distribution of the decoded words, words with a probability distribution greater than a preset probability threshold are stored in the decoding vocabulary. The probability distribution of the decoded words is an adjusted probability, which reduces the probability distribution of commonly used words in the original text and increases the probability distribution of rarely used words in the original text. Thus, the words in the decoding vocabulary can balance the coherence of the evolved text output with the importance of low-frequency words, i.e., rarely used words.

[0114] Understandably, the decoding vocabulary includes both uncommon and common words from the original text, and the probability distribution of uncommon words may differ from that of common words.

[0115] S502. In the pointer generation network, an evolved text of the original text is generated based on the words in the decoding vocabulary.

[0116] In a pointer generation network, an evolved text is generated from the original text based on the vocabulary in the decoded vocabulary. The evolved text is the text processed and output by the pointer generation network. As an example, the evolved text includes summary text and / or keyword text.

[0117] exist Figure 5 In this embodiment, an evolved text of the original text is generated based on the decoded vocabulary. Since the probability distribution of uncommon words in the decoded vocabulary is increased compared to the original probability, the probability of uncommon words appearing in the evolved text is increased.

[0118] Compared to the word distribution and word sampling rate calculated in Word2Vec, the method used... Figure 5 The technical solution used is more consistent with the true probability distribution of words.

[0119] S104, Pointer generation network outputs evolved text of the original text.

[0120] To improve the speed of acquiring evolutionary text, the pointer generation network outputs the evolutionary text of the original text from a preset document. This eliminates the need for searching and allows direct acquisition of the original text from the preset document.

[0121] In the above embodiments, the original text is input into a pointer generation network; in the pointer generation network, the probability distribution of decoded words in the original text is updated according to the weights of the words in the original text; a decoding vocabulary is established based on the probability distribution of the decoded words, and an evolved text of the original text is generated in the pointer generation network according to the decoding vocabulary; the pointer generation network outputs the evolved text of the original text. For text generated from words outside the vocabulary, the probability distribution of those words is updated, thereby improving accuracy and reducing repetition.

[0122] See Figure 6 , Figure 6This is a schematic diagram of the main structure of a text generation apparatus according to an embodiment of the present invention. The text generation apparatus can implement a text generation method, such as... Figure 7 As shown, the text generation device specifically includes:

[0123] Input module 601 is used to generate a network from the raw text input pointer;

[0124] The update module 602 is used to update the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network;

[0125] The generation module 603 is used to establish a decoding vocabulary based on the distribution probability of the decoded words, so as to generate the evolved text of the original text in the pointer generation network according to the decoding vocabulary;

[0126] Output module 604 is used for the pointer generation network to output the evolved text of the original text.

[0127] In one embodiment of the present invention, the input module 601 is specifically used to input the filtered original text into the pointer generation network.

[0128] In one embodiment of the present invention, the update module 602 is specifically used to adjust the pointer coefficients in the original text according to the weights of the words in the original text in the pointer generation network, so as to update the distribution probability of the decoded words in the original text.

[0129] In one embodiment of the present invention, the update module 602 is specifically used to determine the weight of words in the original text based on the number of times each word appears in the original text in the pointer generation network;

[0130] Update the distribution probability of the decoded words in the original text according to the weights of the words in the original text.

[0131] In one embodiment of the present invention, the update module 602 is specifically used to determine the weight of words in the original text based on the occurrence frequency of each word in the original text and a preset sampling coefficient in the pointer generation network.

[0132] In one embodiment of the present invention, the update module 602 is specifically used to adjust the sampling coefficients within a preset range and determine the preset sampling coefficients;

[0133] In the pointer generation network, the weights of words in the original text are determined based on the frequency of each word in the original text and a preset sampling coefficient.

[0134] In one embodiment of the present invention, the preset range includes an interval greater than or equal to 0 and less than or equal to 1.

[0135] In one embodiment of the present invention, the update module 602 is specifically used to adjust the sampling coefficients in descending order within a preset range, and to determine the preset sampling coefficients.

[0136] In one embodiment of the present invention, the preset sampling coefficient is equal to 0.001.

[0137] In one embodiment of the present invention, the update module 602 is specifically used to determine the weight of the words in the original text after smoothing the weight of the words in the original text based on the occurrence frequency of each word in the original text and a preset sampling coefficient in the pointer generation network.

[0138] In one embodiment of the present invention, the updating module 602 is specifically used to reduce the distribution probability of commonly used words decoded in the original text and increase the distribution probability of uncommon words decoded in the original text according to the weight of the words in the original text.

[0139] In one embodiment of the present invention, the pointer generation network includes a Transformer model.

[0140] In one embodiment of the present invention, the generation module 603 is specifically used to establish a decoding word list based on the distribution probability of the decoded words;

[0141] In the pointer generation network, an evolved text of the original text is generated based on the vocabulary in the decoded vocabulary.

[0142] In one embodiment of the present invention, the decoding vocabulary includes uncommon words and common words in the original text, wherein the probability distribution of the uncommon words is different from that of the common words.

[0143] In one embodiment of the present invention, the output module 604 is specifically used for the pointer generation network to output the evolved text of the original text in a preset document.

[0144] In one embodiment of the invention, the evolutionary text includes summary text.

[0145] In one embodiment of the present invention, the evolutionary text includes keyword text.

[0146] Figure 7 An exemplary system architecture 700 is shown for a text generation method or a text generation apparatus to which embodiments of the present invention can be applied.

[0147] like Figure 7As shown, system architecture 700 may include terminal devices 701, 702, and 703, a network 704, and a server 705. Network 704 serves as the medium for providing communication links between terminal devices 701, 702, and 703 and server 705. Network 704 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0148] Users can use terminal devices 701, 702, and 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0149] Terminal devices 701, 702, and 703 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0150] Server 705 can be a server providing various services, such as a backend management server supporting shopping websites browsed by users using terminal devices 701, 702, and 703 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal devices.

[0151] It should be noted that the text generation method provided in the embodiments of the present invention is generally executed by server 705, and correspondingly, the text generation device is generally set in server 705.

[0152] It should be understood that Figure 7 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0153] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing a terminal device of the present invention. Figure 8 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0154] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0155] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0156] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this invention.

[0157] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0159] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an input module, an update module, a generation module, and an output module. The names of these modules do not necessarily limit the module itself; for example, an input module can also be described as "a network for generating raw text input pointers."

[0160] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:

[0161] Generate a network from the original text input pointer;

[0162] In the pointer generation network, the distribution probability of the decoded words in the original text is updated according to the weights of the words in the original text;

[0163] A decoding vocabulary is established based on the probability distribution of the decoded words, and the evolved text of the original text is generated in the pointer generation network according to the decoding vocabulary.

[0164] The pointer generation network outputs the evolved text of the original text.

[0165] According to the technical solution of this invention, the original text is input into a pointer generation network; in the pointer generation network, the distribution probability of the decoded words in the original text is updated according to the weights of the words in the original text; a decoding vocabulary is established based on the distribution probability of the decoded words, and an evolved text of the original text is generated in the pointer generation network according to the decoding vocabulary; the pointer generation network outputs the evolved text of the original text. For text generated from words exceeding the vocabulary, the distribution probability of the words is updated, thereby improving accuracy and reducing repetition.

[0166] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of text generation, characterized by, The method comprises the following steps: inputting an original text into a pointer generation network; adjusting a sampling coefficient in a preset range in a descending order, and determining a preset sampling coefficient; in the pointer generation network, determining a weight of a word in the original text based on a frequency of the word in the original text and the preset sampling coefficient; updating a distribution probability of a decoded word in the original text according to the weight of the word in the original text; establishing a decoding word table based on the distribution probability of the decoded word, and generating an evolved text of the original text in the pointer generation network according to the decoding word table; and the pointer generation network outputs the evolved text of the original text.

2. The method of text generation of claim 1, wherein, The method comprises the following steps: inputting the original text into the pointer generation network.

3. The method of text generation of claim 1, wherein, The method comprises the following steps: in the pointer generation network, adjusting a pointer coefficient in the original text according to the weight of the word in the original text, so as to update the distribution probability of the decoded word in the original text.

4. The method of text generation of claim 1, wherein, The preset range comprises an interval greater than or equal to 0 and less than or equal to 1.

5. The method of text generation of claim 1, wherein, The method comprises the following steps: in the pointer generation network, smoothing the weight of the word in the original text based on the frequency of the word in the original text and the preset sampling coefficient, and then determining the weight of the word in the original text.

6. The method of text generation of claim 1, wherein, The method comprises the following steps: according to the weight of the word in the original text, reducing a distribution probability of a commonly used word in the original text and increasing a distribution probability of a rarely used word in the original text.

7. The method of text generation of claim 1, wherein, The method comprises the following steps: establishing a decoding word table based on the distribution probability of the decoded word; and in the pointer generation network, generating the evolved text of the original text according to the word in the decoding word table.

8. The method of text generation of claim 1, wherein, The decoding word table comprises the rarely used word in the original text and the commonly used word in the original text, and the distribution probability of the rarely used word is different from the distribution probability of the commonly used word.

9. The method of text generation of claim 1, wherein, The pointer generation network outputs the evolved text of the original text, which comprises the following steps: the pointer generation network outputs the evolved text of the original text in a preset document.

10. The method of text generation of claim 1, wherein, The evolved text comprises an abstract text.

11. The method of text generation of claim 1, wherein, The evolved text comprises a keyword text.

12. An apparatus for text generation, the apparatus comprising: The method comprises the following steps: an input module is configured to input an original text into a pointer generation network; an updating module is configured to adjust a sampling coefficient in a preset range in a descending order, and determine a preset sampling coefficient; in the pointer generation network, a weight of a word in the original text is determined based on a frequency of the word in the original text and the preset sampling coefficient; updating a distribution probability of a decoded word in the original text according to a weight of the word in the original text; generating a decoding word table based on the distribution probability of the decoded word, and generating an evolved text of the original text in the pointer generator network according to the decoding word table; outputting the evolved text of the original text by the pointer generator network.

13. An electronic device for text generation, the device comprising: comprising: one or more processors; a memory device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-11.

14. A computer readable medium having stored thereon a computer program, characterized in that, the program is executed by the processor to implement the method according to any one of claims 1-11.

Images