Text generation method, device, storage medium and electronic device

Abstract

The embodiment of the present invention discloses a text generation method, device, storage medium and electronic device, wherein the method comprises: obtaining an initial text, generating an associated word graph of the initial text according to the correlation of each word in the initial text; inputting the initial text into a pre-trained text generation model, obtaining the current predicted word output by the decoder loop, determining an associated subgraph of the current predicted word based on the associated word graph, wherein the associated subgraph includes at least one adjacent word of the current predicted word; determining an updated feature of the current predicted word based on the associated subgraph, inputting the updated feature into the decoder, and determining the next predicted word; obtaining a target text corresponding to the initial text based on each predicted word output by the decoder. During the decoder loop decoding process, the updated feature that aggregates the feature information of the adjacent words is input into the decoder, thereby reducing the influence of the error in the output feature information of the previous loop.

Description

Technical Field

[0001] The embodiments of the present invention relate to data processing technology, and in particular to a text generation method, device, storage medium and electronic device. Background Art

[0002] With the continuous improvement of artificial intelligence technology, machines can now replace human work in various scenarios. Specifically, in the copywriting scene, AI's creative ability in news commentary, product marketing copywriting, product reviews and other directions is shining.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art: the existing sequence-to-sequence generation model framework has an exposure bias problem during decoding. Since the input of the decoder at each step in the test phase is the output of the decoder at the previous moment, if an error occurs in the decoding output of the model at a certain moment, the error will continue to propagate backwards, which has a great impact on the quality of the final generated text. Summary of the invention

[0004] Embodiments of the present invention provide a text generation method, device, storage medium and electronic device to reduce the cumulative error in the text generation process.

[0005] In a first aspect, an embodiment of the present invention provides a text generation method, comprising:

[0006] Acquire an initial text, and generate a related word graph of the initial text according to the relevance of each word in the initial text;

[0007] Inputting the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder;

[0008] Obtaining a current predicted word outputted by the decoder loop, and determining a related subgraph of the current predicted word based on the related word graph, wherein the related subgraph includes at least one adjacent word of the current predicted word;

[0009] Determine an updated feature of the current predicted word based on the associated subgraph, input the updated feature to the decoder, and determine a next predicted word;

[0010] A target text corresponding to the initial text is obtained based on each predicted word output by the decoder.

[0011] In a second aspect, an embodiment of the present invention further provides a text generation device, comprising:

[0012] A related word graph acquisition module is used to acquire an initial text and generate a related word graph of the initial text according to the relevance of each word in the initial text;

[0013] An initial text input module, used to input the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder;

[0014] a correlation subgraph determination module, configured to obtain a current prediction word outputted by the decoder loop, and determine a correlation subgraph of the current prediction word based on the correlation word graph, wherein the correlation subgraph includes at least one adjacent word of the current prediction word;

[0015] an update feature determination module, configured to determine an update feature of the current prediction word based on the associated subgraph, input the update feature to the decoder, and determine a next prediction word;

[0016] The target text determination module is used to obtain the target text corresponding to the initial text based on each predicted word output by the decoder.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a text generation method as provided in any embodiment of the present invention is implemented.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a text generation method as provided in any embodiment of the present invention.

[0019] The technical solution of this embodiment is to predetermine the associated word graph of each word in the initial text. During the processing of the initial text by the text generation model, the decoder of the text generation model performs cyclic decoding processing on the encoded features output by the encoder, and outputs a predicted word in each cycle. The predicted word is used to determine adjacent words with an associated relationship in the associated word graph to form an associated subgraph, which includes the associated relationship between the predicted word and the adjacent words. Based on the adjacent words and the corresponding associated relationship, an updated feature of the predicted word is generated, and the updated feature aggregates the feature information of the adjacent words. The updated feature is input to the decoder for the next cycle decoding to obtain the next predicted word, which is jointly determined based on the updated feature and the output feature information of the previous cycle, thereby reducing the influence of the error in the output feature information of the previous cycle and improving the accuracy of the predicted word and the accuracy of the target text. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of a text generation method provided in Embodiment 1 of the present invention;

[0021] Figure 2It is a schematic diagram of a process of creating a related word graph provided by an embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of the processing flow of the text generation model provided by an embodiment of the present invention;

[0023] Figure 4 is a structural schematic diagram of a text generation device provided in Embodiment 2 of the present invention;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0026] Embodiment 1

[0027] Figure 1 This is a flowchart of a text generation method provided in Embodiment 1 of the present invention. This embodiment is applicable to the automatic generation of conditional text. The method can be executed by a text generation device provided in an embodiment of the present invention. The device can be implemented in software and / or hardware. The device can be integrated into electronic devices such as computers, mobile phones, servers, etc. The method specifically includes the following steps:

[0028] S110: Acquire an initial text, and generate a related word graph of the initial text according to the relevance of each word in the initial text.

[0029] S120: inputting the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder.

[0030] S130: Obtain a current predicted word output by the decoder loop, and determine a related subgraph of the current predicted word based on the related word graph, wherein the related subgraph includes at least one adjacent word of the current predicted word.

[0031] S140, determining an updated feature of the current predicted word based on the associated subgraph, inputting the updated feature into the decoder, and determining the next predicted word. Returning to step S130, until the current predicted word is the end word, and then executing step S150.

[0032] S150: Obtain a target text corresponding to the initial text based on each predicted word output by the decoder.

[0033] In this embodiment, the target text is obtained by processing the initial text based on the pre-trained text generation model. In one embodiment, the text generation model is a translation text generation model, the initial text is the text to be translated, the target text is the translation text of the initial text, and the translation text generation model is trained based on the initial sample text and the standard translation text corresponding to the above initial sample text. In some embodiments, the text generation model is a summary text generation model, the initial text is the full text manuscript to be extracted, such as a full news manuscript, a full paper manuscript or other full article manuscripts, etc., the target text is the summary text of the initial text, and accordingly, the summary text generation model is trained based on the initial sample text and the standard summary text corresponding to the above initial sample text. In other embodiments, the text generation model can also be other conditional text generation models, which are used to generate corresponding conditional texts based on the initial text, without limitation.

[0034] In this embodiment, after obtaining the initial text, the initial text is preprocessed to obtain an associated word graph of the initial text. For example, the initial text may be processed by word segmentation to obtain multiple words in the initial text, and low-frequency words are removed from each of the obtained words to reduce the interference of low-frequency words on the text generation process. Further, each filtered word is set with an index identifier, which may be a word sequence number, such as 1, 2...N. A correspondence between each word and the index identifier is created to facilitate the subsequent rapid determination of the corresponding word through the index identifier. In some embodiments, each word is also converted into a word vector to facilitate the processing of each word.

[0035] The correlation between any two words obtained after screening is calculated to obtain the correlation coefficient between any two words, and a correlation word graph is created based on the correlation coefficient, wherein the correlation coefficient represents the closer the correlation relationship between the two words corresponding to the correlation coefficient. The calculation method of the correlation coefficient is not limited in this embodiment. In some embodiments, the point mutual information (PMI) of any two words can be determined as the correlation coefficient. For example, the point mutual information of word x and word y is:

[0036]

[0037] Where p(x,y) is the probability that word x and word y appear as adjacent words in the initial text, and p(x) and p(y) are the probabilities that word x and word y appear in the initial text. The above p(x,y), p(x) and p(y) are determined based on the co-occurrence matrix.

[0038] On the basis of the above-mentioned embodiment, an associated word graph of the initial text is generated according to the relevance of each word in the initial text, including: sliding a preset window in the initial text to determine each word in the preset window; determining the correlation coefficient between each word in the preset window, establishing an association relationship between words whose correlation coefficients meet a preset threshold, and forming an associated word graph.

[0039] The preset window is used to determine the adjacent range of each word. If the distance between any two words meets the adjacent range, it is determined that the above two words are within the preset window. The preset window is slid in the initial text to determine the adjacent range of each word, as well as the words within the adjacent range for correlation calculation. Exemplarily, the window size of the preset window can be set according to user needs, for example, it can be 3 or 5. Taking the window size of 3 as an example, for any word, the 3 words before and after the word are regarded as words within the adjacent range of the word, and the correlation calculation is performed with the any word respectively. By setting the preset window, the scope of the words for correlation calculation is limited, the amount of correlation calculation is reduced, a large number of invalid calculations are reduced, and the processing efficiency of the initial text is improved.

[0040] For any two words in the adjacent range, the PMI value, i.e., the correlation coefficient, of the above two words is determined based on the co-occurrence matrix of the initial text. The correlation coefficient of the above two words is verified according to a preset threshold, and if the correlation coefficient of the above two words is greater than or equal to the preset threshold, an association relationship between the above two words is created, for example, by connecting the above two words with a line, wherein the association relationship carries the correlation coefficient of the above two words.

[0041] Optionally, before calculating the PMI values ​​of any two words, determine whether the association relationship between the above-mentioned any two words is included in the association word graph. If so, cancel the calculation of the PMI values ​​of any two words. If not, calculate the PMI values ​​of any two words.

[0042] For example, see Figure 2 , Figure 2 Schematic diagram of the process of creating a related word graph provided by an embodiment of the present invention. Figure 2 The preset window size is 1, and the preset threshold is 1.5. It should be noted that Figure 2 This is just an example. In other embodiments, the preset window and preset threshold can be set according to the requirements. Figure 2 The correlation coefficient between the word "is" and the word "a" is 4.52, which is greater than the preset threshold value. Accordingly, in the associated word graph, an associated relationship, i.e., a connecting line, is set between the word "is" and the word "a", and so on. Accordingly, two words with an associated relationship are adjacent words to each other.

[0043] In this embodiment, by preprocessing the initial text, a related word graph of the initial text is obtained, which is used to represent the related relationship between each word, so as to facilitate the query of adjacent words with related relationship of any word.

[0044] In this embodiment, during the process of the text generation model processing the initial text, auxiliary information is provided through the associated word graph, and the adjacent words of each word are referred to, so as to reduce the error accumulation in the processing process and improve the quality of the target text.

[0045] The text generation model includes an encoder and a decoder, which is a model formed based on a sequence-to-sequence (Sequence2Sequence) framework. Optionally, the encoder and decoder are recurrent network models, and exemplary models may be recurrent neural networks such as LSTM (Long Short-Term Memory) and GRU. The encoder performs a cyclic process on the initial text to obtain the encoding features of the initial text, and inputs the encoding features into the decoder, which performs a cyclic decoding on the encoding features to obtain the target text corresponding to the initial text.

[0046] For example, see Figure 3 , Figure 3 It is a processing flow diagram of the text generation model provided by an embodiment of the present invention. The decoder decodes the encoded features, outputs the first prediction value t1, determines the associated subgraph of the first prediction value t1 based on the associated word graph, and determines the updated features of the first prediction value t1 based on the associated subgraph of t1, inputs the updated features and the feature information output by the decoder in the previous cycle into the decoder, determines the next predicted word, that is, the second predicted word t2, and so on. When the decoder completes the decoding, the predictions are combined to obtain the target text. In this embodiment, the next predicted word is determined based on the associated subgraph of the previous predicted word and the feature information output by the previous cycle, which replaces the situation in the prior art where the next predicted word is determined only based on the feature information output by the previous cycle, reduces the impact of the error in the feature information output by the previous cycle on the next predicted word, improves the accuracy of the next predicted word, and further improves the accuracy of the target text.

[0047] In any cycle of the decoder, the prediction result output by the decoder may include multiple candidate words and the confidence probability of each candidate word, and the subsequent word with the largest confidence probability may be determined as the current predicted word. In some embodiments, the prediction result output by the decoder may be the index identifier of each word, and the current predicted word corresponding to the index identifier output by the decoder is determined according to the preset correspondence between the word and the index identifier in the initial text.

[0048] Match the current predicted word in the associated word graph, and determine the current node corresponding to the current predicted word in the associated word graph. Optionally, determining the associated subgraph of the current predicted word based on the associated word graph includes: determining the adjacent words of the current predicted word in the associated word graph, and determining the associated subgraph of the current predicted word based on the adjacent words. Specifically, the adjacent words are determined by the associated nodes of the current node, that is, the words corresponding to each associated node are the adjacent words of the current predicted word. In some embodiments, the associated subgraph of the current predicted word can be determined based on at least one adjacent word. Optionally, a subgraph including the current predicted word and at least one adjacent word is intercepted from the associated word graph and determined as an associated subgraph. For example, a subgraph of the current node and each associated node can be intercepted from the associated word graph, or a subgraph with a local associated node can be intercepted from the associated word graph. Optionally, the association relationship between the current predicted word and at least one adjacent word is extracted from the associated word graph, and the associated subgraph is drawn based on the current predicted word, at least one adjacent word, and the extracted association relationship between the current predicted word and each adjacent word.

[0049] The feature information of the current predicted word is updated through the correlation coefficients of the adjacent words of the current predicted word and the current predicted word, that is, the feature information of the adjacent words is aggregated in the updated features, thereby improving the accuracy of the feature information of the current predicted word.

[0050] In some embodiments, determining the associated subgraph of the current predicted word based on the adjacent words includes: sampling the adjacent words to obtain a preset number or a preset ratio of sampled adjacent words; and determining the associated subgraph of the current predicted word based on the sampled adjacent words. The preset number and the preset ratio are set according to user needs, for example, the preset number can be 5, and the preset ratio can be 50%.

[0051] Optionally, random sampling is performed among the neighboring words of the current prediction word to obtain a preset number of sampled neighboring words, wherein if the number of neighboring words of the current prediction word is less than the preset number, all the neighboring words of the current prediction word are sampled as sampled neighboring words. Optionally, random sampling is performed among the neighboring words of the current prediction word, and based on the number of neighboring words, a preset number ratio of sampled neighboring words is sampled, wherein if the product of the preset number ratio and the number of neighboring words is a decimal, the number of sampled neighboring words is rounded up to determine the number.

[0052] Based on the above embodiment, the associated subgraph includes the current node corresponding to the current predicted word, the associated nodes (ie, subgraph nodes) corresponding to each sampled adjacent word, and the associated relationships (ie, node connection lines in the subgraph) carrying the correlation coefficients.

[0053] In this embodiment, by randomly sampling the adjacent words of the current predicted word, for the same predicted word, different sampled adjacent words are determined at each sampling to generate different update features, and further predict different next predicted words, thereby improving the diversity of the target text and avoiding the situation where the target files are all the same.

[0054] On the basis of the above embodiment, determining the update feature of the current predicted word based on the associated subgraph includes: inputting the associated subgraph into a pre-trained graph self-attention model to obtain the update feature of the current predicted word. Exemplarily, the graph self-attention model is a GAT (graph attention network) model, and the graph self-attention model can process the associated subgraph to obtain the graph structure information of the associated subgraph as the update feature of the current predicted word, so as to use the update feature of the current predicted word as the guidance information of the next predicted word.

[0055] In some embodiments, before applying the text generation model and the graph self-attention model, it also includes creating an initial text generation model and an initial graph self-attention model, and synchronously training the initial text generation model and the initial graph self-attention model, adjusting the model parameters, so as to obtain a text generation model with a target text generation function and a graph self-attention model with a graph self-attention determination function. Specifically, the initial sample text is input into the initial text generation model to be trained, and in the loop decoding of the initial sample text by the text generation model, the associated subgraphs of the predicted words output by each loop are self-attention processed based on the initial graph self-attention model to obtain updated features, which are input into the decoder of the above-mentioned initial text generation model to decode the next predicted word and obtain the predicted text. The loss function is determined based on the standard text corresponding to the predicted text and the initial sample text, and the parameters of the initial text generation model and the initial graph self-attention model are synchronously adjusted based on the loss function. The above training process is iteratively executed until the training end condition is met to obtain the final text generation model and graph self-attention model.

[0056] Optionally, determining the updated features of the current predicted word based on the associated subgraph includes: performing nonlinear mapping on the word vectors of each word in the associated subgraph to determine the feature information of each word; determining the attention weight of the current predicted word to each adjacent word based on the feature information of each word, and normalizing the attention weights to obtain the updated features of the current predicted word.

[0057] The word vector of each word in the associated subgraph is h = {h1, h2, ..., h N}, Among them, h1 can be the word vector of the current predicted word, h2…h NIt can be the word vector of the adjacent word. N is the number of nodes, and F is the number of node features. The size of the matrix h is N×F, which represents the features of all nodes in the associated subgraph, and It only represents the characteristics of a certain node, so its size is F×1.

[0058] The nonlinear mapping of the word vectors of each word in the associated subgraph can be performed through the parameter matrix operation and the nonlinear activation function operation to obtain the new representation of each word, that is, the feature information of each word.

[0059] Optionally, determining the attention weight of the current predicted word to each adjacent word based on the feature information of each word may be determining a self-attention coefficient based on the feature information of each word, and determining the attention weight of the current predicted word to each adjacent word based on the self-attention coefficient.

[0060] Exemplarily, for the current predicted word i and the adjacent word j, the self-attention weight of the current predicted word i is determined, which can be Among them, W is the weight matrix of all nodes, a(·) is a preset parameter. Accordingly, the attention weight can be determined based on the following formula: Among them, Ni is the set of adjacent words.

[0061] Optionally, normalizing the attention weight to obtain the current predicted word update feature may be implemented based on the following formula: Among them, h i 'Update features for the current predicted word, σ(·) is the activation function.

[0062] On the basis of the above embodiment, the updated feature is input into the decoder so that the decoder decodes based on the updated feature that aggregates the feature information of adjacent words to obtain the next predicted word, thereby improving the accuracy of the next predicted word. Optionally, the updated feature is input into the decoder to determine the next predicted word, including: merging the updated feature with the feature information output by the decoder in the previous cycle to obtain a merged feature; inputting the merged feature into the decoder to determine the next predicted word. Among them, the feature information output by the decoder in the previous cycle may be the hidden state of the decoder at the current moment. Exemplarily, merging the updated feature with the feature information output by the decoder in the previous cycle may be merging the updated feature with the feature information output by the previous cycle by column to obtain a merged feature. The decoder decodes the merged feature to obtain the next predicted word.

[0063] Determine whether the next predicted word is the end word. If so, generate the target text based on the output predicted words. If not, loop through the above process based on the next predicted word, determine the associated subgraph of the next predicted word and update the features, and return to the decoder until the end word is determined.

[0064] In some embodiments, when the next predicted word is a pre-input end mark, the next predicted word is determined to be an end word. In some embodiments, it is determined whether the next predicted word meets a preset number of cycles, and if so, the next predicted word is determined to be an end word. The preset number of cycles can be determined according to the length of the predicted word of the target text. For example, the length of the predicted word of the target text is 10. When the number of cycles corresponding to the next predicted word is 10, it is determined that the preset number of cycles is met, that is, the next predicted word is an end word.

[0065] The technical solution of this embodiment is to predetermine the associated word graph of each word in the initial text. During the processing of the initial text by the text generation model, the decoder of the text generation model performs cyclic decoding processing on the encoded features output by the encoder, and outputs a predicted word in each cycle. The predicted word is used to determine adjacent words with an associated relationship in the associated word graph to form an associated subgraph, which includes the associated relationship between the predicted word and the adjacent words. Based on the adjacent words and the corresponding associated relationship, an updated feature of the predicted word is generated, and the updated feature aggregates the feature information of the adjacent words. The updated feature is input to the decoder for the next cycle decoding to obtain the next predicted word, which is jointly determined based on the updated feature and the output feature information of the previous cycle, thereby reducing the influence of the error in the output feature information of the previous cycle and improving the accuracy of the predicted word and the accuracy of the target text.

[0066] Embodiment 2

[0067] Figure 4 : is a structural diagram of a text generation device provided by Embodiment 2 of the present invention, the device comprising:

[0068] The associated word graph acquisition module 210 is used to acquire an initial text and generate an associated word graph of the initial text according to the relevance of each word in the initial text;

[0069] An initial text input module 220, used to input the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder;

[0070] A related subgraph determination module 230 is used to obtain the current predicted word output by the decoder loop, and determine a related subgraph of the current predicted word based on the related word graph, wherein the related subgraph includes at least one adjacent word of the current predicted word;

[0071] An update feature determination module 240, configured to determine an update feature of the current prediction word based on the associated subgraph, input the update feature to the decoder, and determine a next prediction word;

[0072] The target text determination module 250 is used to obtain the target text corresponding to the initial text based on each predicted word output by the decoder.

[0073] Based on the above embodiment, the associated word graph acquisition module 210 is used to:

[0074] Based on the sliding of a preset window in the initial text, determining each word in the initial text in the preset window;

[0075] The correlation coefficients between the words in the preset window are determined, and association relationships are established between the words whose correlation coefficients meet a preset threshold to form an association word graph.

[0076] Based on the above embodiment, the associated subgraph determining module 230 includes:

[0077] an adjacent word determination unit, configured to determine adjacent words of the current predicted word in the associated word graph;

[0078] The associated subgraph determining unit is used to determine the associated subgraph of the current predicted word based on the adjacent words.

[0079] Based on the above embodiment, the associated subgraph determining unit is used to:

[0080] Sampling the adjacent words to obtain a preset number or a preset ratio of sampled adjacent words;

[0081] A related subgraph of the current predicted word is determined based on the sampled neighboring words.

[0082] Based on the above embodiment, the update feature determination module 240 is used to:

[0083] The associated subgraph is input into a pre-trained graph self-attention model to obtain updated features of the current predicted word.

[0084] Based on the above embodiment, the update feature determination module 240 is used to:

[0085] Performing nonlinear mapping on the word vectors of each word in the associated subgraph to determine feature information of each word;

[0086] The attention weight of the current predicted word to each adjacent word is determined based on the feature information of each word, and the attention weight is normalized to obtain the updated feature of the current predicted word.

[0087] Based on the above embodiment, the update feature determination module 240 is used to:

[0088] Merging the updated feature with feature information output by the decoder in the previous cycle to obtain a merged feature;

[0089] The combined features are input to the decoder to determine the next predicted word.

[0090] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0091] Embodiment 3

[0092] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Figure 5 A block diagram of an electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The device 12 is typically an electronic device that undertakes the image classification function.

[0093] like Figure 5 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors 16, a storage device 28, and a bus 18 connecting various system components (including the storage device 28 and the processor 16).

[0094] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0095] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0096] The storage device 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The storage device 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0097] A program 36 having a set (at least one) of program modules 26 may be stored, for example, in a storage device 28. Such program modules 26 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment. The program modules 26 generally perform the functions and / or methods of the embodiments described herein.

[0098] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, cameras, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0099] The processor 16 executes various functional applications and data processing by running the programs stored in the storage device 28, such as implementing the text generation method provided in the above embodiment of the present invention.

[0100] Embodiment 4

[0101] Embodiment 4 of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the text generation method provided in the embodiment of the present invention is implemented.

[0102] Of course, the computer-readable storage medium provided by the embodiment of the present invention stores a computer program which is not limited to the method operation described above, and can also execute the text generation method provided by any embodiment of the present invention.

[0103] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0104] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable source code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0105] Source code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0106] Computer source code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The source code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0107] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A text generation method, characterized in that: include: Acquire an initial text, and generate a related word graph of the initial text according to the relevance of each word in the initial text; Inputting the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder; Obtaining a current predicted word outputted by the decoder loop, and determining a related subgraph of the current predicted word based on the related word graph, wherein the related subgraph includes at least one adjacent word of the current predicted word; Determine an updated feature of the current predicted word based on the associated subgraph, input the updated feature to the decoder, and determine a next predicted word; Obtaining a target text corresponding to the initial text based on each predicted word output by the decoder; Wherein, determining the associated subgraph of the current predicted word based on the associated word graph includes: Neighboring words of the current predicted word in the associated word graph are determined, and an associated subgraph of the current predicted word is determined based on the neighboring words.

2. The method according to claim 1, characterized in that The step of generating a related word graph of the initial text according to the relevance of each word in the initial text includes: Based on the sliding of a preset window in the initial text, determining each word in the initial text in the preset window; The correlation coefficients between the words in the preset window are determined, and association relationships are established between the words whose correlation coefficients meet a preset threshold to form an association word graph.

3. The method according to claim 1, characterized in that The determining the associated subgraph of the current predicted word based on the adjacent words comprises: Sampling the adjacent words to obtain a preset number or a preset ratio of sampled adjacent words; A related subgraph of the current predicted word is determined based on the sampled neighboring words.

4. The method according to claim 1, characterized in that: The determining the update feature of the current predicted word based on the associated subgraph includes: The associated subgraph is input into a pre-trained graph self-attention model to obtain updated features of the current predicted word.

5. The method according to claim 1, characterized in that The determining the update feature of the current predicted word based on the associated subgraph includes: Performing nonlinear mapping on the word vectors of each word in the associated subgraph to determine feature information of each word; The attention weight of the current predicted word to each adjacent word is determined based on the feature information of each word, and the attention weight is normalized to obtain the updated feature of the current predicted word.

6. The method according to claim 1, characterized in that The step of inputting the updated features into the decoder to determine the next predicted word comprises: Merging the updated feature with feature information output by the decoder in the previous cycle to obtain a merged feature; The combined features are input to the decoder to determine the next predicted word.

7. A text generation device, characterized in that: include: A related word graph acquisition module is used to acquire an initial text and generate a related word graph of the initial text according to the relevance of each word in the initial text; An initial text input module, used to input the initial text into a pre-trained text generation model, wherein the text generation model includes an encoder and a decoder; a correlation subgraph determination module, configured to obtain a current prediction word outputted by the decoder loop, and determine a correlation subgraph of the current prediction word based on the correlation word graph, wherein the correlation subgraph includes at least one adjacent word of the current prediction word; an update feature determination module, configured to determine an update feature of the current prediction word based on the associated subgraph, input the update feature to the decoder, and determine a next prediction word; A target text determination module, configured to obtain a target text corresponding to the initial text based on each predicted word output by the decoder; Wherein, the associated subgraph determination module includes an adjacent word determination unit and an associated subgraph determination unit; The adjacent word determination unit is used to determine adjacent words of the current predicted word in the associated word graph; The associated subgraph determining unit is configured to determine the associated subgraph of the current predicted word based on the adjacent words.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the text generation method as described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the text generation method as described in any one of claims 1 to 6 is implemented.

Images