Article generation method, device, intelligent device and storage medium

Abstract

The present application is applicable to the field of text processing technology, and provides an article generation method, apparatus, intelligent device and storage medium, including: obtaining a starting text; inputting the starting text into a trained text generation model for processing, and obtaining a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes obtaining the probability distribution of newly generated words corresponding to the starting text, determining the newly generated words according to the probability distribution of the newly generated words, and updating the starting text according to the newly generated words to generate a preliminary text, and obtaining the generated text corresponding to the starting text when the preliminary text meets the preset requirements. The present application can enhance the contextual coherence and logic of the generated article, and can improve the quality of the article while improving the efficiency of article generation.

Description

Technical Field

[0001] The present application relates to the technical field of text processing, and in particular to an article generation method, apparatus, intelligent device and storage medium. Background Art

[0002] With the continuous development of big data technology, natural language processing and other artificial intelligence technologies, the exploration and practice of using algorithms to automatically write and generate articles has gradually been launched. Writing mainly examines word order, grammar, content and language logic. The text generation model obtained by training a large amount of high-quality composition data can automatically generate high-quality compositions.

[0003] However, in the prior art, it takes a long time for the automatic text generation model to generate an article, and the generated article has poor context coherence and logic, and the article quality is poor. Summary of the invention

[0004] The embodiments of the present application provide an article generation method, apparatus, intelligent device and storage medium, which can solve the problems in the prior art that it takes a long time for an automatic text generation model to generate an article, and the generated article has poor contextual coherence and logic, resulting in poor article quality.

[0005] In a first aspect, an embodiment of the present application provides an article generation method, comprising:

[0006] Get the starting text;

[0007] The starting text is input into a trained text generation model for processing to obtain a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes obtaining the probability distribution of newly generated words corresponding to the starting text, determining the newly generated words according to the probability distribution of the newly generated words, and updating the starting text according to the newly generated words to generate a preliminary text, and when the preliminary text meets preset requirements, obtaining the generated text corresponding to the starting text.

[0008] In a possible implementation manner of the first aspect, the step of obtaining a probability distribution of a newly generated word corresponding to the starting text includes:

[0009] Obtaining encoding information of sentences in the starting text;

[0010] The coded information is converted according to a preset algorithm to obtain a conversion result of the sentence in the starting text;

[0011] According to the conversion processing result, the probability distribution of the newly generated words corresponding to the starting text is obtained.

[0012] In a possible implementation manner of the first aspect, the step of obtaining encoding information of sentences in the starting text includes:

[0013] Obtain a word embedding matrix and an index of each word in the sentence in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary;

[0014] Determine an embedding code of the sentence according to the word embedding matrix and the index of each word in the sentence in the dictionary;

[0015] Obtaining a position code of the sentence, wherein the position code is determined according to relative position information of each word in the sentence;

[0016] The encoding information of the sentence is determined according to the embedded coding and the position coding.

[0017] In a possible implementation manner of the first aspect, the step of converting the coded information according to a preset algorithm to obtain a conversion result of the sentence in the starting text includes:

[0018] Performing conversion processing on the encoded information based on an attention mechanism to obtain an initial conversion result;

[0019] Performing residual connection processing on the coded information and the initial conversion processing result to obtain a residual connection result;

[0020] The residual connection result is fully connected through a fully connected layer to obtain a conversion processing result of the sentence in the starting text.

[0021] In a possible implementation manner of the first aspect, the step of obtaining, according to the conversion processing result, a probability distribution of a newly generated word corresponding to the starting text includes:

[0022] The conversion result is processed through a fully connected layer and a softmax function to obtain a probability distribution of the newly generated words corresponding to the starting text.

[0023] In a possible implementation of the first aspect, the step of determining the newly generated word according to the probability distribution of the newly generated word, and updating the starting text according to the newly generated word to generate the preliminary text includes:

[0024] According to the probability distribution of the newly generated word, searching the dictionary for the corresponding newly generated word;

[0025] The newly generated words found are added to the starting text to generate a preliminary text.

[0026] In a possible implementation of the first aspect, the article generation method further includes:

[0027] If the prepared text does not meet the preset requirements, the prepared text is input into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the prepared text;

[0028] updating the prepared text according to the probability distribution of the newly generated words corresponding to the prepared text;

[0029] Determine whether the updated preliminary text meets the preset requirements;

[0030] If not, the step of updating the prepared text is repeated until the updated prepared text meets the preset requirement, thereby obtaining a generated text.

[0031] In a second aspect, an embodiment of the present application provides an article generating device, comprising:

[0032] A starting text acquisition unit, used for acquiring a starting text;

[0033] An article generation unit is used to input the starting text into a trained text generation model for processing to obtain a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes obtaining the probability distribution of newly generated words corresponding to the starting text, determining the newly generated words based on the probability distribution of the newly generated words, and updating the starting text based on the newly generated words to generate a preliminary text, and when the preliminary text meets preset requirements, obtaining the generated text corresponding to the starting text.

[0034] In a third aspect, an embodiment of the present application provides an intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the article generation method as described in the first aspect above is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the article generation method as described in the first aspect above is implemented.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a smart device, the smart device executes the article generation method described in the first aspect above.

[0037] In the embodiment of the present application, based on the parameter matrix decomposition in the input layer and the parameter sharing in the conversion layer, the text generation model of the present application can generate compositions more efficiently. At the same time, based on a large number of high-quality compositions as training data, as well as the embedded coding, position coding of the input layer and the multi-head self-attention mechanism in the conversion layer, the text generation model in the present application can generate more coherent and logical text. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 It is a flowchart of the implementation of the article generation method provided in the embodiment of the present application;

[0040] Figure 2 It is a specific implementation flow chart of the processing of the starting text by the text generation model in the article generation method provided in the embodiment of the present application;

[0041] Figure 3 It is a specific implementation flow chart of step S201 in the article generation method provided in the embodiment of the present application;

[0042] Figure 4 It is the word embedding matrix in the article generation method provided in the embodiment of the present application M Schematic diagram of

[0043] Figure 5 It is a specific implementation flow chart of step S202 in the article generation method provided in the embodiment of the present application;

[0044] Figure 6 is a flowchart of an implementation method of an article generation method provided by another embodiment of the present application;

[0045] Figure 7 It is a structural block diagram of the article generation device provided in the embodiment of the present application;

[0046] Figure 8 It is a schematic diagram of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0049] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0051] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0053] The article generation method provided in the embodiment of the present application can be applied to intelligent terminals such as mobile devices, servers, and ultra-mobile personal computers (UMPCs). The embodiment of the present application does not impose any restrictions on the specific types of intelligent terminals.

[0054] Figure 1 The implementation process of the article generation method provided in the embodiment of the present application is shown, and the method flow includes steps S101 to S102. The specific implementation principle of each step is as follows:

[0055] S101: Obtain the starting text.

[0056] In the embodiment of the present application, the above-mentioned starting text can be a text defined by the user himself, or can be one of the texts selected by the user from multiple template texts.

[0057] In some embodiments, the starting text is a first text consisting of one sentence or multiple sentences, each sentence consisting of words. In some embodiments, the starting text is a second text consisting of multiple words. In other embodiments, the starting text is a third text consisting of at least one sentence and multiple words.

[0058] In the embodiment of the present application, the above-mentioned starting text is any one of the above-mentioned first text, the above-mentioned second text and the above-mentioned third text.

[0059] S102: Input the starting text into a trained text generation model for processing to obtain a generated text corresponding to the starting text.

[0060] The above text generation model mainly consists of three parts, including an input layer, a conversion processing layer and an output layer. Specifically, the above text generation model is used to output a generated text corresponding to the input starting text, that is, automatically generate an article according to the starting text.

[0061] As a possible implementation of this application, Figure 2 The specific implementation flow of the processing of the starting text by the text generation model in the article generation method provided in the embodiment of the present application is shown, and the following steps S201 to S203 are described in detail:

[0062] S201: Obtain the probability distribution of the newly generated words corresponding to the starting text.

[0063] In an embodiment of the present application, the above probability distribution is used to identify the newly generated words in the dictionary corresponding to the above starting text.

[0064] As a possible implementation of this application, Figure 3The specific implementation process of step S201 of the article generation method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0065] A1: Obtain encoding information of sentences in the starting text.

[0066] The above encoding information is obtained by encoding the sentences in the starting text through the input layer of the text generation model. In the embodiment of the present application, the sentences are encoded by word embedding encoding and position encoding to obtain the encoding information of the sentences in the above starting text.

[0067] In some implementations, if the starting text includes multiple words, a word sequence consisting of the multiple words is encoded through word embedding encoding and position encoding to obtain encoding information of the words in the starting text.

[0068] In the embodiment of the present application, the encoding information of the sentence includes embedded encoding and position encoding.

[0069] As a possible implementation of the present application, the above step A1 specifically includes:

[0070] A11: Obtain a word embedding matrix and an index of each word in the sentence in the dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary.

[0071] In the embodiment of the present application, word segmentation is performed based on the text information in the training corpus to obtain a dictionary Vocabulary , the above sentence S includes word. { }, . Initialize a word embedding matrix using Gaussian distribution , , For Dictionary Vocabulary The number of words in is the dimension of the word vector, the word embedding matrix The schematic diagram is as follows Figure 4 A row in the word embedding matrix corresponds to the initial word vector of the word at the corresponding position in the dictionary.

[0072] In the embodiment of the present application, by searching the dictionary Vocabulary, the index of each word in the sentence S in the dictionary Vocabulary can be obtained.

[0073] Word Embedding Matrix It is continuously updated as the text generation model is continuously trained.

[0074] A12: Determine an embedding code of the sentence according to the word embedding matrix and the index of each word in the sentence in the dictionary.

[0075] Specifically, based on the index of each word in the dictionary and the word embedding matrix in the above sentence, the word vector of each word in sentence S can be obtained, and the word vectors of all words in sentence S are concatenated to form a two-dimensional matrix, which represents the embedding code of the sentence. , .

[0076] A13: Obtain a position code of the sentence, where the position code is determined according to relative position information of each word in the sentence.

[0077] In the embodiment of the present application, the embedded code does not contain the position information of each word. By introducing the position code, the position of the word in the sentence can be effectively identified. Specifically, the position code is randomly initialized using a Gaussian distribution. , Positional encoding It is continuously updated as the text generation model is continuously trained.

[0078] A14: Determine encoding information of the sentence according to the embedded encoding and the position encoding.

[0079] In the embodiment of the present application, the encoding information of the sentence is encoded by embedding and position encoding Sure.

[0080] When training a text generation model, the general approach is to maintain the dimension of the word vector The size of the hidden state is equal to that of the next hidden state. However, the word vector learns a context-independent semantic representation of the word, while the hidden state often learns a context-dependent semantic representation of the word. The context information of the word is very important for text generation. In order to make the hidden state have stronger representation capabilities and learn richer context information, the size of the hidden state is often larger. Since the size of the word vector in the word embedding matrix is ​​equal to the size of the hidden state, the number of parameters in the word embedding matrix will be very large, resulting in a reduction in the training and inference speed of the model.

[0081] In the embodiment of the present application, since the word embedding matrix is a matrix with a large number of parameters. , For Dictionary Vocabulary The number of words in is the dimension of the word vector, The value of is generally large, by introducing an intermediate transformation matrix , for the word embedding matrix Row decomposition decouples the dimension of the word vector from the size of the hidden layer, allowing our text generation model to maintain a smaller word embedding matrix and a larger hidden layer with smaller parameters.

[0082] In some embodiments, the word embedding matrix The decomposition of is as follows:

[0083] (1)

[0084] Specifically, Decompose into and , is the decomposed word embedding matrix, is the intermediate conversion matrix, the original word embedding matrix The parameter quantity is , the decomposed word embedding matrix The parameter quantity is , , s is the dimension size of the word vector in the decomposed word embedding matrix, and The parameters of the two matrices are learned as the text generation model is trained.

[0085] As a possible implementation mode of the present application, in the embedding encoding stage of the sentences in the starting text, by querying the sentences Each word in The index in Embedded coding ,Will Embedded coding After adding the position code, it is added to the intermediate matrix M h Multiply them together to get the sentence Final coded information ,Right now:

[0086] , (2)

[0087] Pair Matrix Perform layer normalization. In the The mean of the corresponding positions of the words for:

[0088] (3)

[0089] Standard Deviation for:

[0090] (4)

[0091] In the The normalized value of the corresponding position of each word is:

[0092] (5)

[0093] in and are model parameters, obtained through learning during model training. , , ϵ is a constant. In order to avoid division by 0, a very small value ϵ is usually added to the denominator. The value of ϵ is generally .

[0094] A2: converting the encoded information according to a preset algorithm to obtain a conversion result of the sentence in the starting text.

[0095] In an embodiment of the present application, the above-mentioned encoded information is converted and processed by the conversion processing layer of the above-mentioned text generation model to obtain the conversion processing result of the sentence in the starting text.

[0096] As a possible implementation of the present application, the above step A2 specifically includes:

[0097] A21: converting the encoded information based on the attention mechanism to obtain an initial conversion result;

[0098] The role of the attention mechanism is to combine the information of other words in the sentence and consider it comprehensively when processing a certain word, so as to effectively include its contextual information.

[0099] In an embodiment of the present application, before the training of the above-mentioned text generation model begins, the model parameters are randomly initialized values. When the training of the text generation model begins, the model parameters are initialized with random values. The model parameters are continuously iteratively updated as the model is trained. The model parameters obtained after the training is completed will enable the text generation model to have good representation capabilities.

[0100] Get the first model parameters of a given text generation model, the first model parameters include , , , , , ,in, q, k, y All are logos.

[0101] In some embodiments, there is attention heads, and satisfy , r is the first model parameter mentioned above , , The size of the second dimension of the model parameter matrix. Attention heads are used to encode the sentence S after layer normalization. Multiplying them with the above three model parameter matrices respectively gives the following:

[0102] , , (6)

[0103] in , , , the above Q , K, Y To calculate the intermediate variables.

[0104] Further transformation is done as follows:

[0105] (7)

[0106] , (8)

[0107] in, , , is the number of words in sentence S, A is the intermediate variable for calculation, k , j is the subscript, , For the i The calculation intermediate quantity of the attention head is The calculation results of the attention heads By concatenating the matrix along the second dimension of the model parameter matrix, we can get the initial transformation processing result Z:

[0108] , (9)

[0109] A22: Perform residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result.

[0110] Specifically, according to the following formula (10), and Make residual connection:

[0111] (10)

[0112] Again Perform layer normalization. The specific calculation process is the same as the above layer normalization, and the result of residual connection can be obtained:

[0113] (11)

[0114] , here we use To represent the layer normalization process.

[0115] A23: Perform full connection processing on the residual connection result through a fully connected layer to obtain a conversion processing result of the sentence in the starting text.

[0116] Specifically, in Two fully connected layers are added later. Get the given second model parameters, which include model parameters , ,in , , d1 represents the matrix The size of the second dimension, Specifically, the following transformation is performed to obtain the conversion processing result of the sentence in the starting text:

[0117] (12)

[0118] (13)

[0119] in, , , , Represents the output of the Transformer Block conversion layer at layer 1.

[0120] In the embodiment of the present application, the entire text generation model has g layers of Transformer Blocks. For example, g can be 12, that is, the text generation model has 12 layers of Transformer Blocks. The calculation logic of each layer is the same, and the model parameters are shared between layers. The output of each Transformer Block layer is As the input of the next Transformer Block layer, is the layer index. The output of the last Transformer Block layer is , This is the result of the conversion of the sentence in the starting text, where is the output of the Transformer Block at layer g, .

[0121] A3: According to the conversion processing result, the probability distribution of the newly generated words corresponding to the starting text is obtained.

[0122] As a possible implementation of the present application, the above step A3 specifically includes:

[0123] A31: Process the conversion result through a fully connected layer and a softmax function to obtain a probability distribution of the newly generated words corresponding to the starting text.

[0124] Specifically, obtain the third model parameter , the probability distribution of the newly generated words corresponding to the starting text is determined according to the following formula (14):

[0125] (14)

[0126] in l is the intermediate variable, , Indicates The output of the word after the g-th layer of Transformer Blocks in the transformation processing layer, , , , For Dictionary Vocabulary The number of words in .

[0127] In the embodiment of the present application, the training and reasoning speed of the model can be effectively improved through parameter decomposition of the encoding layer and parameter sharing of the conversion layer.

[0128] S202: Determine the newly generated word according to the probability distribution of the newly generated word, and update the starting text according to the newly generated word to generate a preliminary text.

[0129] As a possible implementation of the present application, Figure 5 As shown, the specific implementation process of step S202 of the article generation method provided in the embodiment of the present application is described in detail as follows:

[0130] B1: According to the probability distribution of the newly generated word, search the dictionary for the corresponding newly generated word by index. Specifically, the probability distribution of the newly generated word refers to the probability of each word in the dictionary being predicted by the model as a newly generated word. According to the probability distribution, the word with the highest probability is determined as the newly generated word.

[0131] Specifically, the index of the newly generated word in the vocabulary for:

[0132] (15)

[0133] in, is the probability distribution of the newly generated word, j is the subscript index.

[0134] B2: Add the newly generated words found to the starting text to generate a preliminary text.

[0135] In one implementation, the newly generated words found according to the above index are added to the end of the starting text to generate the preliminary text.

[0136] S203: When the prepared text meets the preset requirements, a generated text corresponding to the starting text is obtained.

[0137] As a possible implementation of the present application, Figure 6 As shown, the article generation method provided in the embodiment of the present application also includes:

[0138] C1: If the prepared text does not meet the preset requirements, the prepared text is input into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the prepared text.

[0139] C2: updating the prepared text according to the probability distribution of the newly generated words corresponding to the prepared text.

[0140] C3: Determine whether the updated preliminary text meets the preset requirements.

[0141] C4: If not, repeat the step of updating the prepared text until the updated prepared text meets the preset requirement, thereby obtaining a generated text.

[0142] In an embodiment of the present application, the step of updating the prepared text specifically includes inputting the prepared text into the above-mentioned trained text generation model, obtaining the probability distribution of the newly generated words corresponding to the prepared text, determining the newly generated words corresponding to the prepared text according to the probability distribution of the newly generated words corresponding to the prepared text, and updating the prepared text based on the new words corresponding to the prepared text.

[0143] In the embodiment of the present application, the above preset requirement can be a given generated text length. For example, the newly generated word is ,Right now , Indicates that the index in the dictionary Vocabulary is The newly generated word Add words to sentences middle, ,judge Whether the preset requirements are met, for example, Whether the given generated text length is reached, if it does not meet the preset requirements, Input into the text generation model for further calculation To get the first The probability distribution of the newly generated words and generate the A newly generated word, Newly generated words are added to , This cycle continues until If the preset requirements are met, such as reaching a specified length, the generated text is obtained.

[0144] As can be seen from the above, in the embodiment of the present application, based on the parameter matrix decomposition in the input layer and the parameter sharing in the conversion layer, the text generation model of the present application can generate compositions more efficiently. At the same time, based on a large number of high-quality compositions as training data, as well as the embedded coding, position coding of the input layer and the multi-head self-attention mechanism in the conversion layer, the text generation model in the present application can generate more coherent and logical text.

[0145] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0146] Corresponding to the article generation method described in the above embodiment, Figure 7 A structural block diagram of an article generation device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0147] Reference Figure 7 The article generating device comprises: a starting text obtaining unit 71, an article generating unit 72, wherein:

[0148] A starting text acquisition unit 71, used to acquire a starting text;

[0149] The article generation unit 72 is used to input the starting text into a trained text generation model for processing to obtain a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes obtaining the probability distribution of newly generated words corresponding to the starting text, determining the newly generated words based on the probability distribution of the newly generated words, and updating the starting text based on the newly generated words to generate a preliminary text, and when the preliminary text meets the preset requirements, obtaining the generated text corresponding to the starting text.

[0150] As a possible implementation of the present application, the article generating unit 72 includes:

[0151] A coding information acquisition module, used to acquire coding information of sentences in the starting text;

[0152] A conversion processing module, used to perform conversion processing on the coded information according to a preset algorithm to obtain a conversion processing result of the sentence in the starting text;

[0153] The output module is used to obtain the probability distribution of the newly generated words corresponding to the starting text according to the conversion processing result.

[0154] As a possible implementation manner of the present application, the coding information acquisition module is specifically used for:

[0155] Obtain a word embedding matrix and an index of each word in the sentence in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary;

[0156] Determine an embedding code of the sentence according to the word embedding matrix and the index of each word in the sentence in the dictionary;

[0157] Obtaining a position code of the sentence, wherein the position code is determined according to relative position information of each word in the sentence;

[0158] The encoding information of the sentence is determined according to the embedded coding and the position coding.

[0159] As a possible implementation of the present application, the conversion processing module is specifically used for:

[0160] Performing conversion processing on the encoded information based on an attention mechanism to obtain an initial conversion result;

[0161] Performing residual connection processing on the coded information and the initial conversion processing result to obtain a residual connection result;

[0162] The residual connection result is fully connected through a fully connected layer to obtain a conversion processing result of the sentence in the starting text.

[0163] As a possible implementation manner of the present application, the output module is specifically used for:

[0164] The conversion result is processed through a fully connected layer and a softmax function to obtain a probability distribution of the newly generated words corresponding to the starting text.

[0165] As a possible implementation of the present application, the article generating unit 72 further includes:

[0166] A new word indexing module, used to search for the corresponding newly generated word in the dictionary according to the probability distribution of the newly generated word;

[0167] The forecast text generation module is used to add the newly generated words found to the starting text to generate a preliminary text.

[0168] As a possible implementation manner of the present application, the article generation device further includes a text verification and generation unit, which is specifically used to:

[0169] If the prepared text does not meet the preset requirements, the prepared text is input into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the prepared text;

[0170] updating the prepared text according to the probability distribution of the newly generated words corresponding to the prepared text;

[0171] Determine whether the updated preliminary text meets the preset requirements;

[0172] If not, the step of updating the prepared text is repeated until the updated prepared text meets the preset requirement, thereby obtaining a generated text.

[0173] As can be seen from the above, in the embodiment of the present application, based on the parameter matrix decomposition in the input layer and the parameter sharing in the conversion layer, the text generation model of the present application can generate compositions more efficiently. At the same time, based on a large number of high-quality compositions as training data, as well as the embedded coding, position coding of the input layer and the multi-head self-attention mechanism in the conversion layer, the text generation model in the present application can generate more coherent and logical text.

[0174] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0175] The present application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, Figures 1 to 6 Represents the steps of any article generation method.

[0176] The embodiment of the present application also provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, Figures 1 to 6 Represents the steps of any article generation method.

[0177] The present application also provides a computer program product, which, when executed on a server, enables the server to execute the following Figures 1 to 6 Represents the steps of any article generation method.

[0178] Figure 8Schematic diagram of a smart device provided by an embodiment of the present application. Figure 8 As shown, the smart device 8 of this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, the steps in the above-mentioned article generation method embodiments are implemented, for example Figure 1 Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 7 The functions of the units 71 to 72 are shown.

[0179] Exemplarily, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 82 in the smart device 8.

[0180] The smart device 8 may be a mobile device or a server. The smart device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will appreciate that Figure 8 It is only an example of the smart device 8 and does not constitute a limitation of the smart device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smart device 8 may also include input and output devices, network access devices, buses, etc.

[0181] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0182] The memory 81 may be an internal storage unit of the smart device 8, such as a hard disk or memory of the smart device 8. The memory 81 may also be an external storage device of the smart device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart device 8. Further, the memory 81 may also include both an internal storage unit and an external storage device of the smart device 8. The memory 81 is used to store the computer program and other programs and data required by the smart device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0183] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0184] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for generating an article, characterized in that: include: Obtaining a starting text, where the starting text may be a text defined by the user, or may be a text selected by the user from a plurality of template texts; The starting text is input into a trained text generation model for processing to obtain a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes: Obtaining encoding information of sentences in the starting text, including: obtaining a word embedding matrix and an index of each word in the sentence in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary; determining the embedding code of the sentence according to the word embedding matrix and the index of each word in the sentence in the dictionary, wherein an intermediate conversion matrix is ​​introduced to decompose the word embedding matrix, decouple the dimension of the word vector from the size of the hidden layer, and the decomposition of the word embedding matrix is ​​specifically as follows: in, is the word embedding matrix, Decompose into and , is the decomposed word embedding matrix, is the intermediate conversion matrix, the original word embedding matrix The parameter quantity is , the decomposed word embedding matrix The parameter quantity is , , s is the decomposed word embedding matrix The dimension size of the word vector in ; Obtaining a position code of the sentence, wherein the position code is determined according to relative position information of each word in the sentence; determining encoding information of the sentence according to the embedded code and the position code; The encoded information is converted according to a preset algorithm to obtain the conversion processing result of the sentence in the starting text, including: converting the encoded information based on the attention mechanism to obtain an initial conversion result; performing residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result; performing full connection processing on the residual connection result through a full connection layer to obtain the conversion processing result of the sentence in the starting text; wherein the text generation model has a total of g layers of Transformer Block, the calculation logic of each layer is the same, and the model parameters are shared between layers, and the output of each Transformer Block layer is converted into As the input of the next Transformer Block layer, is the layer index, and the output of the last Transformer Block layer is , That is, the conversion processing result of the sentence in the starting text, where is the output of the Transformer Block at layer g; According to the conversion processing result, the probability distribution of the newly generated words corresponding to the starting text is obtained, the newly generated words are determined according to the probability distribution of the newly generated words, and the starting text is updated according to the newly generated words to generate a preliminary text, including: adding the newly generated words to the starting text to generate a preliminary text; when the preliminary text meets the preset requirements, obtaining the generated text corresponding to the starting text; If the prepared text does not meet the preset requirements, the prepared text is input into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the prepared text; According to the probability distribution of the newly generated words corresponding to the prepared text, the prepared text is updated to determine whether the updated prepared text meets the preset requirements; if not, the step of updating the prepared text is repeated until the updated prepared text meets the preset requirements to obtain the generated text, wherein the step of updating the prepared text includes: inputting the prepared text into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the prepared text, determining the newly generated words corresponding to the prepared text according to the probability distribution of the newly generated words corresponding to the prepared text, and updating the prepared text based on the new words corresponding to the prepared text.

2. The article generation method according to claim 1, characterized in that: The step of obtaining the probability distribution of the newly generated words corresponding to the starting text according to the conversion processing result includes: The conversion result is processed through a fully connected layer and a softmax function to obtain a probability distribution of the newly generated words corresponding to the starting text.

3. The article generation method according to claim 1, characterized in that: The step of determining the newly generated word according to the probability distribution of the newly generated word, and updating the starting text according to the newly generated word to generate the preliminary text comprises: According to the probability distribution of the newly generated word, searching the dictionary for the corresponding newly generated word; The newly generated words found are added to the starting text to generate a preliminary text.

4. An article generating device, characterized in that: include: A starting text acquisition unit, used to acquire a starting text, wherein the starting text may be a text defined by the user, or may be a text selected by the user from a plurality of template texts; The article generation unit is used to input the starting text into a trained text generation model for processing to obtain a generated text corresponding to the starting text; wherein the processing of the starting text by the text generation model includes: Obtaining encoding information of sentences in the starting text, including: obtaining a word embedding matrix and an index of each word in the sentence in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary; determining the embedding code of the sentence according to the word embedding matrix and the index of each word in the sentence in the dictionary, wherein an intermediate conversion matrix is ​​introduced to decompose the word embedding matrix, decouple the dimension of the word vector from the size of the hidden layer, and the decomposition of the word embedding matrix is ​​specifically as follows: in, is the word embedding matrix, Decompose into and , is the decomposed word embedding matrix, is the intermediate conversion matrix, the original word embedding matrix The parameter quantity is , the decomposed word embedding matrix The parameter quantity is , , s is the decomposed word embedding matrix The dimension size of the word vector in ; Obtaining a position code of the sentence, wherein the position code is determined according to relative position information of each word in the sentence; determining encoding information of the sentence according to the embedded code and the position code; The encoded information is converted according to a preset algorithm to obtain the conversion processing result of the sentence in the starting text, including: converting the encoded information based on the attention mechanism to obtain an initial conversion result; performing residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result; performing full connection processing on the residual connection result through a full connection layer to obtain the conversion processing result of the sentence in the starting text; wherein the text generation model has a total of g layers of Transformer Block, the calculation logic of each layer is the same, and the model parameters are shared between layers, and the output of each Transformer Block layer is converted into As the input of the next Transformer Block layer, is the layer index, and the output of the last Transformer Block layer is , That is, the conversion processing result of the sentence in the starting text, where is the output of the Transformer Block at layer g; According to the conversion processing result, the probability distribution of the newly generated words corresponding to the starting text is obtained, the newly generated words are determined according to the probability distribution of the newly generated words, and the starting text is updated according to the newly generated words to generate a preliminary text, including: adding the newly generated words to the starting text to generate a preliminary text; when the preliminary text meets the preset requirements, a generated text corresponding to the starting text is obtained; if the preliminary text does not meet the preset requirements, the preliminary text is input into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the preliminary text; according to the probability distribution of the newly generated words corresponding to the preliminary text, the preliminary text is updated to determine whether the updated preliminary text meets the preset requirements; if not, the step of updating the preliminary text is repeated until the updated preliminary text meets the preset requirements to obtain the generated text, wherein the step of updating the preliminary text includes: inputting the preliminary text into the trained text generation model to obtain the probability distribution of the newly generated words corresponding to the preliminary text, determining the newly generated words corresponding to the preliminary text according to the probability distribution of the newly generated words corresponding to the preliminary text, and updating the preliminary text based on the new words corresponding to the preliminary text.

5. An intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the article generation method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the article generation method according to any one of claims 1 to 3 is implemented.

Images