Text generation method and device

Abstract

The present invention discloses a text generation method and device, which relate to the field of computer technology. A specific implementation of the method includes: generating a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, generating a control signal corresponding to the article sample based on the metadata of the article sample, generating a training sample based on the keyword sequence, label sequence, and control signal of the corresponding article sample, and training a controllable text generation model; generating a target control signal based on the target metadata, and generating a text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the target keyword sequence. This implementation can solve the problem that the length and novelty of the generated text are somewhat random, so that the generated text meets the actual online requirements, and can generate texts of different styles, different lengths, and different article types based on the content output requirements of different scenarios according to the input keyword group.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text generation method and device. Background Art

[0002] Content marketing achieves marketing goals by delivering valuable information to users through data such as text and images. Different scenarios have different requirements for text content, such as novelty and length. For example, mobile clients have strict requirements on the number of words in the content. If the number of words is exceeded, it will affect the aesthetics of the UI (user interface) design and the user experience. Current text generation technologies mainly include generation methods without control mechanisms and simple text continuation. The length and novelty of the text generated by the generation methods without control mechanisms are somewhat random, which cannot meet the actual online requirements and will be filtered out by subsequent modules. Simple text continuation cannot generate texts of different styles, lengths, and article types.

[0003] In the process of implementing the present invention, the inventors discovered that the prior art has at least the following problems:

[0004] The length and novelty of the generated text are somewhat random, which cannot meet the actual online needs, and it is impossible to generate texts of different styles, lengths, and article types based on the content output requirements of different scenarios. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a text generation method and device, which can solve the problem that the length and novelty of the generated text are somewhat random, so that the generated text can fully meet the actual online needs, and can generate texts of different styles, lengths, and article types based on the input keyword groups and the requirements of content output in different scenarios.

[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, a text generation method is provided.

[0007] A text generation method comprises: obtaining the content and metadata of each article sample in a collected original article corpus; generating a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, generating a control signal corresponding to the article sample according to a preset rule based on the metadata of the article sample, generating a training sample based on the keyword sequence, label sequence, and control signal corresponding to the article sample, and training a controllable text generation model; generating a target control signal according to the preset rule based on input target metadata, and generating a text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the input target keyword sequence.

[0008] Optionally, the step of generating a keyword sequence corresponding to the article sample based on the content of the article sample includes: using a first word segmentation algorithm to segment the content of the article sample to obtain a word segmentation sequence; extracting word segmentations belonging to preset parts of speech from the word segmentation sequence, and based on the extracted word segmentations, generating a candidate keyword sequence in accordance with the original order of the extracted word segmentations in the word segmentation sequence; deleting word segmentations in the candidate keyword sequence that do not meet the preset conditions to obtain the keyword sequence; the step of generating a label sequence corresponding to the article sample based on the content of the article sample includes: using a second word segmentation algorithm to segment the content of the article sample to obtain the label sequence.

[0009] Optionally, deleting the segmentations in the candidate keyword sequence that do not meet the preset conditions includes: counting the word frequency of each segmentation in the candidate keyword sequence based on the original article corpus, deleting the segmentations in the candidate keyword sequence whose word frequency is less than a preset threshold, and the segmentations in a preset blacklist.

[0010] Optionally, based on the first metadata, a first control signal is generated according to the following preset rules: when the first metadata is in numerical form, its numerical value is used as the first control signal; when the first metadata is in non-numerical form, it is converted into a discrete numerical value within a finite numerical interval according to a conversion rule to serve as the first control signal; wherein, the first metadata is the metadata of the article sample and the first control signal is the control signal corresponding to the article sample, or, the first metadata is the target metadata and the first control signal is the target control signal.

[0011] Optionally, the metadata of the article sample and the input target metadata are one or more of author, article category, article type, and article length, and the metadata of the article sample and the input target metadata belong to the same one or more types of metadata.

[0012] Optionally, the method generates text corresponding to the target keyword sequence based on the target control signal and the input target keyword sequence using the trained controllable text generation model, including: adding the first word vector and the corresponding first position vector of each word in the target keyword sequence, encoding through the encoder of the controllable text generation model to obtain an encoding vector, wherein the first word vector is obtained by embedding the words of the target keyword sequence, and the first position vector is obtained by position encoding the position information of the words of the target keyword sequence; generating a text corresponding to the target keyword sequence by calculating each step in the decoder of the controllable text generation model. The probability distribution of vocabulary is calculated, and the vocabulary sequence with the largest probability is selected as the text corresponding to the target keyword sequence, wherein the target sequence is obtained according to the specific marker word and the currently calculated vocabulary, and the probability distribution of the vocabulary generated in the next step is calculated based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, the target control signal vector, and the encoding vector, wherein the second word vector is obtained by word embedding processing of the words in the target sequence, the second position vector is obtained by position encoding the position information of the words in the target sequence, and the target control signal vector is obtained by word embedding processing of the target control signal.

[0013] Optionally, the target metadata is the multiple types of metadata, and the target control signal is the corresponding multiple types of control signals; the target control signal vector is obtained by performing word embedding processing on the target control signal, including: performing word embedding processing on each type of control signal of the target control signal to obtain a control signal vector corresponding to each type of control signal, and splicing the obtained control signal vectors of each type in a preset order to obtain the target control signal vector.

[0014] According to another aspect of an embodiment of the present invention, a text generating device is provided.

[0015] A text generation device comprises: a data acquisition module for acquiring the content and metadata of each article sample in a collected original article corpus; a model training module for generating a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, and generating a control signal corresponding to the article sample according to preset rules based on the metadata of the article sample, generating a training sample based on the keyword sequence, label sequence, and control signal corresponding to the article sample, and training a controllable text generation model; a text generation module for generating a target control signal according to the preset rules based on input target metadata, and generating a text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the input target keyword sequence.

[0016] Optionally, the model training module includes a keyword sequence generation submodule, which is used to: use a first word segmentation algorithm to segment the content of the article sample to obtain a word segmentation sequence; extract word segmentations belonging to preset parts of speech from the word segmentation sequence, and based on the extracted word segmentations, generate a candidate keyword sequence in accordance with the original order of the extracted word segmentations in the word segmentation sequence; delete the word segmentations in the candidate keyword sequence that do not meet the preset conditions to obtain the keyword sequence; the model training module 502 can also include a label sequence generation submodule, which is used to: use a second word segmentation algorithm to segment the content of the article sample to obtain the label sequence.

[0017] Optionally, the keyword sequence generation submodule includes a word segmentation filtering subunit, which is used to: count the word frequencies of each word in the candidate keyword sequence based on the original article corpus, and delete the word segments in the candidate keyword sequence whose word frequencies are less than a preset threshold, as well as the word segments in a preset blacklist.

[0018] Optionally, based on the first metadata, a first control signal is generated according to the following preset rules: when the first metadata is in numerical form, its numerical value is used as the first control signal; when the first metadata is in non-numerical form, it is converted into a discrete numerical value within a finite numerical interval according to a conversion rule to serve as the first control signal; wherein, the first metadata is the metadata of the article sample and the first control signal is the control signal corresponding to the article sample, or, the first metadata is the target metadata and the first control signal is the target control signal.

[0019] Optionally, the metadata of the article sample and the input target metadata are one or more of author, article category, article type, and article length, and the metadata of the article sample and the input target metadata belong to the same one or more types of metadata.

[0020] Optionally, the text generation module is also used to: add the first word vector and the corresponding first position vector of each word in the target keyword sequence, and then encode it through the encoder of the controllable text generation model to obtain an encoding vector, wherein the first word vector is obtained by performing word embedding processing on the words of the target keyword sequence, and the first position vector is obtained by performing position encoding on the position information of the words of the target keyword sequence; calculate the probability distribution of generated vocabulary at each step in the decoder of the controllable text generation model, and select the vocabulary sequence with the largest probability as the text corresponding to the target keyword sequence, wherein the target sequence is obtained according to a specific marker word and the currently calculated vocabulary, and the probability distribution of the next step of generating vocabulary is calculated based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, the target control signal vector, and the encoding vector, wherein the second word vector is obtained by performing word embedding processing on the words of the target sequence, the second position vector is obtained by performing position encoding on the position information of the words of the target sequence, and the target control signal vector is obtained by performing word embedding processing on the target control signal.

[0021] Optionally, the target metadata is the multiple types of metadata, and the target control signal is the corresponding multiple types of control signals; the text generation module includes a target control signal vector generation submodule, which is used to: perform word embedding processing on each type of control signal of the target control signal to obtain a control signal vector corresponding to each type of control signal, and splice the obtained control signal vectors of each type in a preset order to obtain the target control signal vector.

[0022] According to yet another aspect of the embodiments of the present invention, an electronic device is provided.

[0023] An electronic device includes: one or more processors; and a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the text generation method provided by an embodiment of the present invention.

[0024] According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.

[0025] A computer-readable medium stores a computer program, which, when executed by a processor, implements a text generation method provided by an embodiment of the present invention.

[0026] One embodiment of the above invention has the following advantages or beneficial effects: generating a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample in the original article corpus, generating a control signal corresponding to the article sample according to the metadata of the article sample according to preset rules, generating a training sample according to the keyword sequence, label sequence, and control signal of the corresponding article sample, training a controllable text generation model, generating a target control signal according to preset rules based on the input target metadata, and generating a text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the input target keyword sequence. It can solve the problem that the length and novelty of the generated text are somewhat random, so that the generated text fully meets the actual online needs, and can generate texts of different styles, lengths, and article types based on the input keyword group and the content output requirements of different scenarios.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0029] Figure 1 1 is a schematic diagram of the main steps of the text generation method according to the first embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of the architecture of a controllable text generation model according to a second embodiment of the present invention;

[0031] Figure 3 2 is a schematic diagram of a method for processing a control signal vector Embedding layer according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of a text generation process according to a third embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of main modules of a text generation device according to a fourth embodiment of the present invention;

[0034] Figure 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0035] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] Figure 1 FIG. 1 is a schematic diagram of the main steps of the text generation method according to the first embodiment of the present invention.

[0038] like Figure 1 As shown, the text generation method according to one embodiment of the present invention mainly includes the following steps S101 to S103.

[0039] Step S101: Obtain the content and metadata of each article sample in the collected original article corpus.

[0040] Step S102: Generate a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, and generate a control signal corresponding to the article sample according to the metadata of the article sample and preset rules, generate a training sample based on the keyword sequence, label sequence, and control signal of the corresponding article sample, and train the controllable text generation model.

[0041] Step S103: Generate a target control signal according to the input target metadata and preset rules, and generate text corresponding to the target keyword sequence using a trained controllable text generation model based on the target control signal and the input target keyword sequence.

[0042] Some article samples can be collected in advance and stored in a database system, and the original article corpus can be collected from the database system, wherein each article sample includes two parts: metadata and content. The metadata of the article sample can include one or more of the author, article category, article type, and article length. Similarly, the input target metadata is also one or more of the author, article category, article type, and article length. Moreover, the metadata of the article sample and the input target metadata belong to the same one or more types of metadata, that is, if the metadata of the article sample of the original article corpus includes the author and article category, then the input target metadata should also include the author and article category.

[0043] In one embodiment, the steps of generating a keyword sequence corresponding to the article sample based on the content of the article sample specifically include: using a first word segmentation algorithm to segment the content of the article sample to obtain a word segmentation sequence; extracting word segments belonging to preset parts of speech from the word segmentation sequence, and based on the extracted word segments, generating a candidate keyword sequence according to the original order of the extracted word segments in the word segmentation sequence; deleting word segments that do not meet the preset conditions in the candidate keyword sequence to obtain a keyword sequence.

[0044] The preset parts of speech can be set according to the needs of the specific field. For example, in the scenario of generating product texts in the e-commerce field, it can be set to extract nouns and adjectives to obtain words that describe the product and the experience of use; for fields such as media, such as generating news texts, the preset parts of speech can be set to extract nouns, verbs and other words that describe events; for other writing fields, the preset parts of speech can be set according to the specific situation, which are not listed here one by one.

[0045] The step of generating a label sequence corresponding to the article sample according to the content of the article sample specifically includes: using the second word segmentation algorithm to segment the content of the article sample to obtain the label sequence.

[0046] The first word segmentation algorithm and the second word segmentation algorithm can adopt a general word segmentation algorithm, and the first word segmentation algorithm and the second word segmentation algorithm can be the same or different. Among them, when generating a keyword sequence, the word segmentation algorithm is required to be as accurate as possible. When generating a label sequence, you can consider using a word combination strategy (such as Sentence Piece, a word segmentation algorithm) to reduce the complexity of the model. Compared with the requirements for word segmentation accuracy when generating a keyword sequence, the requirements for word segmentation accuracy when generating a label sequence are not as high.

[0047] Deleting the segmented words in the candidate keyword sequence that do not meet the preset conditions may specifically include: counting the word frequency of each segmented word in the candidate keyword sequence based on the original article corpus, deleting the segmented words in the candidate keyword sequence whose word frequency is less than a preset threshold, and deleting the segmented words in the preset blacklist.

[0048] The frequency of each word in the candidate keyword sequence is counted based on the original article corpus, that is, the frequency of each word in the candidate keyword sequence in the original article corpus is counted. The preset blacklist can be a list of keywords that cannot be used to generate text based on the needs.

[0049] Based on the first metadata, a first control signal is generated according to the following preset rules: when the first metadata is in numerical form, its numerical value is used as the first control signal; when the first metadata is in non-numerical form, it is converted into a discrete numerical value within a finite numerical interval according to a conversion rule to serve as the first control signal; wherein the first metadata is metadata of an article sample and the first control signal is a control signal corresponding to the article sample, or the first metadata is target metadata and the first control signal is a target control signal.

[0050] According to the target control signal and the input target keyword sequence, a trained controllable text generation model is used to generate text corresponding to the target keyword sequence, specifically including: adding the first word vector and the corresponding first position vector of each word in the target keyword sequence, encoding through the encoder of the controllable text generation model to obtain an encoding vector, wherein the first word vector is obtained by performing word embedding processing on the words of the target keyword sequence, and each word in the target keyword sequence has its own first word vector, and the first position vector is obtained by performing position encoding on the position information of the words in the target keyword sequence, and each word in the target keyword sequence has its own first position vector; the probability distribution of the generated vocabulary is calculated at each step in the decoder of the controllable text generation model, and the vocabulary sequence with the largest probability is selected as the text corresponding to the target keyword sequence.

[0051] The target sequence is obtained based on the specific marker word and the currently calculated vocabulary. The probability distribution of the next vocabulary generation step is calculated based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, as well as the target control signal vector and the encoding vector output by the encoder. The second word vector is obtained by performing word embedding processing on the words in the target sequence, and each word in the target sequence has its own second word vector. The second position vector is obtained by positionally encoding the position information of the words in the target sequence, and each word in the target sequence has its own second position vector. The target control signal vector is obtained by performing word embedding processing on the target control signal.

[0052] With each computational step in the decoder, the currently calculated vocabulary increases, so the target sequence changes dynamically. The specific marker word, or BOS, marks the beginning of a sentence. The vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector is the vector obtained by adding the second word vector and the second position vector of the word corresponding to the same position.

[0053] Based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, as well as the target control signal vector and the encoding vector output by the encoder, when calculating the probability distribution of the next step of generating vocabulary, the vector obtained by first adding the second word vector of each word in the target sequence to the corresponding second position vector is spliced ​​or added to the target control signal vector, and then the obtained vector is input into the decoder, and the encoding vector output by the encoder is also input into the decoder. The decoder calculates the probability distribution of the next step of generating vocabulary based on the above-mentioned vector input into the decoder.

[0054] Word embedding is a mechanism in deep learning that maps discrete values ​​into dense vectors.

[0055] After calculating the probability distribution of the generated vocabulary at each step in the decoder of the controllable text generation model, and before the text corresponding to the target keyword sequence, the output of the decoder can be processed through a linear layer and a normalization layer.

[0056] In one embodiment, the target metadata includes multiple types of metadata, and the target control signals include multiple types of corresponding control signals. A target control signal vector is obtained by performing word embedding processing on the target control signals. Specifically, this may include performing word embedding processing on each type of control signal in the target control signal to obtain a control signal vector corresponding to each type of control signal, and concatenating the obtained control signal vectors of each type in a preset order to obtain the target control signal vector. For example, if the target metadata includes author, article category, article type, and article length, then the control signal vectors corresponding to the author, article category, article type, and article length can be concatenated in that order to obtain the target control signal vector.

[0057] The following takes the generation of product text in the e-commerce field as an example to describe in detail the text generation method of an embodiment of the present invention.

[0058] The embodiment of the present invention can fully utilize the information of commodity data to control the generation of text, so as to meet the needs of generating different commodity texts in different application scenarios. The embodiment of the present invention mainly includes three parts: preparing data, constructing a model, and generating text.

[0059] Part 1: Preparing Data

[0060] First, data collection is carried out, that is, collecting original article corpus from the database system. The original article corpus includes a large number of article samples, and each article sample includes metadata and content.

[0061] The collected data is preprocessed. This is mainly used to construct training samples. Data preprocessing specifically includes the following operations: generating a control signal C, generating a keyword sequence X, and generating a label sequence Y.

[0062] Generate a control signal C. Specifically, convert the metadata into a discrete or continuous control signal. If it is a continuous signal, such as the length of the article, which is in numerical form, it can be directly processed as a floating point number. The length of the article can be obtained by directly counting the length of the content, such as counting the number of characters or the number of words after word segmentation. The embodiment of the present invention does not limit this. If it is a discrete value, such as the author, which is in non-numerical form, these authors can be converted into discrete information of [1, V], where V represents the final number of controllable authors, and [1, V] is a closed interval. The discrete information obtained by converting the author can be used as the ID (identifier) ​​of the author. Convert the author into a discrete control signal. For example, it can be sorted by the number of articles of each author. For authors with a large number of articles, they are mapped one by one to IDs. For a group of authors with a small number of articles, they can be mapped to the same ID. The standard for the number of articles can be customized. For example, the number of articles greater than or equal to 100 is defined as a large number of articles. For example, as shown in Table 1, the IDs of the authors corresponding to the last three rows are the same ID, ID=4.

[0063] Table 1

[0064] author Number of articles Mapped control signal: ID Zhang San 1000 1 Li Si 800 2 Wang Wu 100 3 Liu Qi 7 4 Jiang Ba 6 4 Zhu Jiu 5 4

[0065] Generate a keyword sequence X. Specifically, after the content of the article sample is segmented using the first segmentation algorithm, nouns and adjectives can be extracted based on part-of-speech judgment. The extracted segmented words remain unchanged in their original order to form a candidate keyword sequence. Based on the original article corpus, the word frequencies of these segmented words in the candidate keyword sequence in the original article corpus are counted, and segmented words with a word frequency lower than Thd (Thd represents a preset threshold) or segmented words in the blacklist are removed. The final sequence obtained is the keyword sequence X.

[0066] Through keyword extraction, the article data is converted into a supervised method based on keyword sequence prediction of article sequence, making it possible to utilize massive continuous text data.

[0067] Generate a label sequence Y. Specifically, the article sample content is segmented using the second segmentation algorithm to obtain the label sequence Y.

[0068] The word segmentation algorithms used to generate X and Y can use a common word segmentation algorithm, and the algorithms used can be the same or different. When generating X, the word segmentation algorithm is required to be as accurate as possible, for example, the Stanford CoreNLP Chinese word segmentation algorithm. When generating Y, a word combination strategy (for example, Sentence Piece) can be considered to reduce the complexity of the controllable text generation model. The word segmentation accuracy requirement is not as high as when generating X.

[0069] After completing the above-mentioned data collection and data preprocessing, the training samples of each article sample can be constructed based on the above-mentioned X, Y, and C obtained by preprocessing. The training sample of an article sample is the (X, Y, C) of the article sample, and the (X, Y, C) of all article samples constitute the final data set.

[0070] Table 2 shows the raw data of a sample article. Table 3 shows the data after data preprocessing. The article type, article category, and author are discrete information, and the specific discrete information values ​​are not shown in Table 3. The target text Y in Table 3 is the label sequence Y described above.

[0071] Table 2

[0072]

[0073] Table 3

[0074]

[0075] Part 2: Constructing the Model

[0076] Constructing a model is to construct a controllable text generation model according to an embodiment of the present invention. The controllable text generation model according to an embodiment of the present invention is as follows: Figure 2 As shown, the architecture design mainly adopts an encoder and decoder, and both use the Transformer model (a transformation model based on the self-attention mechanism). The encoder specifically includes N layers of Transformer encoder layers, and the decoder specifically includes N layers of Transformer decoder layers. The two "N×" in the figure respectively indicate the number of Transformer encoder layers and Transformer decoder layers. This embodiment of the present invention introduces a control signal mechanism in the decoder.

[0077] The input keyword sequence X is processed by the input Embedding layer and then combined with the position encoding layer (denoted as the first position encoding layer, such as Figure 2 The output of the position encoding (as shown in I) is added, and then enters the N-layer Transformer encoder layer for encoding to obtain the encoding vector, and the encoding vector output by the encoder is connected to the decoder.

[0078] The words in the sequence obtained by shifting the target text Y right by one position are embedded in the output Embedding layer and then combined with the position encoding layer (denoted as the second position encoding layer, such as Figure 2The outputs of the position encoding II are added to obtain the vector Ye; the control signal C is word embedded through the control signal Embedding layer to obtain the vector Ce; the vectors Ye and Ce are concatenated together and enter the N-layer Transformer decoding layer. Finally, they are processed through the linear layer and Softmax (normalization layer) to output the probability of the predicted sequence.

[0079] As an alternative implementation, the concatenation of Ye and Ce can be replaced by addition or mapping through a linear layer.

[0080] The input Embedding layer is used to perform word embedding processing on each word in the keyword sequence X, so that each word in the keyword sequence X is converted into a word vector (i.e., the first word vector).

[0081] The first position encoding layer is used to positionally encode the position information of each word in the keyword sequence X, mapping each position information into a vector, namely the first position vector. For example, if X = [white, scorching, summer], then the position information of each word is [1, 2, 3]. The position encoding maps each position into a vector, for example, mapping position 1 into a vector. The output of the first position encoding layer is the first position vector of each word in the keyword sequence X.

[0082] The output Embedding layer is used to perform word embedding processing on each word in the sequence obtained by shifting the target text Y right by one position, thereby converting each word into a word vector (i.e., the second word vector). The sequence obtained by shifting the target text Y right by one position uses the BOS (marker indicating the beginning of a sentence) as the first word in the sequence, and the positions after the BOS are the words of the target text Y.

[0083] The second position encoding layer is used to perform position encoding on the position information of each word in the sequence obtained by shifting the target text Y right by one position, so as to map each position information into a vector, namely, the second position vector.

[0084] In the process of calculating the probability of the predicted sequence, the probability distribution of the generated words is calculated at each step in the decoder, and the word sequence with the highest probability is finally selected as the output predicted sequence. Let Y1, Y2, Y3, Y4... represent each element of Y. Accordingly, the sequence BOS, Y1, Y2, Y3... obtained by shifting the target text Y right by one bit is used in the calculation of Y. j When the probability distribution of (representing the jth element of Y) is calculated, it is necessary to use the Y1~Y j-1 Specifically, according to BOS and the currently calculated Y1~Y j-1Get the target sequence, and calculate Y based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector (i.e., Ye), the target control signal vector (i.e., Ce), and the encoding vector output by the encoder. j The probability distribution of .

[0085] When the control signal C is word-embedded through the control signal Embedding layer, if the control signal is of multiple types, the vectors obtained by word embedding for each type of control signal are concatenated to obtain the final control signal vector, i.e., Ce. For discrete control signals, such as author, category, and type, the processing logic is the same as that for generating word vectors. For continuous control signals, such as length, they can be directly converted to floating-point numbers with a dimension of 1. Finally, the discrete control signal vectors and the continuous control signal vectors are concatenated in a fixed order to form the final control signal vector. Figure 3 is a schematic diagram of a method for processing a control signal vector Embedding layer according to an embodiment of the present invention. Figure 3 , perform embedding processing on the control signals such as author, category, and type respectively, and convert the length into a floating point number 88.0. Then, concatenate the four, and the resulting vector is the final control signal vector.

[0086] The decoder in the embodiment of the present invention introduces a control signal at the Embedding layer to realize control text generation.

[0087] The controllable text generation model constructed in the second part is trained using the data set obtained in the data preparation stage in the first part to obtain a trained controllable text generation model.

[0088] Part 3: Generating Text

[0089] The user inputs metadata and keyword phrases, and the metadata is converted into discrete or continuous control signals. For discrete control signals, for example, the user selects within the range supported by the controllable text generation model of the embodiment of the present invention (for example, supported authors, categories, types, etc.); for continuous control signals, the user selects within the length range supported by the controllable text generation model (the length range is determined by the maximum and minimum lengths of the articles in the dataset, for example, the article length is [20,100]).

[0090] The trained controllable text generation model generates the corresponding output text Y according to the control signal and keyword group input by the user.

[0091] For example, the user inputs the keyword phrase X: "elegant and beautiful." Table 4 shows the output text under four different control signal groups. The control Cs in Table 4 represent the original form of the corresponding control signal (i.e., the initially extracted metadata). When generating text, the embodiment of the present invention converts each control C into a corresponding discrete numerical form.

[0092] Table 4

[0093]

[0094] Figure 4 2 is a flowchart of text generation according to the third embodiment of the present invention.

[0095] like Figure 4 As shown, the text generation process of one embodiment of the present invention includes a training phase and a prediction phase of a controllable text generation model.

[0096] During the training phase, original article corpus (raw data), consisting of metadata and content, is collected from the database system. The raw data is preprocessed to obtain training samples (X, C, Y), where C is the control signal, X is the keyword sequence, and Y is the label sequence. The training samples are used to train the controllable text generation model. The output of the model in the training phase is the probability distribution P(Y|X, C) of Y under the conditions of X and C.

[0097] In the prediction stage, the input control signal is C', the input keyword sequence is X', and the output generation result Y is generated by the trained controllable text generation model. est .

[0098] The metadata of the embodiment of the present invention may include the length of the article, the author, the category information corresponding to the article, the article type, etc., so that texts of different styles, lengths, and article types can be planned and generated based on the keyword phrases and metadata input by the user according to the content output requirements of different scenarios, rather than just simple text continuation.

[0099] The text generation process of the embodiment of the present invention is not only applicable to the generation of text about products in the e-commerce field, but also to other writing fields. As long as there is text and metadata, the model of the embodiment of the present invention can be applied. When the metadata only includes the length, the length of the generated text can be controlled.

[0100] Figure 5 4 is a schematic diagram of the main modules of a text generation device according to a fourth embodiment of the present invention.

[0101] like Figure 5 As shown, a text generation device 500 according to an embodiment of the present invention mainly includes: a data acquisition module 501 , a model training module 502 , and a text generation module 503 .

[0102] The data acquisition module 501 is used to acquire the content and metadata of each article sample in the collected original article corpus.

[0103] The model training module 502 is used to generate a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, and generate a control signal corresponding to the article sample according to the metadata of the article sample and preset rules, and generate a training sample based on the keyword sequence, label sequence and control signal of the corresponding article sample to train the controllable text generation model.

[0104] The text generation module 503 is used to generate a target control signal according to the input target metadata and preset rules, and generate text corresponding to the target keyword sequence based on the target control signal and the input target keyword sequence using a trained controllable text generation model.

[0105] The model training module 502 may include a keyword sequence generation submodule, which is used to: use the first word segmentation algorithm to segment the content of the article sample to obtain a word segmentation sequence; extract word segments belonging to preset parts of speech from the word segmentation sequence, and based on the extracted word segments, generate a candidate keyword sequence according to the original order of the extracted word segments in the word segmentation sequence; delete the word segments in the candidate keyword sequence that do not meet the preset conditions to obtain a keyword sequence.

[0106] The model training module 502 may further include a label sequence generation submodule, which is used to segment the content of the article sample using a second word segmentation algorithm to obtain a label sequence.

[0107] The keyword sequence generation submodule may include a word segmentation filtering subunit, which is used to: count the word frequencies of each word in the candidate keyword sequence based on the original article corpus, and delete the word segments in the candidate keyword sequence whose word frequencies are less than a preset threshold and the word segments in the preset blacklist.

[0108] Based on the first metadata, a first control signal is generated according to the following preset rules: when the first metadata is in numerical form, its numerical value is used as the first control signal; when the first metadata is in non-numerical form, it is converted into a discrete numerical value within a finite numerical interval according to a conversion rule to serve as the first control signal; wherein the first metadata is metadata of an article sample and the first control signal is a control signal corresponding to the article sample, or the first metadata is target metadata and the first control signal is a target control signal.

[0109] The metadata of the article sample and the input target metadata may be one or more of author, article category, article type, and article length, and the metadata of the article sample and the input target metadata belong to the same one or more types of metadata.

[0110] The text generation module 503 can be specifically used to: add the first word vector and the corresponding first position vector of each word in the target keyword sequence, and then encode it through the encoder of the controllable text generation model to obtain an encoding vector, wherein the first word vector is obtained by performing word embedding processing on the words of the target keyword sequence, and the first position vector is obtained by performing position encoding on the position information of the words of the target keyword sequence; calculate the probability distribution of the generated vocabulary at each step in the decoder of the controllable text generation model, and select the vocabulary sequence with the largest probability as the text corresponding to the target keyword sequence, wherein the target sequence is obtained according to the specific marker word and the currently calculated vocabulary, and the probability distribution of the vocabulary generated in the next step is calculated based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, the target control signal vector, and the encoding vector, wherein the second word vector is obtained by performing word embedding processing on the words of the target sequence, the second position vector is obtained by performing position encoding on the position information of the words of the target sequence, and the target control signal vector is obtained by performing word embedding processing on the target control signal.

[0111] In one embodiment, the target metadata is multiple types of metadata, and the target control signal is a corresponding multiple types of control signal; the text generation module 503 may include a target control signal vector generation submodule, which is used to: perform word embedding processing on each type of control signal of the target control signal to obtain a control signal vector corresponding to each type of control signal, and splice the obtained control signal vectors of each type in a preset order to obtain a target control signal vector.

[0112] In addition, the specific implementation content of the text generation device in the embodiment of the present invention has been described in detail in the text generation method described above, so the repeated content will not be described again here.

[0113] Figure 6 An exemplary system architecture 600 is shown to which the text generation method or text generation apparatus according to the embodiment of the present invention can be applied.

[0114] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0115] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0116] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0117] Server 605 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 601, 602, and 603. The backend management server may analyze and process received data such as product information query requests, and feed back processing results (for example, target push information and product information—for example only) to the terminal device.

[0118] It should be noted that the text generation method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the text generation device is generally set in the server 605 .

[0119] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0120] Reference below Figure 7 , which shows a schematic structural diagram of a computer system 700 of a terminal device or server suitable for implementing an embodiment of the present invention. Figure 7 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0121] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the system 700 are also stored in the RAM 703. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0122] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0123] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0124] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor including a data acquisition module, a model training module, and a text generation module. The names of these modules do not, in some cases, limit the modules themselves. For example, the data acquisition module may also be described as a module for acquiring the content and metadata of each article sample in the collected original article corpus.

[0127] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining the content and metadata of each article sample in the collected original article corpus; generating a keyword sequence and a label sequence corresponding to the article sample based on the content of the article sample, and generating a control signal corresponding to the article sample according to a preset rule based on the metadata of the article sample; generating a training sample based on the keyword sequence, label sequence, and control signal corresponding to the article sample, and training a controllable text generation model; generating a target control signal according to the preset rule based on the input target metadata, and generating a text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the input target keyword sequence.

[0128] According to the technical solution of the embodiment of the present invention, a keyword sequence and a label sequence corresponding to the article sample are generated based on the content of the article sample in the original article corpus, and a control signal corresponding to the article sample is generated according to the metadata of the article sample in accordance with preset rules. A training sample is generated based on the keyword sequence, label sequence, and control signal of the corresponding article sample, and a controllable text generation model is trained. A target control signal is generated according to preset rules based on the input target metadata. Based on the target control signal and the input target keyword sequence, the trained controllable text generation model is used to generate text corresponding to the target keyword sequence. This can solve the problem of a certain degree of randomness in the length and novelty of the generated text, so that the generated text fully meets the actual online needs, and can generate text of different styles, lengths, and article types based on the input keyword group and the content output requirements of different scenarios.

[0129] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A text generation method, characterized in that: include: Obtain the content and metadata of each article sample in the collected original article corpus; Generate a keyword sequence and a tag sequence corresponding to the article sample based on the content of the article sample, generate a control signal corresponding to the article sample according to a preset rule based on the metadata of the article sample, generate a training sample based on the keyword sequence, tag sequence, and control signal corresponding to the article sample, and train a controllable text generation model; Generate a target control signal according to the input target metadata and the preset rules, and generate text corresponding to the target keyword sequence using the trained controllable text generation model based on the target control signal and the input target keyword sequence; The metadata of the article sample and the input target metadata are one or more of author, article category, article type, and article length, and the metadata of the article sample and the input target metadata belong to the same one or more types of metadata.

2. The method according to claim 1, characterized in that The step of generating a keyword sequence corresponding to the article sample according to the content of the article sample includes: Using a first word segmentation algorithm to segment the content of the article sample to obtain a word segmentation sequence; Extracting segmented words belonging to preset parts of speech from the segmented word sequence, and generating a candidate keyword sequence based on the extracted segmented words and in accordance with the original order of the extracted segmented words in the segmented word sequence; Deleting the word segments that do not meet the preset conditions in the candidate keyword sequence to obtain the keyword sequence; The step of generating a tag sequence corresponding to the article sample according to the content of the article sample includes: The second word segmentation algorithm is used to segment the content of the article sample to obtain the label sequence.

3. The method according to claim 2, characterized in that The step of deleting the segmented words that do not meet the preset conditions in the candidate keyword sequence includes: The word frequency of each segmented word in the candidate keyword sequence is counted based on the original article corpus, and the segmented words in the candidate keyword sequence whose word frequency is less than a preset threshold and the segmented words in a preset blacklist are deleted.

4. The method according to claim 1, wherein According to the first metadata, a first control signal is generated according to the following preset rules: When the first metadata is in numerical form, the numerical value thereof is used as the first control signal; When the first metadata is in a non-numerical form, converting it into a discrete numerical value within a finite numerical range according to a conversion rule to serve as the first control signal; The first metadata is metadata of the article sample, and the first control signal is a control signal corresponding to the article sample, or the first metadata is the target metadata, and the first control signal is the target control signal.

5. The method according to claim 1, wherein The step of generating text corresponding to the target keyword sequence using the trained controllable text generation model according to the target control signal and the input target keyword sequence includes: After adding the first word vector and the corresponding first position vector of each word in the target keyword sequence, encoding is performed through the encoder of the controllable text generation model to obtain an encoding vector, wherein the first word vector is obtained by performing word embedding processing on the words of the target keyword sequence, and the first position vector is obtained by performing position encoding on the position information of the words in the target keyword sequence; The probability distribution of vocabulary is calculated at each step in the decoder of the controllable text generation model, and the vocabulary sequence with the highest probability is selected as the text corresponding to the target keyword sequence, wherein the target sequence is obtained according to the specific marker word and the currently calculated vocabulary, and the probability distribution of vocabulary generation in the next step is calculated based on the vector obtained by adding the second word vector of each word in the target sequence to the corresponding second position vector, the target control signal vector, and the encoding vector, wherein the second word vector is obtained by performing word embedding processing on the words of the target sequence, the second position vector is obtained by performing position encoding on the position information of the words of the target sequence, and the target control signal vector is obtained by performing word embedding processing on the target control signal.

6. The method according to claim 5, characterized in that The target metadata is the multiple types of metadata, and the target control signals are the corresponding multiple types of control signals; The step of obtaining the target control signal vector by performing word embedding processing on the target control signal includes: Perform word embedding processing on each type of control signal of the target control signal to obtain a control signal vector corresponding to each type of control signal, and splice the obtained control signal vectors of each type in a preset order to obtain the target control signal vector.

7. A text generation device, characterized in that: include: The data acquisition module is used to obtain the content and metadata of each article sample in the collected original article corpus; A model training module is configured to generate a keyword sequence and a tag sequence corresponding to the article sample based on the content of the article sample, generate a control signal corresponding to the article sample according to preset rules based on the metadata of the article sample, generate a training sample based on the keyword sequence, tag sequence, and control signal corresponding to the article sample, and train the controllable text generation model; a text generation module, configured to generate a target control signal according to the preset rules based on the input target metadata, and generate text corresponding to the target keyword sequence based on the target control signal and the input target keyword sequence using the trained controllable text generation model; The metadata of the article sample and the input target metadata are one or more of author, article category, article type, and article length, and the metadata of the article sample and the input target metadata belong to the same one or more types of metadata.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images