Text generation method and device, electronic equipment and readable medium

Abstract

Embodiments of the present application provide a text generation method and device, electronic equipment and readable medium, obtain a to-be-processed text with format requirements; the format requirements at least include syllable format requirements; perform word segmentation processing on the to-be-processed text based on syllables to obtain a plurality of syllable units; input the syllable units of the to-be-processed text into a pre-trained text generation model to obtain a target generated text meeting the format requirements. Embodiments of the present application can generate a target generated text meeting the format requirements including syllable format, realize syllable controllability, and improve the quality of the target generated text.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a text generation method, a text generation device, an electronic device, and a computer-readable medium. Background Technology

[0002] In recent years, the rapid progress in machine learning, especially deep learning, has greatly transformed human-computer interaction and led to the widespread adoption of more and more commercial products. Text generation in Natural Language Processing (NLP) is a prime example of this.

[0003] Specifically, text generation refers to training a machine, especially deep learning, to generate text freely, achieving automatic conversion from data to text. It is a significant indicator of the maturity of artificial intelligence. In real life, text generation plays a vital role. For example, in text-based games, automatically generating diverse, high-quality, and engaging text can enhance the player experience and thus increase the game's competitive value.

[0004] However, current text generation technology still needs improvement and cannot generate higher quality text. Summary of the Invention

[0005] The present invention provides a text generation method, apparatus, electronic device, and computer-readable storage medium to address the problem that current text generation technology still needs improvement and cannot generate higher quality text.

[0006] This invention discloses a text generation method, including:

[0007] Obtain the text to be processed that has format requirements; the format requirements include at least syllable format requirements;

[0008] The text to be processed is segmented into words based on syllables to obtain several syllable units;

[0009] The syllable units of the text to be processed are input into a pre-trained text generation model to obtain target generated text that meets the format requirements.

[0010] Optionally, the step of inputting the syllable units of the text to be processed into a pre-trained text generation model to obtain target generated text that meets the format requirements includes:

[0011] input the syllable units of the text to be processed into a pre-trained text generation model, so that the text generation model outputs a target generated text meeting the format requirements based on the format identifier after determining the format identifier of the text to be processed.

[0012] Optionally, the format requirements further include text format requirements, and the format identifier includes at least a format identifier for representing syllable types and prosodic information, a format identifier for representing internal sentence position information, a format identifier for representing sentence position information, a format identifier for representing global position information, and a format identifier for representing syllable format.

[0013] Optionally, the text generation model is trained in the following manner:

[0014] Obtain each sample syllable unit in the vocabulary and the format identifier of each sample syllable unit.

[0015] Train the text generation model using the each sample syllable unit and the format identifier of each sample syllable unit to obtain a trained text generation model.

[0016] Optionally, the vocabulary is generated in the following manner:

[0017] Obtain a first sample corpus ;

[0018] Perform syllable-based word segmentation on the first sample corpus to obtain sample syllable units.

[0019] Iteratively count the frequency of adjacent sample syllable unit pairs, and select the adjacent sample syllable unit pair combination with the highest frequency as a word.

[0020] When the number of words reaches an expected number or the frequency of the next adjacent sample syllable unit pair to be counted is 1, the word is added to the vocabulary.

[0021] Optionally, after the training of the text generation model using the each sample syllable unit and the format identifier of each sample syllable unit to obtain a trained text generation model, the method further includes:

[0022] Determine a target topic word and retrieve a text segment associated with the target topic word.

[0023] Train the text generation model using the text segment as a second sample corpus.

[0024] Optionally, after the text generation model is trained by using the sample syllable units and the format identifiers of the sample syllable units, and a trained text generation model is obtained, the method further comprises:

[0025] obtaining a third sample corpus as a positive sample and a fourth sample corpus as a negative sample;

[0026] inputting the third sample corpus and the fourth sample corpus into a pre-trained classification model, and determining a fifth sample corpus from the fourth sample corpus;

[0027] training the text generation model by using the third sample corpus and the fifth sample corpus.

[0028] Optionally,

[0029] the to-be-processed text at least includes English lyrics;

[0030] the target theme at least includes children, and the to-be-processed text at least includes English nursery rhyme lyrics.

[0031] Embodiments of the present application also disclose a text generation device, comprising:

[0032] a text acquisition module configured to acquire to-be-processed text having format requirements; the format requirements at least include syllable format requirements;

[0033] a syllable segmentation module configured to perform syllable-based segmentation processing on the to-be-processed text to obtain a plurality of syllable units;

[0034] a target text generation module configured to input the syllable units of the to-be-processed text into a pre-trained text generation model to obtain target generated text meeting the format requirements.

[0035] Optionally, the target text generation module is configured to input the syllable units of the to-be-processed text into the pre-trained text generation model, so that the text generation model outputs target generated text meeting the format requirements based on a format identifier of the to-be-processed text after determining the format identifier.

[0036] Optionally, the format requirements further include text format requirements, and the format identifier at least includes a format identifier for representing syllable types and prosodic information, a format identifier for representing internal sentence position information, a format identifier for representing sentence position information, a format identifier for representing global position information, and a format identifier for representing syllable format.

[0037] Optionally, the apparatus further comprises a first text generation model training module configured to acquire each sample syllable unit in the vocabulary and a format identifier of the each sample syllable unit; and train the text generation model using the each sample syllable unit and the format identifier of the each sample syllable unit to obtain a trained text generation model.

[0038] Optionally, the apparatus further comprises a vocabulary generation module configured to acquire a first sample corpus ; perform syllable-based word segmentation on the first sample corpus to obtain sample syllable units; iteratively count frequencies of adjacent sample syllable unit pairs, and select a combination of adjacent sample syllable units with the highest frequency as a word; and add the word to the vocabulary when a number of words reaches an expected number or a frequency of an adjacent sample syllable unit pair counted next time is 1.

[0039] Optionally, the apparatus further comprises a second text generation model training module configured to determine a target theme word, retrieve a text segment associated with the target theme word, and train the text generation model using the text segment as a second sample corpus.

[0040] Optionally, the apparatus further comprises a third text generation model training module configured to acquire a third sample corpus as a positive sample and a fourth sample corpus as a negative sample, input the third sample corpus and the fourth sample corpus into a pre-trained classification model to determine a fifth sample corpus from the fourth sample corpus, and train the text generation model using the third sample corpus and the fifth sample corpus.

[0041] Optionally, the text to be processed at least includes English lyrics; and the target theme at least includes children, and the text to be processed at least includes English children's song lyrics.

[0042] The embodiment of the present application further discloses an electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus.

[0043] The memory is used for storing a computer program.

[0044] The processor is used for executing the program stored on the memory, and realizes the method as described in the embodiment of the present application.

[0045] The embodiment of the present application further discloses one or more computer readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method as described in the embodiment of the present application.

[0046] The embodiment of the present application has the following advantages:

[0047] In the embodiment of the present application, the text to be processed with format requirements is obtained, wherein the format requirements at least include syllable format. After the syllable units of the text to be processed are obtained by performing syllable-based word segmentation processing on the text to be processed, the syllable units of the text to be processed are input into the pre-trained text generation model to obtain the target generated text meeting the format requirements. The embodiment of the present application can generate the target generated text meeting the format requirements including syllable format, realizes syllable controllability, and improves the quality of the target generated text. BRIEF DESCRIPTION OF DRAWINGS

[0048] figure 1 is a step flowchart of a text generation method provided in the embodiment of the present application;

[0049] figure 2 is a model architecture block diagram of text generation provided in the embodiment of the present application;

[0050] figure 3 is a schematic diagram of multi-stage model training provided in the embodiment of the present application;

[0051] figure 4 is a structure block diagram of a text generation device provided in the embodiment of the present application;

[0052] figure 5 is a block diagram of an electronic device provided in the embodiment of the present application;

[0053] figure 6 is a schematic diagram of a computer readable medium provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0055] Some technical terms related to the embodiments of the present application are introduced as follows:

[0056] Syllable: a unit of phonetic sequence, which is also the most natural phonetic structure unit in phonetics.

[0057] Self-Attention: a method of weighting and summing values based on the similarity between query and key to generate a new representation.

[0058] Transformer: a deep learning model based entirely on self-attention mechanism.

[0059] SongNet: Based on the Transformer structure, introduce some special identifiers to achieve a format controllable text generation model, for example, it can generate a word controllable text.

[0060] BPE: Byte Pair Encoding, the main purpose is data compression, BPE algorithm is described as a pair of characters in the string that is most common in frequency is replaced by a character that does not appear in this character in a layer-by-layer iterative process.

[0061] OOV: out of vocabulary, also known as out-of-vocabulary words, words that do not appear during model training but appear during testing.

[0062] GPT-2: A self-regressive language model based on Transformer and large corpus pre-training.

[0063] Current text generation methods are mainly divided into two types: 1) free text generation: the model requires the generated text to be natural text, and the representative model is GPT-2 model; 2) controllable text generation: the model not only generates natural text, but also meets the given format requirements (for example, the generated text needs to meet a certain number of words), and the representative model is SongNet. However, the existing text generation method cannot realize the syllable controllable (meet the specified syllable format).

[0064] In addition, if you need to generate English text, because the English word form is very rich, if still create a text generation model in word unit, will greatly increase the size of the vocabulary of the text generation model, and further increase the parameters of the text generation model. In addition, the text generation model usually depends on more training samples.

[0065] To solve the above problems, the embodiment of the present application provides a text generation method, which can output target generated text meeting the syllable format requirements, so as to generate target generated text meeting the syllable format requirements after determining the specified syllable format according to the to-be-processed text.

[0066] Referring to figure 1 , a step flowchart of a text generation method provided in the embodiment of the present application is shown, which can specifically include the following steps:

[0067] Step 101, obtaining a to-be-processed text with format requirements; the format requirements at least include syllable format requirements.

[0068] Among them, the to-be-processed text is a text with syllable characteristics, for example, English.

[0069] In a specific implementation, a syllable is a basic unit of speech, a unit of speech expressing meaning, a carrier unit of morpheme as the smallest phonetic and semantic combination of language, and for English, there are usually multiple pronunciations (syllables) for a word, and the syllables need to be effectively controlled, therefore, in the embodiment of the application, English words in English need to meet the syllable format requirement, that is, when performing word segmentation on English words or using syllables to form English words, the English words cannot be split into other syllable formats or randomly combined into English words in a syllable format.

[0070] For example, syllable

stu

dent

student

la

bour

labour

tea

cher

teacher

stu

cher

stucher

la

tea

stu

lateastu

[0071] In the embodiment of the application, if a user wants to automatically replace the lyrics of an English children's song, the English children's song lyrics can be used as the text to be processed, or if the user wants to automatically replace the lyrics of an English poem, the English poem can be used as the text to be processed.

[0072] Step 102, performing word segmentation processing on the text to be processed based on syllables to obtain a plurality of syllable units.

[0073] In the embodiment of the application, a BPE algorithm taking a word syllable as a unit is designed, and specifically, a method of syllable segmentation on a word is designed based on language rules, for example, for the English word

wonder

won

der

[0074] Step 103, inputting the syllable units of the text to be processed into a pre-trained text generation model to obtain a target generated text meeting the format requirement.

[0075] In the embodiment of the application, after the text to be processed is subjected to word segmentation processing to obtain syllable units, the syllable units can be input into a pre-trained text generation model to further obtain a target generated text meeting the syllable format requirement.

[0076] In the text generation method, the text to be processed with format requirements is obtained, wherein the format requirements at least include syllable format, and after syllable-based word segmentation processing of the text to be processed is performed to obtain a plurality of syllable units, the syllable units of the text to be processed are input into a pre-trained text generation model to obtain target generated text meeting the format requirements. The embodiment of the present application can generate target generated text meeting the format requirements including syllable format, syllable control is achieved, and the quality of the target generated text is improved.

[0077] On the basis of the above-mentioned embodiments, variant embodiments of the above-mentioned embodiments are proposed, and it should be noted that, in order to make the description brief, only the differences from the above-mentioned embodiments are described in the variant embodiments.

[0078] In an exemplary embodiment, the step 103 of inputting the syllable units of the text to be processed into a pre-trained text generation model to obtain target generated text meeting the format requirements can include the following steps:

[0079] The syllable units of the text to be processed are input into a pre-trained text generation model, so that the text generation model outputs target generated text meeting the format requirements based on the format identifier after determining the format identifier of the text to be processed.

[0080] The format identifier at least includes a format identifier for representing syllable format.

[0081] Specifically, referring to the English word "wonder" in figure 2 , figure 2 , the corresponding syllable block embedding token sequence (format identifier) is [t1, t0], and through the format identifier, the text generation model can perceive the boundary information of syllables, that is, how many syllables should be contained in one English word and the starting position of the syllable.

[0082] In addition, the format requirements further include text format requirements, and the format identifier of the text format requirements at least includes a format identifier for representing syllable type and prosody information, a format identifier for representing intra-sentence position information, a format identifier for representing sentence position information, and a format identifier for representing global position information.

[0083] Specifically, the syllable type refers to the type to which the syllable belongs, and the syllable type includes language symbols, punctuation symbols, sentence ending symbols, and text ending symbols, etc.; the intra-sentence position information refers to the position of the syllable unit in the sentence to which it belongs; the sentence position information refers to the position of the sentence to which the syllable unit belongs in the text to be processed; and the global position information refers to the position of the syllable unit in the text to be processed.

[0084] For the convenience of those skilled in the art to understand, the following is described with examples.

[0085] As a specific example, referring to figure 2 When the input text generation model is to be processed text "How I wonder what you are", the syllable-based BPE segmentation is first performed on the to-be-processed text to obtain syllable units

How

I

won

der

what

you

are

[0086] 1. The format identifier set of the symbol type and prosody information of each syllable unit in the to-be-processed text: C={c0,c0,c0,c0,c0,c0,c1},wherein c0 is the format identifier of the non-foot language symbol, and c1 is the format identifier of the punctuation symbol.

[0087] 2. The format identifier set of the intra-sentence position information of each syllable unit in the to-be-processed text, P={p0,p1,p2,p3,p4,p5,p6},wherein pJ-j is the format identifier of the jth position in the sentence including J syllable units. j is a positive integer.

[0088] 3. The format identifier set of the sentence position information of each syllable unit in the to-be-processed text, S={s0,s0,s0,s0,s0,s0,s0},wherein sr-1 is the format identifier of each syllable unit in the rth sentence in the to-be-processed text. r is a positive integer.

[0089] 4. The format identifier set of the full-text position information of each syllable unit in the to-be-processed text, G={g0,g1,g2,g3,g4,g5,g6},wherein gx-1 is the format identifier of the syllable unit at the xth position in the to-be-processed text. x is a positive integer.

[0090] 5. The format identifier set of the syllable format of each syllable unit in the to-be-processed text, T={t0,t0,t1,t0,t0,t0,t0}.

[0091] In the embodiment of the present application, after the syllable units of the to-be-processed text are input into the pre-trained text generation model to obtain the format identifier of the to-be-processed text, the target generated text meeting the format requirement is output based on the format identifier.

[0092] In an exemplary embodiment, the text generation model is trained in the following manner:

[0093] obtaining each sample syllable unit in the vocabulary and a format identifier of the each sample syllable unit;

[0094] training the text generation model by using the each sample syllable unit and the format identifier of the each sample syllable unit, to obtain a trained text generation model.

[0095] Referring to figure 2 , the text generation model of the embodiment of the present application mainly includes two modules about syllable units: a subword embedding module and a syllable block embedding module. Among them, the subword embedding module is mainly responsible for learning the semantic information of a word (note that the existing method SongNet is to model the entire word directly, while the text generation model of the embodiment of the present application models the word in units of syllables), and the syllable block embedding aims to strengthen the modeling of syllable information. Next, the syllable block embedding module is described, such as figure 2 wonder in the word, and the corresponding syllable block embedding mark sequence is [t1, t0]. Through this mark sequence, the model can perceive the boundary information of the syllable, that is, how many syllables should be contained in a word and the starting position of the syllable.

[0096] In the embodiment of the present application, in order to improve the quality of the target generated text generated by the text generation model, pre-training on general large corpus is designed. Because for the text generation model, the more corpus seen in principle, the more natural and smooth the text generated by the model tends to be. Therefore, the model is pre-trained on the corpus after preprocessing (including cleaning and segmentation, etc.) of the large corpus. After the text generation model is trained, it can basically generate text conforming to the specified syllable, and the target generated text generated based on the text generation model is relatively smooth and fluent.

[0097] In an exemplary embodiment, the vocabulary can be generated in the following way:

[0098] obtaining a first sample corpus ;

[0099] performing word segmentation based on syllables on the first sample corpus , to obtain a sample syllable unit;

[0100] iteratively counting the frequency of adjacent sample syllable unit pairs, and selecting the adjacent sample syllable unit pair combination with the highest frequency as a word;

[0101] when the number of words reaches an expected number or the frequency of the next adjacent sample syllable unit pair to be counted is 1, the word is added to the vocabulary.

[0102] In the embodiment of the present application, inspired by the existing character-based BPE algorithm, the present application designs a BPE algorithm in word syllable units, the specific process is: 1. First, a method for syllable segmentation of English words is designed based on language rules, for example, for the word wonder, after segmentation, it is divided into two syllable units won and der; 2. Determine the desired vocabulary size (set to 20,000), then segment the words of the sample corpus (including large corpus and lyrics corpus, etc.) into syllable units according to step 1, and then use these syllable units as the initial vocabulary; 3. Count the frequency of adjacent unit pairs in the corpus, and select the highest frequency unit pair to merge into a new Subword unit; 4. Repeat step 3 until the Subword vocabulary size set in step 2 is reached or the next highest frequency is 1. The role of the BPE algorithm: 1. The model can learn the syllable information of the word as a basis; 2. It greatly alleviates the OOV problem; 3. Reduces the number of model parameters and improves the prediction speed of the model.

[0103] In an exemplary embodiment, after the text generation model is trained using the sample syllable units and the format identifiers of the sample syllable units to obtain a trained text generation model, the method can further include the following steps:

[0104] determining a target topic word, retrieving a text segment associated with the target topic word;

[0105] training the text generation model using the text segment as a second sample corpus.

[0106] Wherein, the target topic word refers to a keyword related to the target generated text generated by the text generation model. For example, if the target generated text generated by the text generation model is closer to children, some keywords related to children can be obtained as target topic words, such as

toy

good

naughty

naughty

lovely

live

[0107] As a specific example, in order to make the topic of the generated text closer to children, the model is further fine-tuned based on some children's reading data. The specific method of processing children's reading data is: first, extract high-frequency topic words (about 700) from the crawled children's lyrics, then retrieve text segments (the window number is set to 200) based on these topic words on children's reading (CBT) to produce about 600,000 segments. After using the text segments to continue training the text generation model, it is found that the topic of the text generated by the text generation model is more related to children, and the text generation quality of the text generation model is improved.

[0108] In an exemplary embodiment, after the text generation model is trained by using the sample syllable units and the format identifiers of the sample syllable units, the method can further include the following steps:

[0109] obtaining a third sample corpus as a positive sample and a fourth sample corpus as a negative sample;

[0110] inputting the third sample corpus and the fourth sample corpus into a pre-trained classification model, and determining a fifth sample corpus from the fourth sample corpus;

[0111] training the text generation model by using the third sample corpus and the fifth sample corpus.

[0112] As a specific example, in order to make the generated text meet the characteristics of the lyrics, fine-tuning is finally needed on high-quality children's songs. However, there are only more than 600 high-quality children's songs crawled, which is not enough for the generation model. To solve the above problem, the embodiment of the present application pre-trains a classification model. For example, it is assumed that the 600 songs (third sample corpus) are positive samples, and the fourth sample corpus is extracted from 300,000 general English songs by using rules, and then the classification model is used to predict the similar children's songs from the general English songs based on the positive sample English songs, and the fifth sample corpus is extracted. Then, the text generation model is trained by using the third sample corpus and the fifth sample corpus, and the syllable-controllable English children's song lyrics generation model, i.e., the text generation model, is finally obtained.

[0113] The main work and innovation points of the embodiment of the present application are as follows:

[0114] (1) Model design: In order to realize syllable controllability and more effective modeling of words, a syllable-based BPE segmentation algorithm is designed according to the characteristics of English words (including complex forms, multi-syllable words, etc.). On this basis, special characters (t1, t0) for syllables are designed to enhance the model's control ability for syllables, as shown in figure 2 .

[0115] (2) Model training: Referring to figure 3, which shows the main process of text generation model training, including three stages: general large corpus pre-training, related corpus fine-tuning and target corpus fine-tuning. Specifically, in order to improve the quality of text generation (mainly referring to fluency and logicality, etc.), the general large corpus pre-training process is designed; since the number of English children's song lyrics is small, in order to make the generated lyrics as possible as children's style, the related corpus fine-tuning process is designed; in order to alleviate the defect of the serious shortage of the final training sample (English children's song lyrics), the method of expanding samples is designed, and finally the target model is obtained based on the fine-tuning of the expanded target corpus.

[0116] The embodiment of the present application trains the text generation model through the collection and processing of general large corpus, and the three stages of related corpus fine-tuning and target corpus fine-tuning, so that the trained text generation model can generate English children's song lyrics in accordance with the specified syllable format. It can be understood that, since the text generation model can automatically change the words of English children's songs without changing the tune, a large number of new songs are generated, the user experience of players is improved, and the workload of planners is greatly reduced. In addition, the text generation model is relatively simple, the prediction speed is fast, and the efficiency of human-computer interaction is improved.

[0117] In summary, the embodiment of the present application realizes an English song lyrics generation model, which can generate English children's song lyrics in accordance with the specified syllable format. It can automatically change the words of children's songs without changing the tune, thereby generating a large number of new songs and greatly reducing the workload of planners. Therefore, the present method provides a method for generating children's song lyrics with controllable syllables based on Transformer and special identifiers. The main innovations of the method include: 1) designing special identifiers to achieve syllable control; 2) designing a BPE algorithm based on syllables, which greatly alleviates the OOV problem and greatly reduces the number of model parameters, thereby improving the prediction speed of the model; 3) designing a multi-stage training process to make the generated text (here referring to English lyrics) meet the syllable control and conform to the children's style.

[0118] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present application is not limited by the described action sequence, because according to the embodiment of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the involved actions are not necessarily required by the embodiment of the present application.

[0119] Referring to figure 4 , a structure block diagram of a text generation device provided in the embodiment of the present application is shown, which can specifically include the following modules:

[0120] The text acquisition module 401 is configured to acquire a to-be-processed text having a format requirement, wherein the format requirement at least includes a syllable format requirement.

[0121] The syllable segmentation module 402 is configured to perform syllable-based segmentation processing on the to-be-processed text to obtain a plurality of syllable units.

[0122] The target text generation module 403 is configured to input the syllable units of the to-be-processed text into a pre-trained text generation model to obtain a target generated text meeting the format requirement.

[0123] In an example embodiment, the target text generation module 403 is configured to input the syllable units of the to-be-processed text into a pre-trained text generation model, so that the text generation model outputs a target generated text meeting the format requirement based on a format identifier determined by the text generation model.

[0124] In an example embodiment, the format requirement further includes a text format requirement, and the format identifier at least includes a format identifier for representing syllable types and prosodic information, a format identifier for representing internal sentence position information, a format identifier for representing sentence position information, a format identifier for representing global position information, and a format identifier for representing syllable format.

[0125] In an example embodiment, the apparatus further includes a first text generation model training module configured to acquire each sample syllable unit in the vocabulary and a format identifier of the each sample syllable unit; and train the text generation model by using the each sample syllable unit and the format identifier of the each sample syllable unit to obtain a trained text generation model.

[0126] In an example embodiment, the apparatus further includes a vocabulary generation module configured to acquire a first sample corpus corpus perform syllable-based segmentation processing on the first sample figure 5 to obtain sample syllable units; iteratively count frequencies of adjacent sample syllable unit pairs, and select a combination of adjacent sample syllable units with the highest frequency as a word; and when the number of words reaches an expected number or the frequency of the adjacent sample syllable unit pair counted next time is 1, add the word to the vocabulary.

[0127] In an example embodiment, the apparatus further includes a second text generation model training module configured to determine a target theme word, retrieve a text segment associated with the target theme word, and train the text generation model by using the text segment as a second sample corpus.

[0128] In an example embodiment, the device further comprises a third text generation model training module configured to obtain a third sample corpus as a positive sample and a fourth sample corpus as a negative sample; input the third sample corpus and the fourth sample corpus into a pre-trained classification model to determine a fifth sample corpus from the fourth sample corpus; and train the text generation model using the third sample corpus and the fifth sample corpus.

[0129] In an example embodiment, the text to be processed comprises at least English lyrics; and the target theme comprises at least children, and the text to be processed comprises at least English nursery rhyme lyrics.

[0130] In summary, in the example embodiment of the present application, the text to be processed with format requirements is obtained, wherein the format requirements comprise at least syllable format, and after the text to be processed is segmented based on syllables to obtain a plurality of syllable units, the syllable units of the text to be processed are input into a pre-trained text generation model to obtain target generated text meeting the format requirements. The example embodiment of the present application can generate target generated text meeting the format requirements comprising syllable format, syllable controllability is achieved, and the quality of the target generated text is improved.

[0131] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are described in the part of the method embodiment.

[0132] In addition, the example embodiment of the present application further provides an electronic device, such as figure 6 As shown in the figure, the electronic device comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete mutual communication through the communication bus 504,

[0133] The memory 503 is configured to store a computer program.

[0134] The processor 501 is configured to execute the program stored in the memory 503 to implement the text generation method described in the above example embodiment, and specifically comprises the following steps:

[0135] Obtain text to be processed with format requirements; the format requirements comprise at least syllable format requirements;

[0136] Segment the text to be processed based on syllables to obtain a plurality of syllable units;

[0137] Input the syllable units of the text to be processed into a pre-trained text generation model to obtain target generated text meeting the format requirements.

[0138] Optionally, the syllable unit of the text to be processed is input into a pre-trained text generation model to obtain the target generated text meeting the format requirement.

[0139] The syllable unit of the text to be processed is input into a pre-trained text generation model, so that the text generation model outputs the target generated text meeting the format requirement based on the format identifier after determining the format identifier of the text to be processed.

[0140] Optionally, the format requirement further includes a text format requirement, and the format identifier at least includes a format identifier for representing syllable type and prosody information, a format identifier for representing internal sentence position information, a format identifier for representing sentence position information, a format identifier for representing global position information, and a format identifier for representing syllable format.

[0141] Optionally, the text generation model is trained in the following manner:

[0142] The format identifier of each sample syllable unit in the vocabulary is obtained.

[0143] The text generation model is trained using the sample syllable units and the format identifiers of the sample syllable units to obtain a trained text generation model.

[0144] Optionally, the vocabulary is generated in the following manner:

[0145] A first sample corpus is obtained.

[0146] The first sample corpus is processed based on syllables to obtain sample syllable units.

[0147] The frequency of adjacent sample syllable unit pairs is iteratively counted, and the adjacent sample syllable unit pair combination with the highest frequency is selected as a word.

[0148] When the number of words reaches an expected number or the frequency of the next adjacent sample syllable unit pair is 1, the word is added to the vocabulary.

[0149] Optionally, after the text generation model is trained using the sample syllable units and the format identifiers of the sample syllable units to obtain a trained text generation model, the method further includes:

[0150] A target theme word is determined, and a text segment associated with the target theme word is retrieved.

[0151] The text segment is used as a second sample corpus to train the text generation model.

[0152] Optionally, after the text generation model is trained by using the sample syllable units and the format identifiers of the sample syllable units, the method further comprises:

[0153] obtaining a third sample corpus as a positive sample and a fourth sample corpus as a negative sample;

[0154] inputting the third sample corpus and the fourth sample corpus into a pre-trained classification model, and determining a fifth sample corpus from the fourth sample corpus;

[0155] training the text generation model by using the third sample corpus and the fifth sample corpus.

[0156] Optionally, the text to be processed at least includes English lyrics.

[0157] The target theme at least includes children, and the text to be processed at least includes English nursery rhyme lyrics.

[0158] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0159] The communication interface is used for communication between the terminal and other devices.

[0160] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the processor.

[0161] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0162] As shown in FIG. 6, in another embodiment provided by the present application, a computer readable storage medium 601 is also provided, and the computer readable storage medium 601 stores instructions, which, when executed on a computer, cause the computer to perform the text generation method described in the above embodiments. ​

[0163] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, cause the computer to perform the text generation method described in the above embodiments.

[0164] In the above embodiments, the implementation can be achieved entirely or partially by software, hardware, firmware or any combination thereof. When implemented by software, the implementation can be achieved entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is entirely or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0165] ​It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0167] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A text generation method characterized by, The method comprises the following steps: acquiring a to-be-processed text with format requirements; the format requirements at least include syllable format requirements; the to-be-processed text at least includes English lyrics; the target theme at least includes children, and the to-be-processed text at least includes English nursery rhyme lyrics; performing syllable-based word segmentation processing on the to-be-processed text to obtain a plurality of syllable units; inputting the syllable units of the to-be-processed text into a pre-trained text generation model to obtain target generated text meeting the format requirements; wherein the text generation model is trained in the following manner: acquiring each sample syllable unit in a vocabulary and a format identifier of the sample syllable unit; training the text generation model using the sample syllable unit and the format identifier of the sample syllable unit to obtain a trained text generation model; the vocabulary is generated in the following manner: acquiring a first sample corpus; performing syllable-based word segmentation processing on the first sample corpus to obtain sample syllable units; iteratively counting the frequency of adjacent sample syllable unit pairs, and selecting the adjacent sample syllable unit pair combination with the highest frequency as a word; when the number of words reaches an expected number or the frequency of the next counted adjacent sample syllable unit pair is 1, the word is added to the vocabulary.

2. The method of claim 1, wherein, the inputting of the syllable units of the to-be-processed text into the pre-trained text generation model to obtain the target generated text meeting the format requirements comprises: inputting the syllable units of the to-be-processed text into the pre-trained text generation model, so that the text generation model outputs the target generated text meeting the format requirements based on the format identifier after determining the format identifier of the to-be-processed text.

3. The method of claim 2, wherein, the format requirements further include text format requirements, and the format identifier at least includes a format identifier for representing syllable type and prosody information, a format identifier for representing internal sentence position information, a format identifier for representing sentence position information, a format identifier for representing global position information, and a format identifier for representing syllable format.

4. The method of claim 1, wherein, after the training of the text generation model using the sample syllable unit and the format identifier of the sample syllable unit to obtain the trained text generation model, the method further comprises: determining a target theme word and retrieving a text segment associated with the target theme word; training the text generation model using the text segment as a second sample corpus.

5. The method of claim 4, wherein, after the training of the text generation model using the sample syllable unit and the format identifier of the sample syllable unit to obtain the trained text generation model, the method further comprises: acquiring a third sample corpus as a positive sample and a fourth sample corpus as a negative sample; inputting the third sample corpus and the fourth sample corpus into a pre-trained classification model to determine a fifth sample corpus from the fourth sample corpus; training the text generation model using the third sample corpus and the fifth sample corpus.

6. A text generating apparatus characterized by comprising: The method comprises the following steps: a text acquisition module is configured to acquire a to-be-processed text with format requirements; The format requirement comprises at least a syllable format requirement; The text to be processed comprises at least English lyrics; The target subject comprises at least children, and the text to be processed comprises at least English nursery rhyme lyrics; A syllable segmentation module is configured to perform syllable-based segmentation processing on the text to be processed to obtain a plurality of syllable units; A target text generation module is configured to input the syllable units of the text to be processed into a pre-trained text generation model to obtain a target generated text conforming to the format requirement; The device further comprises: A first text generation model training module is configured to obtain each sample syllable unit in a vocabulary and a format identifier of the sample syllable unit; and train the text generation model using the sample syllable unit and the format identifier of the sample syllable unit to obtain a trained text generation model; The device further comprises: A vocabulary generation module is configured to obtain a first sample corpus; perform syllable-based segmentation processing on the first sample corpus to obtain sample syllable units; iteratively count the frequency of adjacent sample syllable unit pairs, and select the adjacent sample syllable unit pair combination with the highest frequency as a word; and when the number of words reaches an expected number or the frequency of the adjacent sample syllable unit pair counted next time is 1, add the word to the vocabulary.

7. An electronic device, comprising: The device comprises a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored on the memory to implement the method of any one of claims 1 to 5.

8. One or more computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method of any one of claims 1 to 5.

Images