Text generation method and device, computer device and computer readable storage medium

Abstract

This application provides a text generation method and related products. The method involves inputting initial text into a natural language generation model to obtain predicted target words; determining the generation probability and copy probability corresponding to the target words, where the generation probability indicates the likelihood of outputting the target word, and the copy probability indicates the likelihood of outputting an initial word corresponding to the target word in the initial text; determining the target probability of the target word based on the generation and copy probabilities, indicating that the target word is output when the target probability is less than a probability threshold, and indicating that the initial word is output when the target probability is greater than or equal to the probability threshold; outputting the target word or initial word as output text based on the target probability and probability threshold, and inputting the output text into a fine-tuning model to obtain the advertising text output by the fine-tuning model. This application can more accurately generate corresponding advertising text from initial text.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a text generation method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] With the development of internet technology, advertising has become a common business promotion method. For example, advertising can be placed on different platforms such as web pages and mobile applications to enhance brand awareness and facilitate user purchases.

[0003] Current advertising placement often involves manually setting corresponding advertising slogans or taglines and then placing them online according to a pre-determined advertising template.

[0004] However, such advertisements are rather simplistic and fail to achieve the desired promotional effect. Summary of the Invention

[0005] This application provides a text generation method, apparatus, computer device, and computer-readable storage medium, which can more accurately generate corresponding advertising text based on initial text.

[0006] In a first aspect, embodiments of this application provide a text generation method, the method comprising:

[0007] The initial text is input into the natural language generation model to obtain the target words predicted by the natural language generation model;

[0008] The generation probability corresponding to the target word is determined, and the copy probability of the target word is determined based on the generation probability. The generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of outputting the initial word in the initial text corresponding to the target word.

[0009] The target probability of the target word is determined based on the generation probability and the replication probability. When the target probability is less than the probability threshold, the target word is output. When the target probability is greater than or equal to the probability threshold, the initial word is output.

[0010] Based on the target probability and the probability threshold, the target word or the initial word is output as output text, and the output text and other words that are input text are input into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0011] Secondly, embodiments of this application provide a text generation apparatus, the apparatus comprising:

[0012] The input module is used to input the initial text into the natural language generation model and obtain the target words predicted by the natural language generation model;

[0013] The first determining module is used to determine the generation probability corresponding to the target word, and to determine the copy probability of the target word based on the generation probability. The generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of outputting the initial word in the initial text corresponding to the target word.

[0014] The second determining module is used to determine the target probability of the target word based on the generation probability and the replication probability, and to indicate the output of the target word when the target probability is less than a probability threshold, and to indicate the output of the initial word when the target probability is greater than or equal to the probability threshold.

[0015] The text generation module is used to output the target word or the initial word as output text based on the target probability and the probability threshold, and input the output text and other words as input text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0016] Thirdly, embodiments of this application also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor calls the computer program in the memory, it implements any of the text generation methods provided in embodiments of this application.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the text generation methods provided in embodiments of this application.

[0018] Fifthly, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement any of the text generation methods provided in the embodiments of this invention.

[0019] From the above, it can be concluded that the embodiments of this application have the following beneficial effects:

[0020] In this embodiment, the computer device inputs initial text into a natural language generation model to obtain the target word predicted by the model. It then determines the generation probability of the target word and, based on the generation probability, determines the copy probability of the target word. The generation probability indicates the likelihood of outputting the target word, while the copy probability indicates the likelihood of outputting the initial word corresponding to the target word in the initial text. A target probability is determined based on the generation and copy probabilities. If the target probability is less than a probability threshold, the target word is output; if the target probability is greater than or equal to the probability threshold, the initial word is output. Based on the target probability and the probability threshold, the target word or initial word is output as the output text. This output text, along with other words used as input text, is then input into a fine-tuning model to obtain the advertising text output by the fine-tuning model. In this embodiment, the target probability of the target word is determined by combining its copy and generation probabilities, thereby more accurately evaluating the probability of outputting the target word and ultimately improving the accuracy of the generated advertising text. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a first flowchart illustrating the text generation method provided in the embodiments of this application;

[0023] Figure 2 This is a first structural diagram of the natural language model in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the second structure of the natural language model in the proposed embodiment;

[0025] Figure 4 This is a schematic diagram illustrating the generation of advertising text through a natural language generation model and a fine-tuning model in the embodiments of the application;

[0026] Figure 5 This is a schematic diagram of the second process of the text generation method provided in the embodiments of this application;

[0027] Figure 6 This is a schematic diagram of the structure of the text generation device provided in the embodiments of this application;

[0028] Figure 7 A schematic diagram of the structure of the computer device involved in the embodiments of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features, rather than being used to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0031] With the development of internet technology, advertising has become a common business promotion method. For example, advertising can be placed on different platforms such as web pages and mobile applications to enhance brand awareness and facilitate user purchases.

[0032] Current advertising placement often involves manually setting corresponding advertising slogans or taglines and then placing them online according to a pre-determined advertising template.

[0033] However, such advertisements are rather simplistic and fail to achieve the desired promotional effect.

[0034] To address the aforementioned technical problems, embodiments of this application provide a text generation method, apparatus, computer device, and computer-readable storage medium. These methods are capable of generating corresponding advertising text based on product text.

[0035] The following is a detailed description in conjunction with the accompanying drawings. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the drawings.

[0036] It should be noted that, in the embodiments of this application, the computer device can act as the execution subject to perform the steps corresponding to the following embodiments and process data. The following description will omit and simplify the description of the execution subject.

[0037] Please see Figure 1 , Figure 1 This is a first flowchart illustrating the text generation method provided in this application embodiment. The text generation method may include the following steps:

[0038] 110. Input the initial text into the natural language generation model to obtain the target words predicted by the natural language generation model.

[0039] In some implementations, the natural language generation model can be a pre-trained model, such as a pre-trained T5 (Transfer Text-to-Text Transformer) model or an mT5 (Multilingual Transfer Text-to-Text Transformer) model. The mT5 model can be seen as an upgraded version of the T5 model. For example, the mT5 model is trained using training samples in multiple languages, and the mT5 model can process text in multiple languages, including Chinese, English, etc.

[0040] Natural language generation models can be a foundational model for text prediction, enabling the prediction of target words based on the initial input text.

[0041] In some implementations, the natural language generation model includes an encoder and a decoder. Initial text is input to the encoder, which then inputs its output to the decoder. The decoder outputs the final prediction result corresponding to the initial text. This prediction result may include predicted words or predicted sentences.

[0042] Please refer to the following: Figure 2 , Figure 2 This is a first structural schematic diagram of the natural language generation model provided in the embodiments of this application.

[0043] like Figure 2 As shown, the natural language model includes multiple encoders and multiple decoders. The initial text is input into the first encoder, which processes the initial text and then inputs the output into the next encoder. The next encoder processes the input from the previous encoder and then outputs the corresponding result. This process continues until the last encoder outputs the corresponding result.

[0044] The output of the last encoder is input into each encoder. The previous encoder processes the result of the decoder input, and then the output of the previous encoder is input into the next decoder, until the last decoder outputs the final result, which can be regarded as the prediction result of the natural language generation model.

[0045] For example, if the product description text is used as the initial text input into a natural language generation model, the natural language generation model can output the corresponding advertising text.

[0046] In some implementations, the base model for the natural language generation model used in this application can be the MT5 model. However, the tokenizer built into the MT5 model is the SentencePiece tokenizer. The SentencePiece tokenizer forcibly converts certain full-width characters into half-width characters, such as changing × to *, - to -, and \ to 、. In other words, the SentencePiece tokenizer may misidentify characters. Furthermore, the SentencePiece tokenizer has weak segmentation capabilities for Chinese text, leading to inaccurate predictions for Chinese text.

[0047] Therefore, this application replaces the built-in SentencePiece segmenter of the MT5 model with the segmenter of the BERT model. Since the BERT model's vocabulary (vocab.txt) is incomplete, omitting common punctuation marks and Chinese characters, this application adds jieba segmentation functionality to the BERT model's segmenter and expands the vocabulary, for example, by using 50,000 high-frequency words, thus covering commonly used Chinese characters and words. This expanded vocabulary can be applied to the MT5 model used in this application.

[0048] In this embodiment, when the base model for the natural language generation model is the MT5 model, the MT5 model is improved in the manner described above, making it more suitable for predicting the output text corresponding to the advertising text from the initial text in this application. This makes it more suitable for advertising generation scenarios.

[0049] 120. Determine the generation probability of the target word, and determine the copying probability of the target word based on the generation probability.

[0050] In some use cases, specific words are needed during ad generation, such as brand names, platform names, and product names. If a pre-trained MT5 model is used directly as the natural language generation model, there will be a discrepancy between the distribution of the corpus to be generated and the distribution of the corpus in the pre-trained model. This will ultimately lead to a difference between the model's output and the expected result. For example, if the name of a product cannot be predicted by the pre-trained model, the final output text will not contain the product name, thus failing to form the required ad text.

[0051] To address this technical problem, this application employs a copy mechanism in the natural language generation model, ensuring that the output of the natural language generation model matches the expected result.

[0052] Please refer to the following: Figure 3 , Figure 3 This is a schematic diagram of the second structure of the natural language generation model provided in the embodiments of this application.

[0053] The left side represents the natural language generation model (NLP), which predicts different target words after processing the initial text. The right side represents the copy probability calculation module for the target words. When the initial text is input into the NLP, the generation probability calculation module obtains vectors from the initial text processing and calculates the generation probability of the target word based on these vectors. The copy probability calculation module calculates the copy probability of the target word. The generation probability indicates the likelihood of outputting the target word, while the copy probability indicates the likelihood of the corresponding initial word appearing in the output initial text.

[0054] It should be noted that, Figure 3 The generation probability calculation module and the copy probability calculation module shown can be part of a natural language generation model. For ease of understanding, Figure 3 It was shown in the middle.

[0055] In some implementations, determining the generation probability corresponding to the target word includes:

[0056] At a preset time, acquire the context vector corresponding to the target word, the input vector of the decoder, and the state vector of the decoder.

[0057] The generation probability is determined based on the context vector, input vector, and state vector.

[0058] For example, during the natural language generation model's processing of the initial text, at a predetermined time, the computer device can acquire the context vector of the target word, as well as the decoder's input vector and the decoder's state vector. The context vector, input vector, and state vector are generated by the natural language generation model during the processing of the initial text.

[0059] In some implementations, the generation probability is determined based on the context vector, input vector, and state vector, including:

[0060] Input the context vector, input vector, and state vector into the generation probability calculation formula, which is as follows:

[0061]

[0062] Among them, P gen Let σ be the generation probability, σ be the sigmoid function, and w be the generation probability. h* w is the first parameter s w is the second learning parameter. x b is the third learning parameter.Ptr The fourth learning parameter is T, where T is time. As a context vector, x t Let s be the input vector. t This is the state vector.

[0063] It should be noted that the first, second, third, and fourth learning parameters can be manually set. For example, these parameters can be adjusted based on the actual generation probability of the word, allowing the generation probability calculation formula to accurately calculate the word's generation probability based on the context vector, input vector, and state vector. The generation probability ranges from 0 to 1.

[0064] like Figure 3 As shown, the generation probability calculation module can obtain the context vector, input vector, and state vector corresponding to the target word when the natural language generation model processes the initial text, and then determine the generation probability of the target word.

[0065] In some implementations, the replication probability of the target word is determined based on the generation probability, including:

[0066] Determine the attention distribution of the target word in the initial text at a preset time;

[0067] The replication probability of the target word is determined based on the generation probability and attention distribution.

[0068] For example, during the process of a natural language generation model processing the initial text, the natural language generation model has a corresponding attention mechanism that can determine the corresponding attention distribution for each word.

[0069] In some implementations, the replication probability of the target word is determined based on the generation probability and the attention distribution, including:

[0070] The generation probability and attention distribution are input into the replication probability calculation formula, which is as follows:

[0071]

[0072] Among them, P cop Let P be the replication probability. gen Let w be the generation probability, w be the target word, and t be the preset time. This represents the distribution of attention.

[0073] In some implementations, if the target word does not appear in the initial text, then the formula above... If the value is zero, the replication probability will ultimately be zero. In other words, this replication probability calculation formula can accurately calculate the replication probability of the target word. It should be noted that the replication probability ranges from 0 to 1. The replication probability calculation module can calculate the replication probability using the above method.

[0074] 130. Determine the target probability of the target word based on the generation probability and the duplication probability.

[0075] In existing technologies, the approach is often to set a corresponding vocabulary in the model. If the target word predicted by the model based on the initial input text does not exist in the vocabulary, the probability of the target word being output will be low, resulting in the text output by the model not containing the target word.

[0076] In practical applications, if the name of a product is not in the vocabulary corresponding to the model, then the text output by the model will not contain the name of the product.

[0077] To avoid the aforementioned technical problems, in this embodiment, the generation probability and the copying probability are calculated, and the target probability of the target word is determined jointly by the generation probability and the copying probability. When the target probability is less than a probability threshold, the target word is output; when it is greater than or equal to the probability threshold, the initial word corresponding to the target word in the initial text is output. The probability threshold can be configured by those skilled in the art according to actual needs, and no specific limitations are imposed here.

[0078] In some implementations, the target probability of a target word is determined based on its generation probability and copy probability, including:

[0079] Obtain the probability distribution of the target word in the vocabulary of the natural language generation model;

[0080] The target probability is determined based on the distribution probability, generation probability, and replication probability.

[0081] During the pre-training process of a natural language generation model, a corresponding vocabulary is introduced. The computer can obtain the probability distribution of the target word in the vocabulary of the natural language generation model. For example, at a preset time, the natural language model outputs multiple words, each with a certain probability distribution relative to the vocabulary. If the target word does not appear in the vocabulary, the probability distribution of the target word is zero.

[0082] In some implementations, the target probability is determined based on the distribution probability, generation probability, and replication probability, including:

[0083] Input the probability distribution, generation probability, and replication probability into the target probability calculation formula, which is as follows:

[0084] P w =Pgen P vocab (w)+P cop

[0085] Where w is the target word, P w Let P be the target probability. cop Let P be the replication probability. gen For the generation probability, P vocab is the probability distribution.

[0086] As can be seen from the target probability calculation formula, if the distribution probability is zero, the final target probability of the target word can be determined based on the copy probability of the target word. The computer device can then determine whether to output the target word as output text based on the target probability.

[0087] 140. Based on the target probability and probability threshold, output the target word or initial word as the output text, and input the output text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0088] Please refer to Figure 4 The fine-tuning model can be a model connected after the natural language generation model. The output of the natural language generation model is input into the fine-tuning model, which is configured to adjust the word order of the input text and output the text with the adjusted word order, or to filter the words in the input text and select some words to form a new text output.

[0089] In this embodiment, for a target word, if the determined target probability is less than the configured probability threshold, the initial word corresponding to the target word in the initial text is output. If the determined target probability is greater than or equal to the configured probability threshold, the target word is output. Thus, the output text is composed of multiple output target words and / or original words. The output text contains both the target words predicted by the natural language generation model based on the initial text and the original words in the initial text.

[0090] As shown above, after mapping the initial text to the output text through the natural language generation model, the output text is further input into the fine-tuning model, and the text output by the fine-tuning model is used as the advertising text corresponding to the initial text.

[0091] As described above, in this embodiment, the computer device inputs initial text into a natural language generation model to obtain the target word predicted by the natural language generation model; determines the generation probability corresponding to the target word, and determines the copy probability of the target word based on the generation probability. The generation probability indicates the likelihood of outputting the target word, and the copy probability indicates the likelihood of the initial word corresponding to the target word in the output initial text. The target probability of the target word is determined based on the generation probability and the copy probability. If the target probability is less than a probability threshold, the target word is output; if the target probability is greater than or equal to the probability threshold, the initial word is output. Based on the target probability and the probability threshold, the target word or the initial word is output as the output text. The output text and other words that are input text are then input into a fine-tuning model to obtain the advertising text output by the fine-tuning model. In this embodiment, the target probability of the target word is determined by the copy probability and generation probability of the target word, thereby accurately evaluating the probability of outputting the target word and more accurately determining the output text to form the advertising text corresponding to the initial text.

[0092] To gain a more detailed understanding of the text generation method provided in the embodiments of this application, please continue to participate. Figure 5 , Figure 5 This is a second flowchart illustrating the text generation method provided in this application embodiment. The text generation method may include the following steps:

[0093] 201. Divide the preset text into a first part and a second part to obtain the corresponding training samples for natural language generation.

[0094] In some implementations, the natural language generation model provided in this application can be trained from a base model. For example, the base model is the MT5 model.

[0095] In some implementations, the base model for the natural language generation model used in this application can be the MT5 model. However, the tokenizer built into the MT5 model is the SentencePiece tokenizer. The SentencePiece tokenizer forcibly converts certain full-width characters into half-width characters, such as changing × to *, - to -, and \ to 、. In other words, the SentencePiece tokenizer may misidentify characters. Furthermore, the SentencePiece tokenizer has weak segmentation capabilities for Chinese text, leading to inaccurate predictions for Chinese text.

[0096] Therefore, this application replaces the built-in SentencePiece segmenter of the MT5 model with the segmenter of the BERT model. Since the BERT model's vocabulary (vocab.txt) is incomplete, omitting common punctuation marks and Chinese characters, this application adds jieba segmentation functionality to the BERT model's segmenter and expands the vocabulary, for example, by using 50,000 high-frequency words, thus covering commonly used Chinese characters and words. This expanded vocabulary can be applied to the MT5 model used in this application.

[0097] In this embodiment, when the base model for the natural language generation model is the MT5 model, the MT5 model is improved in the manner described above, making it more suitable for predicting the output text corresponding to the advertising text from the initial text in this application. This makes it more suitable for advertising generation scenarios.

[0098] For training the base model, corresponding training parameters can be set. These parameters may include:

[0099] Total parameters: 275 million

[0100] • Training max_length = 512

[0101] ·batch_size=96

[0102] ·lr=10e-4

[0103] General corpus: 30G

[0104] • Advertising text corpus: 1.3G (3.3 million entries)

[0105] Training duration: 1 million steps with 4 P40 scans, 13 days

[0106] • train_acc: 47%

[0107] • train_loss: 2.97

[0108] In some implementations, the computer device can divide a preset text into a first part of the text and a second part of the text to obtain training samples for natural language generation; the training samples are then input into the base model corresponding to the natural language generation model for training to obtain the natural language generation model.

[0109] The preset text is divided into a first part and a second part to obtain training samples for natural language generation, including:

[0110] The first part of the text is designated as the abstract text;

[0111] Concatenate all the text in the second part of the text and determine that the concatenated text is the original text. There is a common text sequence between the summary text and the original text.

[0112] The original text and the abstract text were selected as training samples.

[0113] In actual texts formed from original text and abstracts, sentences in the original text and sentences in the abstract do not overlap. Directly using such text to train a base model is difficult, time-consuming, and the training direction is hard to control. Therefore, in this embodiment, a training sample is formed by dividing a preset text into a first part and a second part.

[0114] For example, if there are n sentences in a preset text, select n / 4 sentences (these n / 4 sentences can be non-consecutive) and use them as the first part of the text. Then, concatenate the remaining 3n / 4 sentences to form the second part of the text.

[0115] The goal is to find the longest common subsequence between the text composed of the selected n / 4 sentences and the text composed of the remaining 3n / 4 sentences. The common subsequence should be as large as possible. The common subsequence can be calculated using the Python library pylcs.

[0116] The text composed of 3n / 4 sentences is considered the original text, and the text composed of n / 4 sentences is considered the summary text. These original and summary texts form the training samples. Using these training samples to train the base model allows for faster training, and the resulting natural language generation model is better suited for tasks requiring complete sentence generation.

[0117] 202. Input the training samples into the base model corresponding to the natural language generation model for training to obtain the natural language generation model.

[0118] In some implementations, after obtaining the training samples, the training samples are input into the base model for training until the base model converges, thereby obtaining a natural language generation model.

[0119] 203. Input the initial text into the natural language generation model to obtain the target words predicted by the natural language generation model.

[0120] A natural language generation model can be a foundational model for text prediction, capable of predicting corresponding words and sentences.

[0121] After the initial text is input into the natural language generation model, the natural language generation model can predict the corresponding target word based on the input initial text.

[0122] In some implementations, the natural language generation model includes an encoder and a decoder. Initial text is input to the encoder, which then inputs its output to the decoder. The decoder outputs the final prediction result corresponding to the initial text. This prediction result may include predicted words or predicted sentences.

[0123] like Figure 2 As shown, the natural language model includes multiple encoders and multiple decoders. The initial text is input into the first encoder, which processes the initial text and then inputs the output into the next encoder. The next encoder processes the input from the previous encoder and then outputs the corresponding result. This process continues until the last encoder outputs the corresponding result.

[0124] The output of the last encoder is input into each encoder. The previous encoder processes the result of the decoder input, and then the output of the previous encoder is input into the next decoder, until the last decoder outputs the final result, which can be regarded as the prediction result of the natural language generation model.

[0125] For example, when a product description is input into a natural language generation model, the model can output corresponding text, which is used to compose advertising text.

[0126] 204. At a preset time, obtain the context vector, decoder input vector, and decoder state vector corresponding to the target word.

[0127] For example, during the natural language generation model's processing of the initial text, at a predetermined time, the computer device can acquire the context vector of the target word, as well as the decoder's input vector and the decoder's state vector. The context vector, input vector, and state vector are generated by the natural language generation model during the processing of the initial text.

[0128] 205. Determine the generation probability based on the context vector, input vector, and state vector.

[0129] In some implementations, the generation probability is determined based on the context vector, input vector, and state vector, including:

[0130] Input the context vector, input vector, and state vector into the generation probability calculation formula, which is as follows:

[0131]

[0132] Among them, P gen Let σ be the generation probability, σ be the sigmoid function, and w be the generation probability. h* w is the first parameter s w is the second learning parameter. x b is the third learning parameter. Ptr The fourth learning parameter is T, where T is time. As a context vector, x t Let s be the input vector. t This is the state vector.

[0133] It should be noted that the first, second, third, and fourth learning parameters can be manually set. For example, these parameters can be adjusted based on the actual generation probability of the word, allowing the generation probability calculation formula to accurately calculate the word's generation probability based on the context vector, input vector, and state vector. The generation probability ranges from 0 to 1 and indicates the likelihood of outputting the target word.

[0134] 206. Determine the attention distribution of the target word in the initial text at a preset time.

[0135] For example, during the process of a natural language generation model processing the initial text, the natural language generation model has a corresponding attention mechanism that can determine the corresponding attention distribution for each word.

[0136] 207. Determine the copy probability of the target word based on the generation probability and attention distribution.

[0137] In some implementations, the replication probability of the target word is determined based on the generation probability and the attention distribution, including:

[0138] The generation probability and attention distribution are input into the replication probability calculation formula, which is as follows:

[0139]

[0140] Among them, P cop Let P be the replication probability. gen Let w be the generation probability, w be the target word, and t be the preset time. This represents the distribution of attention.

[0141] In some implementations, if the target word does not appear in the initial text, then the formula above... If the term is zero, the final copy probability is zero. In other words, this copy probability calculation formula can accurately calculate the copy probability corresponding to the target word. It should be noted that the copy probability ranges from 0 to 1, indicating the probability of the target word appearing as an initial word in the output text. The copy probability calculation module can calculate the copy probability using the above method.

[0142] 208. Obtain the probability distribution of the target word in the vocabulary of the natural language generation model.

[0143] During the pre-training process of a natural language generation model, a corresponding vocabulary is introduced. The computer can obtain the probability distribution of the target word in the vocabulary of the natural language generation model. For example, at a preset time, the natural language model outputs multiple words, each with a certain probability distribution relative to the vocabulary. If the target word does not appear in the vocabulary, the probability distribution of the target word is zero.

[0144] 209. Determine the target probability based on the probability distribution, generation probability, and replication probability.

[0145] In some implementations, the target probability is determined based on the distribution probability, generation probability, and replication probability, including:

[0146] Input the probability distribution, generation probability, and replication probability into the target probability calculation formula, which is as follows:

[0147] P w =P gen P vocab (w)+P cop

[0148] Where w is the target word, P w Let P be the target probability. cop Let P be the replication probability. gen For the generation probability, P vocab is the probability distribution.

[0149] As shown in the target probability calculation formula, if the probability distribution is zero, the final target probability of the target word can be determined based on the copy probability of the target word. Computer equipment can then determine the target probability...

[0150] Whether to output the target word as output text.

[0151] 210. Based on the target probability and probability threshold, output the target word or initial word as the output text, and input the output text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0152] The fine-tuning model can be a model connected after the natural language generation model. The output of the natural language generation model is input into the fine-tuning model, which is configured to adjust the word order of the input text and output the text with the adjusted word order, or to filter the words in the input text and select some words to form a new text output.

[0153] In this embodiment, for a target word, if the determined target probability is less than the configured probability threshold, the initial word corresponding to the target word in the initial text is output. If the determined target probability is greater than or equal to the configured probability threshold, the target word is output. Thus, the output text is composed of multiple output target words and / or original words. The output text contains both the target words predicted by the natural language generation model based on the initial text and the original words in the initial text.

[0154] As shown above, after mapping the initial text to the output text through the natural language generation model, the output text is further input into the fine-tuning model, and the text output by the fine-tuning model is used as the advertising text corresponding to the initial text.

[0155] In some implementations, the output text is input into the fine-tuning model to obtain the advertising text output by the fine-tuning model, including:

[0156] Determine the attribute items corresponding to each output text;

[0157] The input text for fine-tuning the model is obtained by concatenating each attribute item and its corresponding attribute text.

[0158] Input the text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0159] For example, the output text of a natural language generation model can be used to construct the input text for fine-tuning the model using the structure [attribute item: attribute text]. Multiple attribute contents can be concatenated using the asterisk (*), and attribute items and attribute text can be concatenated using the colon (:). This allows for compatibility with missing and newly added attribute items.

[0160] For example, in real-world applications, the input text for fine-tuning the model is:

[0161] Promotion Target: Product Promotion * Primary Industry: Food * Secondary Industry: Dairy Products * E-commerce Product Name: Brand A 1.88 Meters * Primary Category: Food and Beverages * Secondary Category: Instant Dairy Products * Tertiary Category: Teen Milk Powder * Brand Name: Brand A * Information Flow Title: Brand A 1.88 Meters, more nutritious than ordinary milk powder, buy 2 bags and get a discount of 220 yuan, only 14.15 yuan per pack.

[0162] After fine-tuning the model, the output advertising text is:

[0163] Brand A milk powder special offer, huge daily discounts* Brand A 1.88 is more nutritious than ordinary milk powder. Buy 2 bags and get a discount of 220 yuan, only 14.15 yuan per pack.

[0164] As can be seen from the above embodiments, the advertising text processed by the fine-tuning model is more personalized and more suitable for promotion.

[0165] In some implementations, the following steps are included before inputting the input text into the fine-tuning model:

[0166] Determine if there are duplicate words in the input text;

[0167] If the input text contains duplicate words, the duplicate words will be removed to obtain an output text without duplicate words.

[0168] For example, the input text can be processed by using the Beam Search algorithm and a penalty mechanism to delete duplicate words or reduce the probability of words that have appeared before, so that the final input text to the fine-tuning model does not contain duplicate words.

[0169] Furthermore, by removing repetitive words, a more direct advertising effect can be achieved with the fewest words possible. For example, if the headline text of an advertisement is less than 30 words, mainly selling a certain product, and the sentence structure is relatively simple, the above method can reduce repetitive sentences.

[0170] In some implementations, as described in this application, advertising text is generated using a natural language generation model and a fine-tuning model. For example, the following is Example 1:

[0171] Input: Promotion Goal: App Promotion * Primary Industry: Daily Necessities * Secondary Industry: OTA Platform * Product Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Company Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Original Brand Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Button Text: Download Now * News Feed Title: Backpack, Camera, Destination, Next Stop, Brand B Takes You On Your Way

[0172] Prediction: Brand B Travel, book hotels, flights, and train tickets* Brand B Travel app, backpack, camera, destination, next stop, Brand B takes you on your journey.

[0173] For example, here is example 2:

[0174] Input: Promotion Target: App Promotion * Primary Industry: Daily Necessities * Secondary Industry: OTA Platform * Product Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Company Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Original Brand Name: Brand B Travel - Book Hotels, Flights, and Train Tickets * Button Text: Download Now

[0175] Prediction: Book your hotel at Brand B for unbeatable deals! *Book your hotel at unbeatable deals! Millions of real guest reviews for your reference, book with confidence! Stay with peace of mind and have a great time!

[0176] In some application scenarios, natural language generation models can generate fluent search ad titles even for products not seen in the training set, demonstrating excellent transfer learning capabilities. For example, see Example 3 below:

[0177] Input: Promotion Target: Sales Leads * Primary Industry: Automobile * Secondary Industry: Automobile Manufacturers - Domestic * E-commerce Product Name: Brand C Smart SUV * Primary Category: Brand C Automobile * Secondary Category: Mid-to-Large SUV * Third Category: Brand C Automobile * News Feed Title: Brand C Super Fast Charging Smart SUV * News Feed Description: Equipped with 800V super fast charging, truly achieving a 5-minute charge for a range of 200km+.

[0178] Prediction 1: Buy a car from brand C – rich features, high cost-performance ratio! Plus, enjoy a lifetime warranty on the entire vehicle. *This brand C car is a highly recommended choice – a high-quality, reliable option with near-perfect appearance and performance.

[0179] Prediction 2: If you're looking to buy a mid-to-large SUV, consider this brand C car. Long range and effortless driving* The brand C car offers long range with zero range anxiety, guaranteed quality for peace of mind, stable performance, and easy operation.

[0180] As can be seen from the above examples, in practical application scenarios, the natural language generation model and fine-tuning model provided in this application can output corresponding advertising text based on the initial text of the input product, thereby improving the efficiency of advertising generation. At the same time, the generated advertisements are more personalized, which can enhance the richness of the advertisements.

[0181] In this embodiment, the computer device divides a preset text into a first part and a second part to obtain training samples for natural language generation. The training samples are input into the base model corresponding to the natural language generation model for training, resulting in the natural language generation model. The initial text is input into the natural language generation model to obtain the target word predicted by the model. At a preset time, the context vector, decoder input vector, and decoder state vector corresponding to the target word are acquired. The generation probability is determined based on the context vector, input vector, and state vector. At a preset time, the attention distribution corresponding to the target word in the initial text is determined. The copy probability of the target word is determined based on the generation probability and attention distribution. The distribution probability of the target word in the vocabulary of the natural language generation model is obtained. The target probability is determined based on the distribution probability, generation probability, and copy probability. Based on the target probability and a probability threshold, the target word or the initial word is output as output text, and the output text is input into a fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0182] In this embodiment, the target probability of the target word is determined by combining the copy probability and the generation probability of the target word, thereby accurately evaluating the output probability of the target word and more accurately determining the output text to form the advertising text corresponding to the initial text.

[0183] To better implement the text generation method in this application embodiment, based on the text generation method, this application embodiment also provides a text generation device, which can be integrated into computer equipment, such as servers or terminals.

[0184] Please continue reading. Figure 6 , Figure 6 The text generation apparatus provided in this application embodiment may include:

[0185] The input module 310 is used to input the initial text into the natural language generation model to obtain the target word predicted by the natural language generation model;

[0186] The first determining module 320 is used to determine the generation probability corresponding to the target word and determine the copy probability of the target word based on the generation probability. The generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of the initial word corresponding to the target word in the output initial text.

[0187] The second determining module 330 is used to determine the target probability of the target word based on the generation probability and the replication probability. When the target probability is less than the probability threshold, the target word is output. When the target probability is greater than or equal to the probability threshold, the initial word is output.

[0188] The text generation module 340 is used to output target words or initial words as output text based on target probabilities and probability thresholds, and input the output text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0189] In some implementations, the input module 310 is further configured to divide the preset text into a first part of the text and a second part of the text to obtain training samples corresponding to natural language generation.

[0190] The training samples are input into the base model corresponding to the natural language generation model for training, and the natural language generation model is obtained.

[0191] In some implementations, the input module 310 is further configured to determine the first portion of the text as a summary text;

[0192] Concatenate all the text in the second part of the text and determine that the concatenated text is the original text. There is a common text sequence between the summary text and the original text.

[0193] The original text and the abstract text were selected as training samples.

[0194] In some implementations, the first determining module 320 is further configured to acquire the context vector corresponding to the target word, the input vector of the decoder, and the state vector of the decoder at a preset time.

[0195] The generation probability is determined based on the context vector, input vector, and state vector.

[0196] In some implementations, the first determining module 320 is further configured to input the context vector, input vector, and state vector into the generation probability calculation formula, which is as follows:

[0197]

[0198] Among them, P gen Let σ be the generation probability, σ be the sigmoid function, and w be the generation probability. h * represents the first parameter, w s w is the second learning parameter. x b is the third learning parameter. Ptr The fourth learning parameter is T, where T is time. As a context vector, x t Let s be the input vector. t This is the state vector.

[0199] In some implementations, the first determining module 320 is further configured to determine the attention distribution of the target word in the initial text at a preset time.

[0200] The replication probability of the target word is determined based on the generation probability and attention distribution.

[0201] In some implementations, the first determining module 320 is further configured to input the generation probability and attention distribution into the replication probability calculation formula, the replication probability calculation formula being as follows:

[0202]

[0203] Among them, P cop Let P be the replication probability. gen Let w be the generation probability, w be the target word, and t be the preset time. This represents the distribution of attention.

[0204] In some implementations, the second determining module 330 is further configured to obtain the distribution probability of the target word in the vocabulary of the natural language generation model;

[0205] The target probability is determined based on the distribution probability, generation probability, and replication probability.

[0206] In some implementations, the second determining module 330 is further configured to input the distribution probability, generation probability, and replication probability into the target probability calculation formula, the target probability calculation formula being as follows:

[0207] P w =P gen P vocab (w)+P cop

[0208] Where w is the target word, P w Let P be the target probability. cop Let P be the replication probability. gen For the generation probability, P vocab is the probability distribution.

[0209] In some implementations, the text generation module 340 is also used to determine the attribute items corresponding to each output text;

[0210] The input text for fine-tuning the model is obtained by concatenating each attribute item and its corresponding attribute text.

[0211] Input the text into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0212] In some implementations, the text generation module 340 is also used to determine whether there are duplicate words in the input text;

[0213] If there are duplicate words in the input text, the duplicate words will be removed to obtain the input text without duplicate words.

[0214] For details of each of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0215] Therefore, the computer device of this embodiment can bring the following technical effects:

[0216] In this embodiment, the input module 310 inputs the initial text into the natural language generation model to obtain the target word predicted by the natural language generation model; the first determining module 320 determines the generation probability corresponding to the target word and determines the copy probability of the target word based on the generation probability, wherein the generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of outputting the initial word corresponding to the target word in the initial text; the second determining module 330 determines the target probability of the target word based on the generation probability and the copy probability, indicating that the target word is output when the target probability is less than the probability threshold, and indicating that the initial word is output when the target probability is greater than or equal to the probability threshold; the text generation module 340 outputs the target word or the initial word as the output text based on the target probability and the probability threshold, and inputs the output text and other words that are input text into the fine-tuning model to obtain the advertising text output by the fine-tuning model. In this embodiment, the target probability of the target word is determined by the copy probability and the generation probability of the target word, thereby accurately evaluating the probability of the target word output, and thus more accurately determining the output text to form the advertising text corresponding to the initial text.

[0217] Furthermore, to better implement the text generation method in the embodiments of this application, based on the text generation method, the embodiments of this application also provide a computer device, such as... Figure 7 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0218] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 7 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0219] The processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0220] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, video playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0221] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0222] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0223] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0224] The initial text is input into the natural language generation model to obtain the target words predicted by the natural language generation model;

[0225] Determine the generation probability of the target word, and determine the copy probability of the target word based on the generation probability. The generation probability is used to indicate the likelihood of outputting the target word, and the copy probability is used to indicate the likelihood of the initial word corresponding to the target word in the output initial text.

[0226] The target probability of the target word is determined based on the generation probability and the replication probability. When the target probability is less than the probability threshold, the target word is output. When the target probability is greater than or equal to the probability threshold, the initial word is output.

[0227] Based on the target probability and probability threshold, the target word or initial word is output as the output text, and the output text is input into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0228] The specific operations performed by each of the above modules can be found in the previous embodiments, and will not be repeated here.

[0229] Therefore, the computer device of this embodiment can bring the following technical effects:

[0230] In this embodiment, the initial text is input into a natural language generation model to obtain the target word predicted by the model. The generation probability of the target word is determined, and its replication probability is determined based on the generation probability. The generation probability indicates the likelihood of outputting the target word, and the replication probability indicates the likelihood of the initial word corresponding to the target word in the output initial text. The target probability of the target word is determined based on the generation and replication probabilities. If the target probability is less than a probability threshold, the target word is output; if the target probability is greater than or equal to the probability threshold, the initial word is output. Based on the target probability and the probability threshold, the target word or initial word is output as the output text. This output text, along with other words used as input text, is input into a fine-tuning model to obtain the advertising text output by the fine-tuning model. In this embodiment, the target probability of the target word is determined by combining its replication and generation probabilities, thereby accurately evaluating the probability of the target word output and more accurately determining the output text to form the advertising text corresponding to the initial text.

[0231] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0232] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the text generation methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0233] The initial text is input into the natural language generation model to obtain the target words predicted by the natural language generation model;

[0234] Determine the generation probability of the target word, and determine the copy probability of the target word based on the generation probability. The generation probability is used to indicate the likelihood of outputting the target word, and the copy probability is used to indicate the likelihood of the initial word corresponding to the target word in the output initial text.

[0235] The target probability of the target word is determined based on the generation probability and the replication probability. When the target probability is less than the probability threshold, the target word is output. When the target probability is greater than or equal to the probability threshold, the initial word is output.

[0236] Based on the target probability and probability threshold, the target word or initial word is output as the output text, and the output text is input into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

[0237] For details of each of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0238] As can be seen, a computer program can be loaded by a processor to execute the steps in any of the text generation methods provided in the embodiments of this application. Therefore, the computer-readable storage medium of the embodiments of this application can bring the following technical effects:

[0239] In this embodiment, the initial text is input into a natural language generation model to obtain the target word predicted by the model. The generation probability of the target word is determined, and its replication probability is determined based on the generation probability. The generation probability indicates the likelihood of outputting the target word, and the replication probability indicates the likelihood of the initial word corresponding to the target word in the output initial text. The target probability of the target word is determined based on the generation and replication probabilities. If the target probability is less than a probability threshold, the target word is output; if the target probability is greater than or equal to the probability threshold, the initial word is output. Based on the target probability and the probability threshold, the target word or initial word is output as the output text. This output text, along with other words used as input text, is input into a fine-tuning model to obtain the advertising text output by the fine-tuning model. In this embodiment, the target probability of the target word is determined by combining its replication and generation probabilities, thereby accurately evaluating the probability of the target word output and more accurately determining the output text to form the advertising text corresponding to the initial text.

[0240] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0241] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0242] According to the text generation method of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0243] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and beneficial effects of the text generation apparatus, computer-readable storage medium, computer equipment and their corresponding units described above can be referred to the description of the text generation method in the above embodiments, and will not be repeated here.

[0244] The foregoing has provided a detailed description of a text generation method, apparatus, computer device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A text generation method characterized by, include: The initial text is input into the natural language generation model to obtain the target words predicted by the natural language generation model; The generation probability corresponding to the target word is determined, and the copy probability of the target word is determined based on the generation probability. The generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of outputting the initial word in the initial text corresponding to the target word. The target probability of the target word is determined based on the generation probability and the replication probability. When the target probability is less than the probability threshold, the target word is output. When the target probability is greater than or equal to the probability threshold, the initial word is output. Based on the target probability and the probability threshold, the target word or the initial word is output as output text, and the output text is input into the fine-tuning model to obtain the advertising text output by the fine-tuning model; Before the initial text is input into the natural language generation model and the natural language generation model predicts the corresponding target word, the method further includes: The preset text is split into sentences, and at least one sentence obtained from the split is spliced ​​together according to a specified ratio to obtain a first part of text and a second part of text. The first part of text and the second part of text satisfy a specified splicing condition, which is that the first part of text and the second part of text have the most common text sequences. The first part of the text is determined as the summary text, the second part of the text is determined as the original text, and the original text and the summary text are determined as training samples; The training samples are input into the base model corresponding to the natural language generation model for training, thereby obtaining the natural language generation model.

2. The text generation method according to claim 1, characterized in that, The natural language generation model includes a decoder, and determining the generation probability corresponding to the target word includes: At a preset time, the context vector corresponding to the target word, the input vector of the decoder, and the state vector of the decoder are acquired. The generation probability is determined based on the context vector, the input vector, and the state vector.

3. The text generation method according to claim 2, characterized in that, Determining the generation probability based on the context vector, the input vector, and the state vector includes: The context vector, the input vector, and the state vector are input into the generation probability calculation formula, which is as follows: in, For generation probability, For the sigmoid function, As the first parameter, As the second learning parameter, As the third learning parameter, The fourth learning parameter is T, where T is time. For context vectors, For the input vector, This is the state vector.

4. The text generation method according to claim 2, characterized in that, Determining the replication probability of the target word based on the generation probability includes: The attention distribution corresponding to the target word in the initial text is determined at the preset time. The replication probability of the target word is determined based on the generation probability and the attention distribution.

5. The text generation method according to claim 4, characterized in that, Determining the replication probability of the target word based on the generation probability and the attention distribution includes: The generation probability and the attention distribution are input into the replication probability calculation formula, which is as follows: in, For the replication probability, Let w be the generation probability, w be the target word, and t be the preset time. This represents the distribution of attention.

6. The text generation method according to claim 1, characterized in that, Determining the target probability of the target word based on the generation probability and the replication probability includes: Obtain the probability distribution of the target word in the vocabulary of the natural language generation model; The target probability is determined based on the distribution probability, the generation probability, and the replication probability.

7. The text generation method according to claim 6, characterized in that, Determining the target probability based on the distribution probability, the generation probability, and the replication probability includes: The distribution probability, the generation probability, and the replication probability are input into the target probability calculation formula, which is as follows: Where w is the target word, Let the target probability be... For the replication probability, For generation probability, is the probability distribution.

8. The text generation method according to claim 1, characterized in that, The step of inputting the output text into the fine-tuning model to obtain the advertising text output by the fine-tuning model includes: Determine the attribute items corresponding to each of the output texts; The input text of the fine-tuning model is obtained by concatenating each attribute item and its corresponding attribute text. The input text is fed into the fine-tuning model to obtain the advertising text output by the fine-tuning model.

9. The text generation method according to claim 8, characterized in that, Before inputting the input text into the fine-tuning model, the method further includes: Determine whether there are duplicate words in the input text; If there are repeated words in the input text, the repeated words are deleted to obtain input text without repeated words.

10. A text generation device, characterized in that, include: The input module is used to input the initial text into the natural language generation model to obtain the target word predicted by the natural language generation model; The first determining module is used to determine the generation probability corresponding to the target word, and to determine the copy probability of the target word based on the generation probability. The generation probability is used to indicate the possibility of outputting the target word, and the copy probability is used to indicate the possibility of outputting the initial word in the initial text corresponding to the target word. The second determining module is used to determine the target probability of the target word based on the generation probability and the replication probability, and to indicate the output of the target word when the target probability is less than a probability threshold, and to indicate the output of the initial word when the target probability is greater than or equal to the probability threshold. The text generation module is used to output the target word or the initial word as output text based on the target probability and the probability threshold, and input the output text and other words as input text into the fine-tuning model to obtain the advertising text output by the fine-tuning model; The device is further configured to split the preset text into sentences, and splice at least one sentence obtained from the splitting into a first part of text and a second part of text according to a specified ratio, wherein the first part of text and the second part of text satisfy a specified splicing condition, wherein the specified splicing condition is that the first part of text and the second part of text have the most common text sequences; the first part of text is determined as the summary text, the second part of text is determined as the original text, and the original text and the summary text are determined as training samples; The training samples are input into the base model corresponding to the natural language generation model for training, thereby obtaining the natural language generation model.

11. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the text generation method as described in any one of claims 1-9 when it invokes the computer program in the memory.

12. A computer-readable storage medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the text generation method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the text generation method as described in any one of claims 1 to 9.

Images