Text generation methods, apparatus, computer equipment and storage media

Abstract

This application discloses a text generation method, apparatus, computer device, and storage medium, belonging to the field of computer technology. By extracting sentence features from the prompt text on a sentence-by-sentence basis, this application can predict the target sentence features of each target sentence in the following text. These target sentence features serve as semantic guidance information for the decoding process, enabling the decoding of each target sentence on a sentence-by-sentence basis, thereby obtaining the target text. Essentially, in the process of generating the target text, relying on the guidance of each target sentence feature ensures that even if a character in a target sentence deviates semantically, this deviation will be corrected by the next target sentence feature, thus preventing it from being propagated throughout the entire target text, greatly improving the accuracy of the text generation process.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a text generation method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, long text generation technology (also known as robot writing or artificial intelligence writing) has been widely used to assist in story creation. For example, based on the prompt text given by the writer, the machine automatically generates a draft of the story, which the writer can further improve upon to reduce the writer's workload.

[0003] In the process of generating long text, the user provides a prompt text, which is then input into the Transformer Language Model. The model outputs a character sequence (e.g., a draft) of the long text corresponding to the prompt text, character by character. Because long text is predicted character by character, if a character deviates semantically at a certain point, this error will gradually propagate to characters at subsequent points, leading to a gradual accumulation of semantic deviations and low accuracy in text generation. Therefore, a method to improve the accuracy of text generation is urgently needed. Summary of the Invention

[0004] This application provides a text generation method, apparatus, computer device, and storage medium, which can improve the accuracy of text generation. The technical solution is as follows:

[0005] On the one hand, a text generation method is provided, which includes:

[0006] Based on the prompt text containing at least one prompt statement, obtain the statement features of the at least one prompt statement, the statement features being used to characterize the semantics of the prompt statement;

[0007] Based on the statement features of the at least one prompt statement, at least one target statement feature is predicted, the target statement feature is used to characterize the semantics of the target statement, and the target statement is the following text of the prompt statement;

[0008] Based on the features of the at least one target statement, target text containing at least one target statement is decoded, and the target text has a semantic correspondence with the prompt text.

[0009] On the one hand, a text generation apparatus is provided, the apparatus comprising:

[0010] The first acquisition module is used to acquire statement features of the at least one prompt statement based on the prompt text containing at least one prompt statement, wherein the statement features are used to characterize the semantics of the prompt statement;

[0011] The prediction module is used to predict at least one target statement feature based on the statement features of the at least one prompt statement, wherein the target statement feature is used to characterize the semantics of the target statement, and the target statement is the following text of the prompt statement;

[0012] The decoding module is used to decode target text containing at least one target statement based on the features of the at least one target statement, wherein the target text has a semantic correspondence with the prompt text.

[0013] In one possible implementation, the first acquisition module includes:

[0014] The first acquisition unit is used to acquire the embedding feature of at least one character in any of the at least one prompting statements;

[0015] An encoding unit is used to encode the embedding features of the at least one character to obtain the semantic features of the at least one character;

[0016] The first fusion unit is used to fuse the semantic features of the at least one character to obtain the statement features of any prompt statement.

[0017] In one possible implementation, the first acquisition unit is used to: input at least one character in any of the prompt statements to the statement embedding layer in the statement encoding model, perform embedding processing on the at least one character through the statement embedding layer, and output the embedding features of the at least one character, wherein the statement encoding model is used to encode the input statement to extract statement features;

[0018] The encoding unit is used to: input the embedding features of the at least one character into at least one encoding layer in the statement encoding model, perform weighted processing on the embedding features of the at least one character through the at least one encoding layer, and output the semantic features of the at least one character;

[0019] The first fusion unit is used to: perform mean pooling on the semantic features of the at least one character to obtain the statement features of any prompt statement.

[0020] In one possible implementation, the prediction module includes:

[0021] The second acquisition unit is used to acquire the positional features of the at least one prompting statement, wherein the positional features are used to characterize the word order information of the prompting statement in the prompting text;

[0022] The second fusion unit is used to fuse the statement features of the at least one prompt statement and the position features of the at least one prompt statement to obtain the fused features of the at least one prompt statement;

[0023] A decoding unit is used to decode the fusion features of the at least one prompt statement to obtain the at least one target statement feature.

[0024] In one possible implementation, the decoding unit is used for:

[0025] The fusion features of the at least one prompt statement are input into at least one decoding layer in the statement planning model. The at least one decoding layer decodes the fusion features of the at least one prompt statement, and the last decoding layer outputs the at least one target statement feature. The statement planning model is used to predict the target statement feature of the following statement based on the fusion features of the prompt statement.

[0026] In one possible implementation, the device further includes:

[0027] The input / output module is used to input the fusion features of the at least one sample prompt statement into the initial planning model and output at least one predicted statement feature for the sample text containing at least one sample prompt statement.

[0028] The second acquisition module is used to acquire a loss function value based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text, wherein the labeled text is the following text of the sample text;

[0029] The iterative adjustment module is used to iteratively adjust the parameters of the initial planning model until the loss function value meets the stopping condition, thereby obtaining the statement planning model.

[0030] In one possible implementation, the second acquisition module is used to:

[0031] Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein a first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature;

[0032] The loss function value is obtained based on the at least one first similarity.

[0033] In one possible implementation, the second acquisition module is used to:

[0034] Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein a first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature;

[0035] Based on the at least one predicted statement feature and the at least one negative sample statement feature, at least one second similarity is obtained, wherein a second similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding negative sample statement feature;

[0036] The loss function value is obtained based on the at least one first similarity and the at least one second similarity.

[0037] In one possible implementation, the decoding module is used for:

[0038] For any one of the at least one target statement features, decode the target statement to obtain the target statement corresponding to the target statement feature. Repeat the above operation to obtain the target text containing the at least one target statement.

[0039] In one possible implementation, the decoding module is used for:

[0040] The target sentence feature is input into at least one decoding layer in the sentence decoding model. The target sentence feature is decoded by the at least one decoding layer to output at least one target character. The at least one target character is determined as the target sentence corresponding to the target sentence feature. The sentence decoding model is used to decode the input sentence feature to convert it into each character in the corresponding sentence.

[0041] On one hand, a computer device is provided, the computer device including one or more processors and one or more memories, the one or more memories storing at least one computer program, the at least one computer program being loaded and executed by the one or more processors to implement the text generation method as described above.

[0042] On the one hand, a storage medium is provided that stores at least one computer program, which is loaded and executed by a processor to implement the text generation method described above.

[0043] On one hand, a computer program product or computer program is provided, the computer program product or computer program comprising one or more lines of program code, the one or more lines of program code being stored in a computer-readable storage medium. One or more processors of a computer device are capable of reading the one or more lines of program code from the computer-readable storage medium, and the one or more processors execute the one or more lines of program code, enabling the computer device to perform the above-described text generation method.

[0044] The beneficial effects of the technical solutions provided in this application include at least the following:

[0045] By extracting sentence features from the prompt text on a sentence-by-sentence basis, the target sentence features of each target sentence in the following text can be predicted. These target sentence features serve as semantic guidance information for the decoding process, enabling the decoding of each target sentence on a sentence-by-sentence basis, thereby obtaining the target text. In essence, during the generation of the target text, the process relies on the guidance of each target sentence feature, ensuring that even if a character in a target sentence deviates semantically, this deviation will be corrected by the next target sentence feature and will not be transmitted throughout the entire target text, greatly improving the accuracy of the text generation process. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the implementation environment of a text generation method provided in an embodiment of this application;

[0048] Figure 2 This is a flowchart of a text generation method provided in an embodiment of this application;

[0049] Figure 3 This is a flowchart of a text generation method provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram illustrating the principle of a text generation method provided in an embodiment of this application;

[0051] Figure 5 This is a flowchart illustrating the training process of a statement planning model provided in an embodiment of this application.

[0052] Figure 6 This is a schematic diagram of the structure of a text generation device provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

[0054] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0056] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0057] In this application, the term "at least one" means one or more, and "multiple" means two or more, for example, multiple first positions means two or more first positions.

[0058] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0059] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include audio processing, computer vision, natural language processing, and machine learning / deep learning.

[0060] Enabling computers to hear, see, speak, and feel represents the future direction of human-computer interaction. Natural Language Processing (NLP) is a crucial area within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus has a close connection to linguistic research. NLP techniques typically include text generation, machine translation, text preprocessing, semantic understanding, question answering, and knowledge graphs.

[0061] In the field of Natural Language Processing (NLP), text generation technology has a wide range of applications. Also known as Natural Language Generation, it studies techniques that enable computers to express and write like humans. Examples include robot writing, AI-generated writing, automatic dialogue generation, and robots writing classical Chinese poems. In short, based on key information (i.e., prompt text) and its internal machine representation, a planning process enables a machine to automatically generate a high-quality piece of natural language text (i.e., the target text). Text generation technology is a branch of artificial intelligence and computational linguistics. Corresponding language generation systems are computer models based on language information processing. Their operation is the reverse of natural language analysis; they start from an abstract conceptual level and generate target text by selecting and executing certain semantic and grammatical rules.

[0062] The technical solutions provided in this application relate to artificial intelligence text generation technology. Based on given prompt text, the machine can automatically generate semantically corresponding target text. Since the prompt text is usually short (e.g., a single sentence) while the target text is longer (e.g., multiple sentences), this process can also be called long text generation technology. Long text generation technology can be applied to various scenarios. For example, in story creation, it can automatically generate stories related to a specific topic; in news writing assistance, it can automatically generate news drafts; and in e-commerce scenarios, it can automatically generate product description documents.

[0063] In the above process, the prompt text, also known as the prompt message, refers to the input text generated from the long text; the target text, also known as the statement that continues writing from the prompt message, refers to the output text generated from the long text, and generally contains multiple statements.

[0064] When generating long text, the user provides a prompt text, which is then input into the Transformer Language Model (TLM). The TLM outputs the character sequence of the long text corresponding to the prompt text, character by character. On one hand, because long text is predicted character by character, if a character deviates semantically at a certain time, this error will gradually propagate to characters at subsequent time points, leading to an accumulation of semantic deviations. On the other hand, since the TLM samples each character in the long text from the vocabulary according to probability when predicting it character by character, it lacks global semantic guidance, which also easily leads to semantic deviations. Therefore, the accuracy of text generation is relatively low.

[0065] In view of this, embodiments of this application provide a text generation method that first generates a sequence of target sentence features based on the prompt text, and then decodes each target sentence feature sequentially to obtain each target sentence of the target text. This provides a two-stage generation model based on sentence features, which can effectively improve the quality of long text generation. Furthermore, the sentence planning model can generate each target sentence feature sequentially from left to right, and the decoder guided by the sentence features can decode the corresponding target sentence based on the target sentence features, effectively improving semantic deviation and increasing the accuracy of text generation.

[0066] Figure 1 This is a schematic diagram illustrating the implementation environment of a text generation method provided in this application. See also... Figure 1 This implementation environment includes terminal 101 and server 102, which are described in detail below:

[0067] Terminal 101 is used to provide prompt text, which typically includes one or more statements, such as a single sentence. Terminal 101 has an application that supports text generation installed and running, such as a writing robot, intelligent assistant, draft generator, text editor, writing application, online document, etc. This embodiment does not specifically limit the type of application.

[0068] In some embodiments, after a user logs into the application on terminal 101, they enter prompt text in the application, triggering terminal 101 to send a text generation request carrying the prompt text to server 102. The text generation request is used to request target text that semantically corresponds to the prompt text. Optionally, the methods for entering the prompt text include, but are not limited to, manual input, voice commands, gesture commands, and selecting preset text from shortcut options. This application embodiment does not specifically limit the input method of the prompt text.

[0069] The terminal 101 and the server 102 can be connected directly or indirectly through wired or wireless communication, and this application does not impose any restrictions on this.

[0070] Server 102 is used to provide text generation services. Server 102 may include at least one of a single server, multiple servers, a cloud computing platform, or a virtualization center. Optionally, server 102 undertakes the main computing work, and terminal 101 undertakes the secondary computing work; or, server 102 undertakes the secondary computing work, and terminal 101 undertakes the main computing work; or, terminal 101 and server 102 use a distributed computing architecture for collaborative computing.

[0071] In some embodiments, server 102 receives a text generation request from terminal 101, parses the text generation request to obtain a prompt text, obtains a target text corresponding to the semantics of the prompt text based on the text generation method provided in this application embodiment, and sends the target text to terminal 101. The target text can be used to assist users in story creation, news writing, product description, etc., and has a wide range of application scenarios.

[0072] In an exemplary scenario, taking story creation as an example, the writer logs into an application on terminal 101, inputs a prompt text into the application, and clicks the draft generation function option. Terminal 101 sends a draft generation request carrying the prompt text to server 102. This draft generation request is used to request a story draft corresponding to the semantics of the prompt text. Server 102 receives the draft generation request, parses it, obtains the prompt text, and generates the story draft based on the text generation method provided in this application embodiment. Typically, the prompt text input by the writer is a sentence, while the generated story draft is an article. Server 102 returns the story draft to terminal 101. Terminal 101 receives the story draft, and the author can view the story draft on the application and further refine it, thereby reducing the writer's workload.

[0073] In an exemplary scenario, taking an e-commerce scenario as an example, a merchant logs into an application on terminal 101, inputs product attributes (as prompt text) into the application, and clicks the description generation function option. Terminal 101 sends a description generation request carrying the product attributes to server 102. This description generation request is used to request a description document corresponding to the semantics of the product attributes. Server 102 receives the description generation request, parses it to obtain the product attributes, and generates the description document based on the text generation method provided in this application embodiment. Typically, the product attributes input by the merchant are a sentence, while the generated description document includes one or more descriptive paragraphs. Server 102 returns the description document to terminal 101. Terminal 101 receives the description document, and the merchant can view the description document on the application and further refine it to obtain the final product description details, thereby reducing the workload of the merchant.

[0074] In an exemplary scenario, taking news writing as an example, a reporter logs into an application on terminal 101, inputs a news summary into the application, and clicks the draft generation function option. Terminal 101 sends a draft generation request carrying the news summary to server 102. This draft generation request is used to request a news draft corresponding to the semantics of the news summary. Server 102 receives the draft generation request, parses it, obtains the news summary, and generates the news draft based on the text generation method provided in this application embodiment. Typically, the news summary input by the reporter is a sentence, while the generated news draft is an article. Server 102 returns the news draft to terminal 101. Terminal 101 receives the news draft, and the reporter can view the news draft on the application and further refine it, thereby reducing the reporter's workload.

[0075] It should be noted that the various prompt texts and target texts involved in the embodiments of this application can be stored on the blockchain.

[0076] The aforementioned server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0077] The aforementioned terminal 101 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, e-book reader, etc., but is not limited to these.

[0078] Those skilled in the art will understand that terminal 101 can refer to one of a plurality of terminals, and the number of such terminals can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0079] Figure 2 This is a flowchart of a text generation method provided in an embodiment of this application. See also... Figure 2This embodiment is applied to a computer device. The following description uses a server as an example of the computer device. This embodiment includes the following steps:

[0080] 201. The server obtains the statement features of the at least one prompt statement based on the prompt text containing the prompt statement, and the statement features are used to characterize the semantics of the prompt statement.

[0081] The prompt text, also known as the prompt message, refers to the input text in the long text generation process. The prompt text may include at least one prompt statement; for example, the prompt text may contain only a single prompt statement, or it may contain multiple prompt statements. This application embodiment does not specifically limit the number of prompt statements included in the prompt text.

[0082] In some embodiments, the server first obtains a prompt text containing at least one prompt statement, and then obtains the statement features of the at least one prompt statement, with one prompt statement corresponding to one statement feature. The statement feature is used to characterize the semantics of the prompt text; that is, the statement feature refers to the semantic features of the prompt text at the statement level (rather than the character level), and can reflect the semantics of each prompt statement in the prompt text at the statement level.

[0083] Optionally, if the prompt text contains a single prompt statement, then a single statement feature is extracted from the single prompt statement; if the prompt text contains multiple prompt statements, then the corresponding statement feature is extracted from each of the multiple prompt statements, that is, multiple statement features are finally obtained.

[0084] In some embodiments, when the server obtains the prompt text, the user may input the prompt text on the server side, or read any pre-stored prompt text from a local database, or download the prompt text from a cloud database, or the prompt text may be sent by the receiving terminal.

[0085] In an exemplary scenario, a user logs into an application on a terminal, enters prompt text within the application, and triggers a text generation function option. In response to this triggering action, the terminal sends a text generation request carrying the prompt text to the server. The server receives the text generation request, parses it, and obtains the prompt text. Optionally, the triggering action includes, but is not limited to, click operations, double-click operations, long-press operations, voice commands, gesture commands, etc. This application embodiment does not specifically limit the form of the triggering action.

[0086] In some embodiments, when the server obtains the statement features of any of the at least one prompt statement, which includes at least one character, it may obtain the embedding features of at least one character in the prompt statement; encode the embedding features of the at least one character to obtain the semantic features of the at least one character; and fuse the semantic features of the at least one character to obtain the statement features of the prompt statement.

[0087] In this context, a token is the smallest semantic unit in a sentence, also known as a word segment. In English, a token refers to a word (i.e., vocabulary) or an English punctuation mark. In Chinese, a token refers to a Chinese word (a word may include one or more Chinese characters) or a Chinese punctuation mark.

[0088] In the above process, by extracting the corresponding sentence features for each prompt statement in the prompt text, this sentence feature integrates the semantic features of each character in the prompt statement, so that each sentence feature corresponds one-to-one with each prompt statement. Finally, the sentence features of each prompt statement can reflect the feature information of the entire prompt text at the sentence level, on a sentence-by-sentence basis.

[0089] Optionally, when extracting the embedding feature, the server may perform word segmentation on any prompt statement to obtain at least one character, and perform embedding processing on at least one character to obtain the embedding vector of at least one character, which is also the embedding feature of at least one character.

[0090] Optionally, when acquiring the semantic feature, the process of the server encoding the embedding feature of the at least one character is also the process of weighting the embedding vector of the at least one character. The embedding vector of each character is weighted to obtain the semantic vector of each character. Repeating the above operation can obtain the semantic vector of the at least one character, which is also the semantic feature of the at least one character.

[0091] Optionally, when obtaining statement features, since it is necessary to fuse the semantic features of at least one character, the server can perform mean-pooling on the semantic features of at least one character to obtain the statement features of any prompt statement. The mean-pooling process refers to averaging the semantic features of at least one character to obtain the statement features of any prompt statement. In other words, the average value of the values ​​at the same position in the semantic features of each character is calculated to obtain the value at the same position in the semantic features. By traversing all the values ​​in the semantic features of each character, the final semantic features can be obtained.

[0092] In some embodiments, in addition to mean pooling, the server may also perform element-wise addition, element-wise multiplication, bilinear concatenation, concat, or other methods to fuse the semantic features of the at least one character. This application does not specifically limit this.

[0093] In an exemplary embodiment, for any prompt statement, the server can call a word vector model to extract the embedding features of each character, and input the embedding features of each character into a sentence encoding model (also known as a sentence encoder). The sentence encoding model encodes the embedding features of each character, outputs the semantic features of each character, and finally performs mean pooling on the semantic features of each character to obtain the sentence features of any prompt statement.

[0094] In an exemplary embodiment, for any prompt statement, the server may also invoke a statement encoding model to extract statement features. This statement encoding model encodes the input statement to extract statement features and includes a statement embedding layer and at least one encoding layer. The server can input at least one character from any prompt statement into the statement embedding layer of the statement encoding model, perform embedding processing on the at least one character, and output the embedding features of the at least one character. Further, the server inputs the embedding features of the at least one character into at least one encoding layer of the statement encoding model, performs weighted processing on the embedding features of the at least one character, and outputs the semantic features of the at least one character. Finally, the server performs mean pooling on the semantic features of the at least one character to obtain the statement features of the any prompt statement.

[0095] In some embodiments, since the at least one character in any prompt statement can constitute a character sequence, similarly, the embedding features of the at least one character can constitute an embedding feature sequence, and similarly, the semantic features of the at least one character can constitute a semantic feature sequence. Alternatively, the above process can also be described as follows: inputting the character sequence of any prompt statement into a statement encoding model; performing embedding processing on the character sequence through the statement embedding layer to output the embedding feature sequence of any prompt statement; performing weighted processing on the embedding feature sequence through the at least one encoding layer to output the semantic feature sequence of any prompt statement; and then performing mean pooling on each semantic feature in the semantic feature sequence to obtain the statement features of any prompt statement. Optionally, the at least one encoding layer is cascaded, that is, the output of the previous encoding layer is used as the input of the next encoding layer.

[0096] Optionally, the sentence encoding model can be any encoder structure, where the encoder structure refers to the module used for encoding in the S2S (Sequence-to-Sequence) model. The S2S model is a basic modeling framework for natural language generation and machine translation. The input of the S2S model is a sequence (e.g., a character sequence), and the output of the S2S model is also a sequence (e.g., a sequence of semantic features). The S2S model is usually based on an encoder-decoder machine learning model, which is the core module of the S2S framework. For example, the sentence encoding model includes, but is not limited to: BERT (Bidirectional Encoder Representation From Transformers), ERNIE (Enhanced Language Representation with Informative Entities), SpanBERT (Improving Pre-training by Representing and Predicting Spans), ELMo (Embeddings From Language Models), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Bi-LSTM (Bidirectional Long Short-Term Memory), GRU (Gated Recurrent Unit), etc. The embodiments of this application do not specifically limit the architecture of the sentence encoding model.

[0097] To illustrate, taking the BERT model as an example, assuming the prompt text contains only one prompt statement, the server performs word segmentation on the prompt statement to obtain a character sequence consisting of at least one character contained in the prompt statement. This character sequence is then input into the BERT model. The BERT model includes a statement embedding layer and at least one encoding layer. First, the character sequence is input into the statement embedding layer, which performs embedding processing on each character in the sequence, mapping each character to an embedding space to obtain an embedding vector (i.e., embedding feature) for each character, resulting in a sequence of embedding vectors. This embedding vector sequence is then input into the at least one encoding layer, which performs bidirectional encoding (including forward encoding and backward encoding) on ​​each embedding vector in the sequence, outputting a semantic vector (i.e., semantic feature) for each character, resulting in a sequence of semantic vectors. Each encoding layer in the BERT model is a bidirectional encoding layer, capable of forward and backward encoding of the input signal. The encoding layers are cascaded, meaning the output of one encoding layer serves as the input to the next. Each encoding layer consists of two parts: an attention network and a forward fully connected layer. Each hidden layer in the attention network is obtained by weighted averaging of the hidden layers in the previous layer, so that each hidden layer can be directly associated with all the hidden layers in the previous layer. Using the input character sequence, a hidden layer vector representing global information can be obtained. The forward fully connected layer is used to further process the global information obtained by the attention network to enhance the learning ability of the entire BERT model.

[0098] Taking the first encoding layer as an example, this layer includes an attention network and a forward fully connected layer. The embedded vector sequence is input into the attention network of the first encoding layer. The attention network weights the embedded vector sequence to extract an attention feature sequence. This attention feature sequence is then input into the forward fully connected layer of the first encoding layer. The forward fully connected layer performs bidirectional semantic encoding (including forward and backward encoding) on ​​this attention feature sequence, outputting a latent vector sequence. This latent vector sequence is then input into the second encoding layer, and so on. The processing logic of subsequent encoding layers is similar to that of the first encoding layer, and will not be elaborated here. Because an attention mechanism is introduced into the encoding layer, each character can focus on characters with a strong (closest) relationship to itself during each semantic encoding, resulting in higher accuracy of the final semantic features.

[0099] In some embodiments, the semantic vectors of each character output from the penultimate encoding layer can be used as the semantic features of each character; in other embodiments, the semantic vectors of each character output from the last encoding layer can also be used as the semantic features of each character; or, the semantic vectors of each character output from any encoding layer can also be used as the semantic features of each character. This application does not specifically limit this approach. For example, in a BERT model with 12 encoding layers, the semantic vectors of each character output from the 11th layer can be used as the semantic features of each character.

[0100] In the above process, the server performs forward encoding and reverse encoding on each character in the character sequence through the encoding layer. Forward encoding enables the semantic features of each character to be integrated with the relevant information of the historical characters that appeared before it, while reverse encoding enables the semantic features of each character to be integrated with the relevant information of the future characters that appear after it. The two-way encoding operation can greatly improve the expressive power of the semantic features of each character, that is, it can improve the expressive power of the sentence features.

[0101] 202. Based on the statement features of the at least one prompt statement, the server predicts at least one target statement feature, which is used to characterize the semantics of the target statement, and the target statement is the following text of the prompt statement.

[0102] In this context, the target statement is the following text of the prompt statement, meaning that the prompt statement is the preceding text of the target statement. In other words, the target statement and the prompt statement are semantically related, and the target statement is the following text generated based on the prompt statement. For example, when the prompt statement is a question, the target statement can be the answer to the question. Or, when the prompt statement is a news summary, the target statement can be the first sentence in the first paragraph of the news draft.

[0103] In some embodiments, when predicting target statement features, the server may obtain the positional features of the at least one prompt statement, which are used to characterize the word order information of the prompt statement in the prompt text; fuse the statement features of the at least one prompt statement and the positional features of the at least one prompt statement to obtain the fused features of the at least one prompt statement; and decode the fused features of the at least one prompt statement to obtain the at least one target statement feature.

[0104] In the above process, by fusing the sentence features and positional features of each prompt statement, when predicting the target sentence features from left to right, it is possible to refer not only to the sentence features of the preceding text, but also to the word order information of the preceding text, so that the predicted target sentence features conform to the semantic order and improve the prediction accuracy of the target sentence features.

[0105] In some embodiments, the server may invoke a statement planning model for decoding. This statement planning model includes at least one decoding layer, which is used to predict the target statement features of the following statement based on the fusion features of the prompt statement. That is, the server inputs the fusion features of the at least one prompt statement into the at least one decoding layer in the statement planning model, decodes the fusion features of the at least one prompt statement through the at least one decoding layer, and outputs the at least one target statement feature by the last decoding layer.

[0106] In the above process, the sentence planning model can predict at least one target sentence feature on a sentence-by-sentence basis, based on the fusion features of the prompt sentence. Each target sentence feature is used to characterize the semantics of a target sentence in the following text of the prompt text, i.e., the target text. However, at this time, the specific natural language content of each target sentence has not yet been decoded, but only the abstract space representation of each target sentence has been predicted.

[0107] Since the sentence planning model outputs at least one target sentence feature on a sentence-by-sentence basis, it can be regarded as a sentence-level language model (Language Modeling, LM, which is structurally similar to the decoder in the S2S framework). The sentence planning model can perform language modeling on the sentence feature sequence. Given a series of prompt sentence features (one or more), as the sentence feature sequence of the preceding text, the sentence planning model learns to predict the next target sentence feature containing the expression based on the sentence feature sequence of the preceding text, and outputs a series of target sentence features one by one as the target sentence feature sequence of the following text.

[0108] In some embodiments, the sentence planning model is a sentence vector planner similar to the GPT2 (Generative Pre-Training 2) model, which includes at least one decoding layer connected in series, meaning the output of one decoding layer serves as the input to the next. For example, the sentence vector planner includes 12 decoding layers based on a self-attention structure. Unlike the GPT2 model, this sentence vector planner models at the sentence level, while the GPT2 model models at the character level. Optionally, in this sentence vector planner, there is no need to set an embedding layer; instead, the fused features of each prompt sentence are directly used as the input representation. Optionally, when predicting the target sentence features, since there is no need to predict the characters of the natural language, there is no need to set an activation function for the output layer; the target sentence features predicted by the last encoding layer can be directly output.

[0109] Each decoding layer consists of two parts: a mask-based self-attention network and a feedforward neural network. The mask-based self-attention network only focuses on the features of each target statement predicted before the current time step (as well as the fusion features of the input prompt statements). Information after the current time step needs to be masked. The fusion feature sequence composed of the fusion features of each prompt statement is input into the mask-based self-attention network, which outputs a hidden vector sequence. The feedforward neural network layer is used to fully connect the hidden vector sequence output by the mask-based self-attention network.

[0110] Taking the first decoding layer as an example, this encoding layer includes a mask-based self-attention network and a feedforward neural network. The fused feature sequence is input into the mask-based self-attention network of the first decoding layer. The mask-based self-attention network weights the fused feature sequence to extract a latent vector sequence. This latent vector sequence is then input into the feedforward neural network of the first decoding layer. The feedforward neural network performs a fully connected operation on this latent vector sequence, outputting a fully connected latent vector sequence. This fully connected latent vector sequence is then input into the second decoding layer, and so on. The processing logic of subsequent decoding layers is similar to that of the first decoding layer, and will not be elaborated here.

[0111] It should be noted that when predicting the target statement features at subsequent time steps, only the target statement features predicted before the current time step are considered. For example, assuming the output target statement feature sequence is "target statement feature A, target statement feature B, target statement feature C", which includes a total of 3 target statement features, then when predicting target statement feature B, target statement feature C needs to be masked. That is, target statement feature B can only be predicted based on the fusion features of the input prompt statements and target statement feature A.

[0112] 203. Based on the feature of the at least one target statement, the server decodes the target text containing the at least one target statement, and the target text has a semantic correspondence with the prompt text.

[0113] In this context, the semantic correspondence between the target text and the prompt text means that the target text is a continuation text generated by machine intelligence, dependent on the prompt text. Since the target text follows the prompt text, and the prompt text follows the target text, they have a semantic correspondence. For example, the target text and the prompt text express the same semantics, or their semantics belong to the same context and have a contextual dependency. For instance, when the prompt text is a news summary, the target text can be a news draft automatically generated based on that summary. In this case, the main content and central idea of ​​the news summary and the news draft need to be consistent, meaning the news summary and the news draft have a semantic correspondence. As another example, when the prompt text is a product attribute, the target text can be a product description document automatically generated based on that product attribute. In this case, the product attribute and the product description document need to ensure that they describe the same product, meaning the product attribute and the product description document have a semantic correspondence.

[0114] The target text, also known as the statement that continues writing from the prompt information, refers to the output text of the long text generation process. The target text may include at least one target statement; for example, the target text may contain multiple target statements, or it may contain only a single target statement. This application embodiment does not specifically limit the number of target texts contained in the target text.

[0115] In some embodiments, when the server decodes any one of the at least one target statement features, it can decode the target statement corresponding to the target statement and repeat the above operation to obtain the target text containing the at least one target statement.

[0116] In some embodiments, for any target statement feature, a statement decoding model (also known as a vector-guided decoder) can be invoked to decode the corresponding target statement. This statement decoding model decodes the input statement feature to convert it into individual characters within the corresponding statement. The input to the statement decoding model is a target statement feature, and the output is a character sequence composed of the individual characters within the corresponding target statement. That is, the server inputs any target statement feature into at least one decoding layer in the statement decoding model, decodes the target statement feature through this at least one decoding layer, outputs at least one target character, and identifies this at least one target character as the target statement corresponding to the target statement feature. This process is repeated for each target statement feature to obtain various target statements, which together constitute the target text.

[0117] In some embodiments, the statement decoding model includes, but is not limited to: GPT2 model, Transformers model, CNN model, RNN model, LSTM model, Bi-LSTM model, GRU model, etc. The embodiments of this application do not specifically limit the architecture of the statement decoding model.

[0118] To illustrate, taking the GPT2 model as an example, the server inputs the i-th (i≥1) target sentence feature into the GPT2 model. The GPT2 model includes at least one decoding layer, which are connected in series. That is, the output of the previous decoding layer is used as the input of the next decoding layer. For example, the GPT2 model includes 12 decoding layers based on a self-attention structure.

[0119] Each decoding layer consists of two parts: a mask-based self-attention network and a feedforward neural network. The mask-based self-attention network focuses only on the t-1 characters of the predicted target sentence before the current time t (t≥1) (as well as the i-th target sentence feature). Information after the current time t needs to be masked. The i-th target sentence feature is input into the mask-based self-attention network, which outputs a hidden vector sequence. The feedforward neural network layer is used to fully connect the hidden vector sequence output by the mask-based self-attention network.

[0120] Taking the first decoding layer as an example, this encoding layer includes a mask-based self-attention network and a feedforward neural network. At time t=0, the feature of the i-th target sentence is input into the mask-based self-attention network of the first decoding layer. The mask-based self-attention network weights the i-th target sentence feature to extract a latent vector sequence. This latent vector sequence is then input into the feedforward neural network of the first decoding layer. The feedforward neural network performs a fully connected operation on this latent vector sequence, outputting a fully connected latent vector sequence. This fully connected latent vector sequence is then input into the second decoding layer, and so on. The processing logic of subsequent decoding layers is similar to that of the first decoding layer, and will not be elaborated here. Next, the fully connected latent vector sequence from the last decoding layer is input into a linear layer and then into an exponential normalization (softmax) layer to obtain the first character of the i-th target sentence, which is the prediction result at time t=1. The linear layer is a fully connected layer with a dimension equal to the total number of characters in the corpus. The exponential normalization layer is used to obtain the probability of each character in the output corpus at the current time. Usually, the character with the highest probability can be selected as the prediction result at the current time. Alternatively, one character can be randomly sampled from the top k (k≥1) characters with the highest probability in descending order as the prediction result at the current time. This application does not impose specific limitations on this.

[0121] In the above process, when predicting the target text, the natural language characters corresponding to each target sentence are predicted by using the target sentence features corresponding to each target sentence. This is equivalent to predicting the corresponding target sentence on a sentence-by-sentence basis, guided by the target sentence features. In other words, a decoding method guided by target sentence features is provided. In this way, even if individual characters in a target sentence deviate semantically, this semantic deviation will not be passed on to the next target sentence, because the next target sentence is predicted based on its own target sentence features, thereby greatly improving the accuracy of text generation.

[0122] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0123] The method provided in this application extracts sentence features from the prompt text on a sentence-by-sentence basis, and can predict the target sentence features of each target sentence in the following text. These target sentence features serve as semantic guidance information for the decoding process, and can decode each target sentence on a sentence-by-sentence basis to obtain the target text. This is equivalent to relying on the guidance of each target sentence feature during the generation of the target text, so that even if a character in a target sentence deviates semantically, this semantic deviation will be corrected by the next target sentence feature, and will not be transmitted in the entire target text, which greatly improves the accuracy of the text generation process.

[0124] Figure 3 This is a flowchart of a text generation method provided in an embodiment of this application. Please refer to it. Figure 3 This embodiment is applied to a computer device. The following description uses a server as an example of the computer device. This embodiment includes the following steps:

[0125] 301. The server obtains a prompt text, which contains at least one prompt statement.

[0126] In some embodiments, when the server obtains the prompt text, the user may input the prompt text on the server side, or read any pre-stored prompt text from a local database, or download the prompt text from a cloud database, or the prompt text may be sent by the receiving terminal.

[0127] In an exemplary scenario, a user logs into an application on a terminal, enters prompt text within the application, and triggers a text generation function option. In response to this triggering action, the terminal sends a text generation request carrying the prompt text to the server. The server receives the text generation request, parses it, and obtains the prompt text. Optionally, the triggering action includes, but is not limited to, click operations, double-click operations, long-press operations, voice commands, gesture commands, etc. This application embodiment does not specifically limit the form of the triggering action.

[0128] 302. The server obtains the statement features of the at least one prompt statement based on the prompt text containing the prompt statement, and the statement features are used to characterize the semantics of the prompt statement.

[0129] Step 302 is similar to step 201 above, and will not be described in detail here.

[0130] 303. The server obtains the positional features of the at least one prompt statement, which are used to characterize the word order information of the prompt statement in the prompt text.

[0131] In some embodiments, the server encodes the word order information (also known as position information) of each prompt statement in the target text to obtain the position feature of each prompt statement. By repeating the above operation, the position feature of at least one prompt statement can be obtained.

[0132] 304. The server fuses the statement features of the at least one prompt statement with the position features of the at least one prompt statement to obtain the fused features of the at least one prompt statement.

[0133] In some embodiments, the server concatenates the statement features and the position features of each prompt statement to obtain the fused features of the prompt statement, thereby simplifying the computation of obtaining the fused features.

[0134] In some embodiments, in addition to the splicing method, the statement features and their respective position features of each prompt statement can be added element-wise, multiplied element-wise, bilinearly merged, or vector producted to obtain the fused features of each prompt statement. The embodiments of this application do not specifically limit the fusion method of statement features and position features.

[0135] 305. The server decodes the fusion features of the at least one prompt statement to obtain at least one target statement feature, which is used to characterize the semantics of the target statement, and the target statement is the following text of the prompt statement.

[0136] In some embodiments, the server may invoke a statement planning model for decoding. This statement planning model includes at least one decoding layer, which is used to predict the target statement features of the following statement based on the fusion features of the prompt statement. That is, the server inputs the fusion features of the at least one prompt statement into the at least one decoding layer in the statement planning model, decodes the fusion features of the at least one prompt statement through the at least one decoding layer, and outputs the at least one target statement feature by the last decoding layer.

[0137] In the above process, the sentence planning model can predict at least one target sentence feature on a sentence-by-sentence basis, based on the fusion features of the prompt sentence. Each target sentence feature is used to characterize the semantics of a target sentence in the following text of the prompt text, i.e., the target text. However, at this time, the specific natural language content of each target sentence has not yet been decoded, but only the abstract space representation of each target sentence has been predicted.

[0138] Since the sentence planning model outputs at least one target sentence feature on a sentence-by-sentence basis, it can be regarded as a sentence-level language model (Language Modeling, LM, which is structurally similar to the decoder in the S2S framework). The sentence planning model can perform language modeling on the sentence feature sequence. Given a series of prompt sentence features (one or more), as the sentence feature sequence of the preceding text, the sentence planning model learns to predict the next target sentence feature containing the expression based on the sentence feature sequence of the preceding text, and outputs a series of target sentence features one by one as the target sentence feature sequence of the following text.

[0139] In some embodiments, the sentence planning model can be a sentence vector planner similar to the GPT2 model, including at least one decoding layer connected in series, meaning the output of one decoding layer serves as the input to the next. For example, the sentence vector planner includes 12 decoding layers based on a self-attention structure. Unlike the GPT2 model, this sentence vector planner models at the sentence level, while the GPT2 model models at the character level. Optionally, in this sentence vector planner, there is no need to set an embedding layer; instead, the fused features of each prompt sentence are directly used as the input representation. Optionally, when predicting the target sentence features, since there is no need to predict the characters of the natural language, there is no need to set an activation function for the output layer; the target sentence features predicted by the last encoding layer can be directly output.

[0140] Each decoding layer consists of two parts: a mask-based self-attention network and a feedforward neural network. The mask-based self-attention network only focuses on the features of each target statement predicted before the current time step (as well as the fusion features of the input prompt statements). Information after the current time step needs to be masked. The fusion feature sequence composed of the fusion features of each prompt statement is input into the mask-based self-attention network, which outputs a hidden vector sequence. The feedforward neural network layer is used to fully connect the hidden vector sequence output by the mask-based self-attention network.

[0141] Taking the first decoding layer as an example, this encoding layer includes a mask-based self-attention network and a feedforward neural network. The fused feature sequence is input into the mask-based self-attention network of the first decoding layer. The mask-based self-attention network weights the fused feature sequence to extract a latent vector sequence. This latent vector sequence is then input into the feedforward neural network of the first decoding layer. The feedforward neural network performs a fully connected operation on this latent vector sequence, outputting a fully connected latent vector sequence. This fully connected latent vector sequence is then input into the second decoding layer, and so on. The processing logic of subsequent decoding layers is similar to that of the first decoding layer, and will not be elaborated here.

[0142] It should be noted that when predicting the target statement features at subsequent time steps, only the target statement features predicted before the current time step are considered. For example, assuming the output target statement feature sequence is "target statement feature A, target statement feature B, target statement feature C", which includes a total of 3 target statement features, then when predicting target statement feature B, target statement feature C needs to be masked. That is, target statement feature B can only be predicted based on the fusion features of the input prompt statements and target statement feature A.

[0143] In steps 303-305 above, a possible implementation is shown whereby the server predicts at least one target statement feature based on the statement features of the at least one prompt statement. By fusing the statement features and position features of each prompt statement, when predicting the target statement feature from left to right, not only the statement features of the preceding text can be referenced, but also the word order information of the preceding text. This makes the predicted target statement feature conform to the semantic order and improves the prediction accuracy of the target statement feature.

[0144] 306. Based on the feature of the at least one target statement, the server decodes to obtain target text containing at least one target statement, and the target text has a semantic correspondence with the prompt text.

[0145] Step 306 above is similar to step 203 above, and will not be described in detail here.

[0146] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0147] The method provided in this application extracts sentence features from the prompt text on a sentence-by-sentence basis, and can predict the target sentence features of each target sentence in the following text. These target sentence features serve as semantic guidance information for the decoding process, and can decode each target sentence on a sentence-by-sentence basis to obtain the target text. This is equivalent to relying on the guidance of each target sentence feature during the generation of the target text, so that even if a character in a target sentence deviates semantically, this semantic deviation will be corrected by the next target sentence feature, and will not be transmitted in the entire target text, which greatly improves the accuracy of the text generation process.

[0148] Figure 4 This is a schematic diagram illustrating the principle of a text generation method provided in an embodiment of this application. Please refer to it. Figure 4 The following example illustrates the sentence encoding model as a sentence encoder 401, the sentence planning model as a sentence vector planner 402, and the sentence decoding model as a vector-guided decoder 403.

[0149] The function of the sentence encoder 401 is to convert the character sequence of the prompt statement into a sentence vector (i.e., sentence feature) of the prompt statement. Assuming that the input prompt text contains only one prompt statement, and the character sequence of the prompt statement is represented as [x1, x2, x3, x4], inputting [x1, x2, x3, x4] into the sentence encoder 401 will output the sentence vector z of the prompt statement. The sentence vector z is also the sentence feature of the prompt statement, which can represent the semantics of the entire prompt statement.

[0150] In the above process, the sentence encoder 401 treats sentences as semantic units and performs language modeling at the sentence level rather than the character level, thereby converting prompt sentences into sentence features to represent the semantic information of the entire sentence. The quality of the sentence features directly affects the performance of the sentence vector planner 402. Considering the strong understanding ability of the BERT model and its widespread application in various natural language understanding tasks, the sentence encoder 401 can adopt the BERT model to obtain the sentence features of each prompt sentence.

[0151] The BERT model can be viewed as an encoder containing 12 layers of self-attention structure. Optionally, the BERT model weights can be loaded first and kept fixed. Then, the character sequence of each prompt is input into the BERT model, and the output representation of the 11th layer of the BERT model is taken as the semantic representation (i.e., the semantic feature of each character) of the prompt. Finally, mean pooling is performed on the semantic features of each character output from the 11th layer to obtain the sentence features of the entire prompt. Optionally, this sentence encoder 401 does not need to be trained; instead, the original BERT model weights can be used directly to save server computing resources.

[0152] The function of the sentence vector planner 402 is to generate sentence vector sequences from left to right (i.e., generate at least one target sentence feature in order). Assuming that the sentence vector z output by the sentence encoder 401 is used as the input vector z1 at time t=1, after inputting z1 into the sentence vector planner 402, the sentence vector sequence [z2, z3, z4, z5] following the sentence can be output in ascending order of time. This sentence vector sequence is also the target sentence feature sequence composed of at least one target sentence feature.

[0153] The training process of the sentence vector planner 402, i.e., the sentence planning model, will be detailed in the next embodiment and will not be repeated here. In short, the sentence vector planner 402 is trained using a sequence of sentence vectors (i.e., a sequence of sentence features) pre-processed by the sentence encoder 401. After training, given the sentence vector (i.e., semantic features) of the first prompt sentence, it can predict the subsequent sequence of sentence vectors (i.e., the sequence of target sentence features constituted by the target sentence features) from left to right.

[0154] The function of the vector-guided decoder 403 is to decode the target sentence features into target sentences, so that the target sentences can be concatenated to obtain the final target text. For example, z represents any sentence vector in the sentence vector sequence [z2, z3, z4, z5] output by the sentence vector planner 402. Then, if the sentence vector z (i.e. any target sentence feature) is input into the vector-guided decoder 403, the output is the character sequence [y1, y2, y3, y4] of the target sentence corresponding to the sentence vector z.

[0155] The sentence vector planner 402 generates a sequence of sentence vectors, and the vector-guided decoder 403 is responsible for decoding this sequence of sentence vectors into corresponding target sentences. Concatenating these target sentences yields the final target text. Considering the strong generation capabilities of the GPT2 model and its widespread application in various natural language generation tasks, the vector-guided decoder 403 can employ the GPT2 model to decode and generate the target text.

[0156] In the above process, a sentence vector (i.e., any target sentence feature) can be used as the first input signal of the vector-guided decoder 403, and then each character in the target sentence is generated sequentially from left to right. Optionally, the vector-guided decoder 403 uses pre-processed <sentence vector, sentence text> pairs as training data, that is, a target sentence and a target sentence feature constitute a training sample pair, and is trained using the loss function of the GPT2 model itself. After training, the vector-guided decoder 403 is obtained, which can convert the sentence vector (i.e., target sentence feature) predicted by the sentence vector planner 402 into sentence text (i.e., target sentence).

[0157] In this embodiment, sentence encoder 401 can extract sentence features on a sentence-by-sentence basis, sentence vector planner 402 can predict the features of subsequent target sentences, and vector-guided decoder 403 can decode each target sentence and concatenate them to obtain the target text. In long text generation tasks, it can maintain consistency between sentences and generate each character in the target sentence under the guidance of global sentence semantics (i.e. target sentence features), effectively improving the quality of long text generation.

[0158] Figure 5 This is a flowchart illustrating the training process of a statement planning model provided in an embodiment of this application. Please refer to it. Figure 5 This embodiment is applied to a computer device. The following description uses a server as an example of the computer device. This embodiment includes the following steps:

[0159] 501. The server obtains sample text and annotation text, wherein the sample text contains at least one sample prompt statement and the annotation text contains at least one annotation statement, and the annotation text is the following text of the sample text.

[0160] In some embodiments, the server can collect long texts that have been filtered to have high training value, and use the first sentence of the long text as sample text, which contains only a prompt statement, and use all the remaining sentences of the long text as labeled text. The long texts can be automatically filtered by a pre-trained machine learning model or manually filtered by technicians; this application embodiment does not specifically limit this.

[0161] In some embodiments, technical personnel may collect sample texts and then manually write annotation texts for the sample texts. This application does not specifically limit the method of obtaining sample texts and annotation texts.

[0162] 502. The server obtains the fusion features of at least one sample prompt statement from sample text containing at least one sample prompt statement.

[0163] Step 502 above is similar to steps 302-304 above, and will not be described in detail here.

[0164] Optionally, the server obtains the statement features of the at least one sample prompt statement based on the sample text containing the at least one sample prompt statement; obtains the positional features of the at least one sample prompt statement, which are used to characterize the word order information of the sample prompt statement in the sample text; and fuses the statement features and positional features of the at least one sample prompt statement to obtain the fused features of the at least one sample prompt statement.

[0165] 503. For sample text containing at least one sample prompt statement, the server inputs the fusion features of the at least one sample prompt statement into the initial planning model and outputs at least one predicted statement feature.

[0166] Step 503 is similar to step 305 above, and will not be described in detail here.

[0167] Optionally, the server inputs the fusion features of the at least one sample prompt statement into at least one decoding layer in the initial planning model, decodes the fusion features of the at least one sample prompt statement through the at least one decoding layer, and outputs the at least one predicted statement feature by the last decoding layer.

[0168] 504. The server obtains a loss function value based on at least one predicted statement feature and at least one labeled statement feature of the labeled text, wherein the labeled text is the following text of the sample text.

[0169] In some embodiments, the server obtains at least one first similarity based on the at least one predicted statement feature and the at least one labeled statement feature, wherein the first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature; and obtains the loss function value based on the at least one first similarity.

[0170] Optionally, the similarity can be cosine similarity or the reciprocal of Euclidean distance, etc. The embodiments of this application do not specifically limit the form of the similarity.

[0171] In the above process, assuming that at time t, that is, the feature of the t-th predicted statement is... The feature of the corresponding t-th annotation statement is z. t Then the value of the loss function The expression can be as follows:

[0172]

[0173] in, represents the loss function value, and m represents the total number of annotated sentences in the annotated text. m is an integer greater than or equal to 1, and the value of m can be manually set by technicians. Since during training, for each prompt text, m predicted sentences are predicted backward and the loss function value is obtained between them and their corresponding m annotated sentences, then m also represents the target sentence features that the sentence planning model will predict for each of the m target sentences in the following text after the sentence planning model is trained.

[0174] In the above process, the loss function value takes into account the similarity between the predicted sentence features and the labeled sentence features, which can fully reflect the gap between the model prediction results and the real labeled results, and the amount of computation is small, which can improve the training speed of the model.

[0175] In some embodiments, the server obtains at least one first similarity based on the at least one predicted statement feature and the at least one labeled statement feature, wherein the first similarity is used to characterize the degree of similarity between a predicted statement feature and a corresponding labeled statement feature; obtains at least one second similarity based on the at least one predicted statement feature and at least one negative sample statement feature, wherein the second similarity is used to characterize the degree of similarity between a predicted statement feature and a corresponding negative sample statement feature; and obtains the loss function value based on the at least one first similarity and the at least one second similarity.

[0176] The first similarity and the second similarity can both be cosine similarity, or the reciprocal of Euclidean distance, etc. The embodiments of this application do not specifically limit the form of the first similarity or the second similarity.

[0177] In the above process, assuming that at time t, that is, the feature of the t-th predicted statement is... The feature of the corresponding t-th annotation statement is z. t Suppose that n negative sample sentence features are obtained by random sampling, and the i-th (1≤i≤n) negative sample sentence feature is represented as z. i Then the expression for the loss function value can be as follows:

[0178]

[0179] in, The loss function value is represented by m, which represents the total number of annotated sentences in the annotated text. m is an integer greater than or equal to 1, and its value can be manually set by technicians. Since during training, for each prompt text, m predicted sentences are generated and their corresponding m annotated sentences are compared to obtain the loss function value, m also represents the target sentence features that the sentence planning model will predict for each of the m target sentences in the following text after the sentence planning model is trained. Furthermore, n represents the number of negative sample sentence features obtained by random sampling, that is, the n negative sample sentence features of each of the n randomly sampled negative samples.

[0180] In some embodiments, when randomly sampling n negative samples, random sampling can be performed on all training samples in the training set except for the labeled text. Therefore, the only correct context is the labeled text itself, and all other training samples in the training set can be regarded as negative samples. This enables the construction of a loss function calculation method based on contrastive learning.

[0181] In the above process, the loss function value takes into account the first similarity between the predicted sentence features and the labeled sentence features. The first similarity can fully reflect the difference between the model prediction result and the real labeled result. In addition, the second similarity between the predicted sentence features and the features of other negative sample sentences is also considered. The second similarity can fully reflect the difference between the model prediction result and other negative samples. The optimization objective is to increase the first similarity and decrease the second similarity, which can improve the training performance of the model.

[0182] 505. The server iteratively adjusts the parameters of the initial planning model until the loss function value meets the stopping condition, thus obtaining the statement planning model.

[0183] Optionally, the stopping condition may be that the loss function value is less than a loss threshold, which may be any value greater than or equal to 0 and less than or equal to 1; or the stopping condition may be that the number of iterations is greater than a number threshold, which may be any integer greater than or equal to 1.

[0184] In the above process, the server iteratively executes steps 501-504 until the loss function value meets the stopping condition, at which point training stops and the initial planning model used in the last iteration is determined as the final statement planning model.

[0185] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0186] In this embodiment, the initial planning model is iteratively trained, and the similarity between the predicted sentence features and the labeled sentence features is introduced into the loss function. This fully reflects the gap between the model's prediction results and the actual labeled results. In addition, the similarity between the predicted sentence features and the sentence features of other negative samples is also introduced. This fully reflects the gap between the model's prediction results and other negative samples. As a result, the finally trained sentence planning model can predict the features of each target sentence from left to right based on the sentence features of a prompt sentence. This serves as global semantic guidance for decoding the target sentence and is applied to long text generation tasks to improve the accuracy of the text generation process.

[0187] Figure 6 This is a schematic diagram of the structure of a text generation device provided in an embodiment of this application. Please refer to it. Figure 6 The device includes:

[0188] The first acquisition module 601 is used to acquire the statement features of the at least one prompt statement based on the prompt text containing at least one prompt statement, the statement features being used to characterize the semantics of the prompt statement;

[0189] The prediction module 602 is used to predict at least one target statement feature based on the statement features of the at least one prompt statement, the target statement feature being used to characterize the semantics of the target statement, the target statement being the following text of the prompt statement;

[0190] The decoding module 603 is used to decode target text containing at least one target statement based on the features of the at least one target statement, and the target text has a semantic correspondence with the prompt text.

[0191] The apparatus provided in this application extracts sentence features from the prompt text on a sentence-by-sentence basis, and can predict the target sentence features of each target sentence in the following text. These target sentence features serve as semantic guidance information for the decoding process, and can decode each target sentence on a sentence-by-sentence basis to obtain the target text. This is equivalent to relying on the guidance of each target sentence feature during the generation of the target text, so that even if a character in a target sentence deviates semantically, this semantic deviation will be corrected by the next target sentence feature and will not be transmitted in the entire target text, which greatly improves the accuracy of the text generation process.

[0192] In one possible implementation, based on Figure 6 The device comprises the following components: the first acquisition module 601 includes:

[0193] The first acquisition unit is used to acquire the embedding feature of at least one character in any of the at least one prompting statements;

[0194] An encoding unit is used to encode the embedding features of the at least one character to obtain the semantic features of the at least one character;

[0195] The first fusion unit is used to fuse the semantic features of the at least one character to obtain the statement features of any prompt statement.

[0196] In one possible implementation, the first acquisition unit is used to: input the at least one character in any prompt statement to the statement embedding layer in the statement encoding model, perform embedding processing on the at least one character through the statement embedding layer, and output the embedding features of the at least one character. The statement encoding model is used to encode the input statement to extract statement features.

[0197] The encoding unit is used to: input the embedding features of at least one character into at least one encoding layer in the sentence encoding model, perform weighted processing on the embedding features of at least one character through the at least one encoding layer, and output the semantic features of at least one character;

[0198] The first fusion unit is used to: perform mean pooling on the semantic features of the at least one character to obtain the statement features of any prompt statement.

[0199] In one possible implementation, based on Figure 6 The device comprises, wherein the prediction module 602 includes:

[0200] The second acquisition unit is used to acquire the positional features of the at least one prompting statement, the positional features being used to characterize the word order information of the prompting statement in the prompting text;

[0201] The second fusion unit is used to fuse the statement features of the at least one prompt statement with the position features of the at least one prompt statement to obtain the fused features of the at least one prompt statement.

[0202] The decoding unit is used to decode the fusion features of the at least one prompt statement to obtain the features of the at least one target statement.

[0203] In one possible implementation, the decoding unit is used for:

[0204] The fusion features of at least one prompt statement are input into at least one decoding layer in the statement planning model. The fusion features of at least one prompt statement are decoded by the at least one decoding layer. The at least one target statement feature is output by the last decoding layer. The statement planning model is used to predict the target statement feature of the following statement based on the fusion features of the prompt statement.

[0205] In one possible implementation, based on Figure 6The device comprises:

[0206] The input / output module is used to take sample text containing at least one sample prompt statement, input the fusion features of the at least one sample prompt statement into the initial planning model, and output at least one predicted statement feature;

[0207] The second acquisition module is used to acquire a loss function value based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text, wherein the labeled text is the following text of the sample text;

[0208] The iterative adjustment module is used to iteratively adjust the parameters of the initial planning model until the loss function value meets the stopping condition, thus obtaining the statement planning model.

[0209] In one possible implementation, the second acquisition module is used to:

[0210] Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein the first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature;

[0211] The loss function value is obtained based on at least one first similarity.

[0212] In one possible implementation, the second acquisition module is used to:

[0213] Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein the first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature;

[0214] Based on the at least one predicted statement feature and the at least one negative sample statement feature, at least one second similarity is obtained, wherein the second similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding negative sample statement feature;

[0215] The loss function value is obtained based on the at least one first similarity and the at least one second similarity.

[0216] In one possible implementation, the decoding module 603 is used for:

[0217] For any one of the at least one target statement features, decode the target statement corresponding to the target statement feature, and repeat the above operation to obtain the target text containing the at least one target statement.

[0218] In one possible implementation, the decoding module 603 is used for:

[0219] The target statement feature is input into at least one decoding layer in the statement decoding model. The target statement feature is decoded by the at least one decoding layer to output at least one target character. The at least one target character is determined as the target statement corresponding to the target statement feature. The statement decoding model is used to decode the input statement feature to convert it into each character in the corresponding statement.

[0220] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0221] It should be noted that the text generation device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the text generation device and the text generation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the text generation method embodiments, which will not be repeated here.

[0222] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Please refer to it. Figure 7 Taking a computer device as a terminal 700 as an example, the terminal 700 loads a statement encoding model, a statement planning model, a statement decoding model, etc. locally to complete the text generation method provided in the embodiments of this application locally without incurring additional communication overhead with the server.

[0223] Optionally, the device type of the terminal 700 includes: a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 700 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0224] Typically, terminal 700 includes a processor 701 and a memory 702.

[0225] Optionally, the processor 701 includes one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Optionally, the processor 701 is implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). In some embodiments, the processor 701 includes a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 701 integrates a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, the processor 701 also includes an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0226] In some embodiments, memory 702 includes one or more computer-readable storage media, optionally non-transitory. Optionally, memory 702 also includes high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 702 is used to store at least one program code, which is executed by processor 701 to implement the text generation methods provided in the various embodiments of this application.

[0227] In some embodiments, the terminal 700 may also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning assembly 708, and a power supply 709.

[0228] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 are implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0229] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. Optionally, the RF circuit 704 communicates with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 also includes circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0230] Display screen 705 is used to display a UI (User Interface). Optionally, the UI includes graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, display screen 705 also has the ability to collect touch signals on or above the surface of display screen 705. The touch signals can be input to processor 701 for processing as control signals. Optionally, display screen 705 is also used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there is one display screen 705, which is set on the front panel of terminal 700; in other embodiments, there are at least two display screens 705, which are respectively set on different surfaces of terminal 700 or have a folded design; in still other embodiments, display screen 705 is a flexible display screen, which is set on the curved surface or folded surface of terminal 700. Furthermore, optionally, display screen 705 is set as a non-rectangular irregular shape, that is, an irregularly shaped screen. Optionally, the display screen 705 is made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0231] The camera assembly 706 is used to acquire images or videos. Optionally, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 also includes a flash. Optionally, the flash is a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, used for light compensation at different color temperatures.

[0232] In some embodiments, the audio circuit 707 includes a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 to realize voice communication. For stereo acquisition or noise reduction purposes, multiple microphones are used, respectively located at different parts of the terminal 700. Optionally, the microphone is an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. Optionally, the speaker is a conventional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves that are audible to humans, but also convert electrical signals into sound waves that are inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 707 also includes a headphone jack.

[0233] The positioning component 708 is used to determine the current geographic location of the terminal 700 for navigation or LBS (Location Based Service). Optionally, the positioning component 708 is a positioning component based on the US GPS (Global Positioning System), China's BeiDou system, Russia's Granas system, or the EU's Galileo system.

[0234] The power supply 709 is used to power the various components in the terminal 700. Optionally, the power supply 709 is AC power, DC power, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery supports wired or wireless charging. The rechargeable battery also supports fast charging technology.

[0235] In some embodiments, the terminal 700 further includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: an accelerometer 711, a gyroscope 712, a pressure sensor 713, a fingerprint sensor 714, an optical sensor 715, and a proximity sensor 716.

[0236] In some embodiments, the accelerometer 711 detects the magnitude of acceleration along the three coordinate axes of a coordinate system established with the terminal 700. For example, the accelerometer 711 is used to detect the components of gravitational acceleration along the three coordinate axes. Optionally, the processor 701 controls the display screen 705 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by the accelerometer 711. The accelerometer 711 is also used for acquiring motion data from games or the user.

[0237] In some embodiments, the gyroscope sensor 712 detects the orientation and rotation angle of the terminal 700. The gyroscope sensor 712 and the accelerometer sensor 711 work together to acquire the user's 3D movements on the terminal 700. Based on the data acquired by the gyroscope sensor 712, the processor 701 implements the following functions: motion sensing (e.g., changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0238] Optionally, the pressure sensor 713 is disposed on the side bezel of the terminal 700 and / or on the lower layer of the display screen 705. When the pressure sensor 713 is disposed on the side bezel of the terminal 700, it can detect the user's grip signal on the terminal 700, and the processor 701 performs left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed on the lower layer of the display screen 705, the processor 701 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 705. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0239] The fingerprint sensor 714 is used to collect a user's fingerprint. The processor 701 identifies the user based on the fingerprint collected by the fingerprint sensor 714, or vice versa. When the user's identity is identified as trusted, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. Optionally, the fingerprint sensor 714 can be located on the front, back, or side of the terminal 700. When the terminal 700 has a physical button or manufacturer logo, the fingerprint sensor 714 can be integrated with the physical button or manufacturer logo.

[0240] An optical sensor 715 is used to collect ambient light intensity. In one embodiment, the processor 701 controls the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is decreased. In another embodiment, the processor 701 also dynamically adjusts the shooting parameters of the camera assembly 706 based on the ambient light intensity collected by the optical sensor 715.

[0241] The proximity sensor 716, also known as a distance sensor, is typically located on the front panel of the terminal 700. The proximity sensor 716 is used to detect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually decreasing, the processor 701 controls the display screen 705 to switch from a screen-on state to a screen-off state; when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually increasing, the processor 701 controls the display screen 705 to switch from a screen-off state to a screen-on state.

[0242] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on terminal 700, which may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0243] Figure 8 This is a schematic diagram of a computer device 800 provided in an embodiment of this application. The computer device 800 can vary significantly due to differences in configuration or performance. The computer device 800 includes one or more Central Processing Units (CPUs) 801 and one or more memories 802. The memories 802 store at least one computer program, which is loaded and executed by the one or more processors 801 to implement the text generation methods provided in the various embodiments described above. Optionally, the computer device 800 also includes wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The computer device 800 also includes other components for implementing device functions, which will not be elaborated here.

[0244] In exemplary embodiments, a computer-readable storage medium is also provided, such as a memory including at least one computer program, which can be executed by a processor in a terminal to perform the text generation methods in the various embodiments described above. For example, the computer-readable storage medium includes ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices, etc.

[0245] In an exemplary embodiment, a computer program product or computer program is also provided, including one or more lines of program code stored in a computer-readable storage medium. One or more processors of a computer device are capable of reading the one or more lines of program code from the computer-readable storage medium, and the one or more processors execute the one or more lines of program code, enabling the computer device to perform the text generation method described in the above embodiments.

[0246] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. Optionally, the program is stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0247] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A text generation method, characterized in that, The method includes: Based on the prompt text containing at least one prompt statement, obtain the statement features of the at least one prompt statement, the statement features being used to characterize the semantics of the prompt statement; Obtain the positional features of the at least one prompting statement, wherein the positional features are used to characterize the word order information of the prompting statement in the prompting text; The statement features and positional features of the at least one prompt statement are fused together to obtain the fused features of the at least one prompt statement; The fusion features of the at least one prompt statement are input into at least one decoding layer in the statement planning model. The at least one decoding layer decodes the fusion features of the at least one prompt statement. The last decoding layer outputs the at least one target statement feature. The statement planning model is used to predict the target statement feature of the following statement based on the fusion features of the prompt statement. The target statement feature is used to characterize the semantics of the target statement. The target statement is the following statement of the prompt statement. Based on the at least one target statement feature, target text containing at least one target statement is decoded, and the target text has a semantic correspondence with the prompt text; The training process of the statement planning model includes: For a sample text containing at least one sample prompt statement, the fusion features of the at least one sample prompt statement are input into the initial planning model, and at least one predicted statement feature is output. Based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text, a loss function value is obtained, wherein the labeled text is the following text of the sample text; The parameters of the initial planning model are iteratively adjusted until the loss function value meets the stopping condition, thus obtaining the statement planning model.

2. The method according to claim 1, characterized in that, The step of obtaining the statement features of the at least one prompt statement based on the prompt text containing at least one prompt statement includes: For any one of the at least one prompting statements, obtain the embedding feature of at least one character in the prompting statement; Encode the embedding features of the at least one character to obtain the semantic features of the at least one character; The semantic features of the at least one character are fused to obtain the statement features of any prompt statement.

3. The method according to claim 2, characterized in that, The step of obtaining the embedding feature of at least one character in any prompt statement includes: inputting at least one character in any prompt statement into the statement embedding layer in the statement encoding model, performing embedding processing on the at least one character through the statement embedding layer, and outputting the embedding feature of the at least one character. The statement encoding model is used to encode the input statement to extract statement features. Encoding the embedding features of the at least one character to obtain the semantic features of the at least one character includes: inputting the embedding features of the at least one character into at least one encoding layer in the statement encoding model, performing weighted processing on the embedding features of the at least one character through the at least one encoding layer, and outputting the semantic features of the at least one character; The step of fusing the semantic features of the at least one character to obtain the statement features of any prompt statement includes: performing mean pooling on the semantic features of the at least one character to obtain the statement features of any prompt statement.

4. The method according to claim 1, characterized in that, The step of obtaining the loss function value based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text includes: Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein a first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature; The loss function value is obtained based on the at least one first similarity.

5. The method according to claim 4, characterized in that, The step of obtaining the loss function value based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text includes: Based on the at least one predicted statement feature and the at least one labeled statement feature, at least one first similarity is obtained, wherein a first similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding labeled statement feature; Based on the at least one predicted statement feature and the at least one negative sample statement feature, at least one second similarity is obtained, wherein a second similarity is used to characterize the degree of similarity between a predicted statement feature and the corresponding negative sample statement feature; The loss function value is obtained based on the at least one first similarity and the at least one second similarity.

6. The method according to claim 1, characterized in that, The process of decoding the target text containing at least one target statement based on the at least one target statement feature includes: For any one of the at least one target statement features, decode the target statement to obtain the target statement corresponding to the target statement feature. Repeat the above operation to obtain the target text containing the at least one target statement.

7. The method according to claim 6, characterized in that, Decoding any target statement feature to obtain the target statement corresponding to any target statement feature includes: The target sentence feature is input into at least one decoding layer in the sentence decoding model. The target sentence feature is decoded by the at least one decoding layer to output at least one target character. The at least one target character is determined as the target sentence corresponding to the target sentence feature. The sentence decoding model is used to decode the input sentence feature to convert it into each character in the corresponding sentence.

8. A text generation device, characterized in that, The device includes: The first acquisition module is used to acquire statement features of the at least one prompt statement based on the prompt text containing at least one prompt statement, wherein the statement features are used to characterize the semantics of the prompt statement; A prediction module is used to obtain the positional features of the at least one prompting statement, the positional features being used to characterize the word order information of the prompting statement in the prompting text; to fuse the sentence features and the positional features of the at least one prompting statement to obtain the fused features of the at least one prompting statement; to input the fused features of the at least one prompting statement into at least one decoding layer in a sentence planning model, to decode the fused features of the at least one prompting statement through the at least one decoding layer, and to output the at least one target sentence feature by the last decoding layer; the sentence planning model is used to predict the target sentence feature of the following statement based on the fused features of the prompting statement, the target sentence feature being used to characterize the semantics of the target statement, and the target statement being the following statement of the prompting statement; A decoding module is used to decode target text containing at least one target statement based on the features of the at least one target statement, wherein the target text has a semantic correspondence with the prompt text; The device further includes: The input / output module is used to input the fusion features of the at least one sample prompt statement into the initial planning model and output at least one predicted statement feature for the sample text containing at least one sample prompt statement. The second acquisition module is used to acquire a loss function value based on the at least one predicted statement feature and the at least one labeled statement feature of the labeled text, wherein the labeled text is the following text of the sample text; The iterative adjustment module is used to iteratively adjust the parameters of the initial planning model until the loss function value meets the stopping condition, thereby obtaining the statement planning model.

9. The apparatus according to claim 8, characterized in that, The first acquisition module includes: The first acquisition unit is used to acquire the embedding feature of at least one character in any of the at least one prompting statements; An encoding unit is used to encode the embedding features of the at least one character to obtain the semantic features of the at least one character; The first fusion unit is used to fuse the semantic features of the at least one character to obtain the statement features of any prompt statement.

10. The apparatus according to claim 9, characterized in that, The first acquisition unit is used to: input at least one character in any prompt statement to the statement embedding layer in the statement encoding model, perform embedding processing on the at least one character through the statement embedding layer, and output the embedding features of the at least one character. The statement encoding model is used to encode the input statement to extract statement features. The encoding unit is used to: input the embedding features of the at least one character into at least one encoding layer in the statement encoding model, perform weighted processing on the embedding features of the at least one character through the at least one encoding layer, and output the semantic features of the at least one character; The first fusion unit is used to: perform mean pooling on the semantic features of the at least one character to obtain the statement features of any prompt statement.

11. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the text generation method as described in any one of claims 1 to 7.

12. A storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the text generation method as described in any one of claims 1 to 7.

Images