Text generation method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses a text generation method and device, electronic equipment, a storage medium and a program product, and belongs to the field of artificial intelligence. The method comprises the steps that a first cue word and a first answer are obtained, the first answer is generated based on the first cue word, and the first answer comprises a first text; inputting the first cue word and the first answer into a text generation model, and generating at least one first lexical element according to the first cue word and the first answer through the text generation model; and outputting a second answer through a text generation model according to the at least one first lexical element and the first text.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence, and specifically relates to a text generation method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of artificial intelligence technology, large language models (LLMs) in the field of natural language processing (NLP) have the ability to handle complex language tasks and can be used for a variety of tasks such as text generation, machine translation, and sentiment analysis, greatly improving the efficiency and accuracy of information processing. After a user uses LLM to generate an answer, the user may be satisfied with part of the answer generated by LLM, but not with other parts. The user can repeatedly input the same prompt word (prompt) to LLM to make LLM regenerate different answers until LLM generates an answer that the user is satisfied with.

[0003] However, LLM cannot effectively identify the user's specific satisfaction with the generated content, resulting in that after the user repeatedly inputs the same prompt, LLM may still only generate answers that only partially satisfy the user, making it difficult to improve the user's satisfaction with the answer. As a result, the existing text generation methods are not flexible enough. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a text generation method, device, electronic device, storage medium and program product, which can improve the flexibility of the text generation method.

[0005] In a first aspect, an embodiment of the present application provides a text generation method, the method comprising: obtaining a first prompt word and a first answer, the first answer being generated based on the first prompt word, and the first answer comprising a first text; inputting the first prompt word and the first answer into a text generation model, and generating at least one first word element according to the first prompt word and the first answer through the text generation model; and outputting a second answer according to the at least one first word element and the first text through the text generation model.

[0006] In a second aspect, an embodiment of the present application provides a text generation device, which includes: an acquisition module and an execution module. The acquisition module is used to acquire a first prompt word and a first answer, the first answer is generated based on the first prompt word, and the first answer includes a first text; the execution module is used to input the first prompt word and the first answer acquired by the acquisition module into a text generation model, and generate at least one first word unit according to the first prompt word and the first answer through the text generation model; the execution module is also used to output a second answer according to the at least one first word unit and the first text through the text generation model.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, a first prompt word and a first answer generated based on the first prompt word can be obtained, the first answer including a first text, and then the first prompt word and the first answer can be input into a text generation model, and at least one first word element is generated according to the first prompt word and the first answer by the text generation model, and then a second answer is output according to the at least one first word element and the first text by the text generation model. In this solution, since the electronic device can generate at least one first word element according to the first prompt word and the first answer, that is, a new text content is generated according to the first prompt word and the answer generated based on the first prompt word, and then the second answer is output according to the at least one first word element and the first text by the text generation model, that is, the new text content is combined with the user-satisfied part of the answer generated based on the first prompt word to obtain the second answer, so that the user-satisfied part can be retained while generating a new answer different from the first answer according to the first prompt word, thereby improving the probability of generating a user-satisfied answer, thereby improving the flexibility of the text generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is one of the flow charts of the text generation method provided in the embodiment of the present application;

[0013] Figure 2 is a schematic diagram of the first text in the first answer provided in the embodiment of the present application;

[0014] Figure 3 This is the second flowchart of the text generation method provided in the embodiment of the present application;

[0015] Figure 4 This is the third flowchart of the text generation method provided in the embodiment of the present application;

[0016] Figure 5 is a schematic diagram of the structure of a text generation model provided in an embodiment of the present application;

[0017] Figure 6 This is the fourth flowchart of the text generation method provided in the embodiment of the present application;

[0018] Figure 7 This is the fifth flowchart of the text generation method provided in the embodiment of the present application;

[0019] Figure 8 is a schematic diagram of calculating the fourth probability provided by an embodiment of the present application;

[0020] Fig. 9 is a schematic diagram of the second answer provided in the embodiment of the present application;

[0021] Fig.10 This is the sixth flowchart of the text generation method provided in the embodiment of the present application;

[0022] Fig.11 This is the seventh flowchart of the text generation method provided in the embodiment of the present application;

[0023] Fig.12 It is a schematic diagram of the execution process of the text generation method provided in the embodiment of the present application;

[0024] Fig.13 It is one of the schematic diagrams of the text generation device provided in the embodiment of the present application;

[0025] Fig.14 This is the second schematic diagram of the text generation device provided in the embodiment of the present application;

[0026] Fig.15 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Fig.16 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The terms "at least one (item)", "at least one of" and the like in this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0031] The text generation method, device, electronic device, storage medium and program product provided in the embodiments of the present application are described in detail below in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0032] The embodiments of the present application can be applied to the scenario of generating text content based on prompt words input by the user. Specifically, after the user inputs specific prompt words so that the electronic device generates relevant text content according to these prompt words, the user may be satisfied with only part of the text content, and the user needs to make the electronic device generate new text content according to the prompt words to obtain a satisfactory answer.

[0033] The text generation method provided in the embodiments of the present application is exemplarily described below by taking some specific scenarios of the embodiments of the present application as examples.

[0034] Scenario 1: Assume that a user wants to obtain a self-introduction text when preparing for an interview. After the user enters "self-introduction for interview" as the prompt word, the electronic device generates a self-introduction text based on the prompt word, but the user is only satisfied with the "personal background and education experience" part of the self-introduction text. The user can mark the "personal background and education experience" part so that the electronic device can regenerate a new self-introduction text including the "personal background and education experience" part, thereby ensuring that the new text retains the part of the first self-introduction text that the user is satisfied with, while optimizing and updating the other parts of the text, thereby improving the user's satisfaction with the text.

[0035] Scenario 2: Assume that the user is writing an article about healthy eating. The user enters "The importance of healthy eating" as the prompt word, and the electronic device generates a relevant text, which includes two parts: "Nutrients" and "The importance of healthy eating". However, the user finds that the description of "Nutrients" in the text is not detailed enough. The user can mark the "Importance of healthy eating" part so that the electronic device generates relevant text that retains "The importance of healthy eating", thereby retaining the original "Importance of healthy eating" part while further expanding the "Nutrients" part to improve the user's satisfaction with the text.

[0036] Scenario 3: Assume that a user wants to post a dynamic about travel on social media. The user enters "Share my travel experience" as the prompt word, and the electronic device generates a dynamic text, including two parts: "Travel destination" and "Travel experience". The user is very satisfied with the description of the "travel destination" part, but wants to modify the "travel experience" part. The user can mark the "travel destination" part so that the electronic device generates a dynamic text that retains this part of the content, and rewrites the "travel experience" part to improve the user's satisfaction with the dynamic.

[0037] It should be noted that the above-mentioned scenarios 1 to 3 are merely illustrative examples of some scenarios in which the embodiments of the present application may be applied. In actual implementation, the embodiments of the present application can also be applied to more demand text generation scenarios, and the embodiments of the present application are not limited here.

[0038] The embodiments of the present application provide a text generation method, device, electronic device, storage medium and program product. Since the electronic device can generate at least one first word-gram according to a first prompt word and a first answer, that is, generate new text content according to the first prompt word and the answer generated based on the first prompt word, and then output a second answer according to the at least one first word-gram and the first text through a text generation model, that is, combine the new text content with the part of the answer generated based on the first prompt word that satisfies the user to obtain the second answer, so that while generating a new answer different from the first answer according to the first prompt word, the part that satisfies the user is retained, thereby improving the probability of generating an answer that satisfies the user, thereby improving the flexibility of the text generation method.

[0039] The execution subject of the text generation method provided in the embodiment of the present application may be a text generation device, which may be an electronic device, or a functional module or functional entity in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0040] Figure 1A flowchart of a text generation method provided by an embodiment of the present application is shown. Figure 1 As shown, the text generation method provided in the embodiment of the present application may include the following steps 201 to 203.

[0041] Step 201: The electronic device obtains a first prompt word and a first answer.

[0042] In the embodiment of the present application, the first answer is generated based on the first prompt word, and the first answer includes a first text.

[0043] It should be noted that the above-mentioned prompt words are also called prompts, which are specific prompt words or phrases input by the user to guide the text generation model to generate relevant text. For example, the prompt can be a question, topic, keyword or any other form of instruction. The text generation model can understand the user's needs based on the instruction and generate corresponding text.

[0044] In an embodiment of the present application, the first answer is a text related to the prompt word generated by a text generation model based on the first prompt word. The text may include multiple paragraphs, the content of which may include topics, background information, relevant data or suggestions related to the prompt word.

[0045] It should be noted that the above-mentioned text generation model is also called a large language model or a large model. It is a computing model based on machine learning and natural language processing technology, which can generate new text content based on the input text (such as prompt words or context). The text generation model is usually trained based on massive text data, with parameters at the level of billions, tens of billions, or even hundreds of billions and trillions. It has a large amount of knowledge information and can be fine-tuned to different tasks based on different training data. In an embodiment of the present application, the electronic device can generate a first answer by inputting a prompt word into the text generation model, and after the user marks the first text from the first answer, the electronic device can generate a new answer by inputting the first answer and the prompt word into the text generation model.

[0046] In the embodiment of the present application, the first text is the text that the user wants to keep in the first answer, that is, the first text in the first answer can meet the needs of the user. The first text is the text selected by the user, which reflects the user's recognition and satisfaction with the text.

[0047] In the embodiment of the present application, the electronic device can analyze the meaning of the first prompt word based on the first prompt word input by the user, through the knowledge and language patterns learned in the text generation model training process, and generate text content related to it, that is, the first answer. The user can mark the first text in the first answer so that the electronic device can know the text that the user is satisfied with, and retain the first text in the newly generated answer while regenerating the answer.

[0048] For example, a mobile phone is used as an example. Figure 2 As shown, assuming that the first prompt word 10 input by the user is "self-introduction during an interview", the mobile phone can generate a first answer 11 based on the first prompt word 10. Assuming that the content of the first answer 11 is as follows:

[0049] "The self-introduction during an interview is the key to leaving a good first impression on the interviewer. A good self-introduction should be concise and to the point, while fully demonstrating your strengths and personality. The following is a basic self-introduction template that you can adjust according to your actual situation:

[0050] [beginning]

[0051] Dear interviewers, hello! I am [your name], graduated from [your school], majored in [your major]. Thank you very much [company name] for giving me this interview opportunity, giving me the opportunity to show myself to you.

[0052] [Educational Background]

[0053] During my university years, I systematically studied [relevant courses] and actively participated in [relevant projects or activities]. Through these studies and practices, I not only mastered solid professional knowledge, but also developed strong learning ability and teamwork spirit.

[0054] [Work Experience]

[0055] (If you have relevant work or internship experience) During my internship at [internship company], I was mainly responsible for [specific work content], during which I learned [learned skills] and achieved [specific results]. These experiences have given me a deeper understanding of [position name] and have strengthened my career development direction in this field.

[0056] [Personal advantages]

[0057] I think I have the following strengths: First, I have strong [certain skills or qualities] and can quickly adapt to new environments and new tasks; second, I am good at communication and teamwork, and can effectively cooperate with colleagues from different backgrounds; finally, I take my work seriously and responsibly, pay attention to details and pursue excellence.

[0058] [Expectations for future work]

[0059] I am full of enthusiasm and anticipation for [Company Name] and [Position Name]. I believe that my professional background and work experience can bring new vitality to the team. If I am lucky enough to join [Company Name], I will work hard, constantly improve myself, and contribute my own strength to the development of the company.

[0060] [End]

[0061] Thank you again for giving me this opportunity. I look forward to further communication with you. Thank you!

[0062] I hope this template was helpful. Good luck with your interview! ”

[0063] The user can mark the text that he or she is satisfied with in the first answer 11, such as first text 12, first text 13, and first text 14, so that the electronic device can learn the text that the user is satisfied with and generate a new answer containing these texts.

[0064] Step 202: The electronic device inputs the first prompt word and the first answer into a text generation model, and generates at least one first word-unit according to the first prompt word and the first answer through the text generation model.

[0065] In the embodiment of the present application, the above-mentioned word element is also called a token, which is the basic unit in text generation, usually referring to a word, subword, character or punctuation mark. During the generation process, the text generation model can gradually generate these word elements to construct a complete text content.

[0066] In the embodiment of the present application, the above “generating at least one first word” can be understood as the text generation model generating text including at least one word according to the input prompt word and answer. For example, the generated text can be a single word, or the generated text can be a phrase.

[0067] In the embodiment of the present application, the electronic device can analyze the content of the first prompt word and the first answer through the text generation model to understand the user's needs and context. Based on this understanding, the model will generate word units related to the prompt word, and these word units will be spliced ​​together to form new text content.

[0068] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 3 As shown, the above step 202 can be specifically implemented through the following steps 202a and 202b.

[0069] Step 202a: The electronic device inputs the first prompt word and the first answer into a multi-head prediction module of a text generation model, and obtains N second word-units based on the first answer through the multi-head prediction module.

[0070] In the embodiment of the present application, N is a positive integer.

[0071] In the embodiment of the present application, the multi-head prediction module is an important component of the text generation model, which is usually based on the transformer architecture. Its main function is to generate multiple subsequent word units in parallel, thereby improving the efficiency and diversity of text generation. In the process of multi-head prediction by the electronic device, multiple heads can be used simultaneously to predict multiple subsequent word units.

[0072] It should be noted that the common structure of existing large language models is the decoder-only transformer structure. In this structure, the process of inferring word units in the large language model is as follows:

[0073] During model training, since the target answers that are expected to be achieved are additionally input, the decoder-specific transformer can use the word-grams in these target answers to generate all word-grams in parallel.

[0074] Unlike model training, when a large language model using this structure is inferencing and decoding, it cannot obtain all the information, that is, it cannot access the subsequently generated word-grams, and can usually only generate word-grams one by one in serial. During the reasoning process, the model can only rely on part of the text that has been generated to predict the next word-gram, which means that only one word-gram can be generated at each step of reasoning. Specifically, at each generation step, the model calculates the probability distribution of the next word-gram based on the currently generated text and input prompts, and selects one word-gram as the output. Although this serial generation method ensures the coherence of the generated text, it also leads to low reasoning efficiency, especially when generating longer texts, the model's reasoning time will increase significantly.

[0075] Optionally, in an embodiment of the present application, since it is necessary to compare with the first text to determine the position of the first text segment, the efficiency of using serial generation of word units is too low, so the electronic device can use a multi-head prediction module to achieve parallel generation of word units through a multi-head mechanism. Specifically, on the basis of retaining the original language model head (LanguageModel Head, LM Head) in the last hidden layer of the large model, n additional heads (add heads) are added to predict subsequent tokens respectively. The structure and parameters of the newly added head are consistent with those of the original LM Head. The first add head predicts the next token of the next token, and the second add head predicts the next token of the next token of the next token, and the subsequent add heads are deduced by analogy. During the training process, the parameters of the original model can be frozen, and only the parameters of these add heads are trained and updated, so as to train a multi-head prediction module. The newly added n add heads can enable the large model to obtain the probability distribution of the subsequent n+1 tokens in one reasoning.

[0076] Optionally, in the embodiment of the present application, the number n of heads newly added in the multi-head prediction module may be a system default number or a user preset number. For example, n may be 2, 3, 4, etc. The specific number may be determined according to actual usage requirements and is not limited in the embodiment of the present application.

[0077] In the embodiment of the present application, the above-mentioned N second word-grams are N possible candidate word-grams predicted by the large model based on the text content of the first answer.

[0078] In an embodiment of the present application, the electronic device can input the first prompt word and the first answer into the multi-head prediction module at the same time, and then the large model can obtain N second word units for generating a new answer based on the content of the first answer.

[0079] Through parallel generation, the model can provide multiple candidate tokens in a shorter time, increasing the diversity of generated text.

[0080] Step 202b: The electronic device determines at least one first word-gram from the N second word-grams according to the first prompt word and the N second word-grams through the multi-head prediction module.

[0081] In the embodiment of the present application, the at least one first word-element is a word-element related to the first prompt word determined by the electronic device from among the N candidate second word-element.

[0082] Optionally, in an embodiment of the present application, after obtaining N second word-grams for generating a new answer, the electronic device can determine the next word-gram starting with the first prompt word from the N second word-grams based on the first prompt word, that is, the first word-gram, and then based on the first word-gram, determine the next word-gram of the first word-gram from the N second word-grams, that is, the second word-gram, and so on, to generate at least one first word-gram.

[0083] Optionally, in an embodiment of the present application, the at least one first word-gram is divided into at least one word-gram group, and each word-gram group includes a part of the word-grams in at least one first word-gram. Figure 3 ,like Figure 4 As shown, the above step 202b can be specifically implemented through the following steps 202b1 and 202b2.

[0084] Step 202b1: For the i-th word-tuple in at least one word-tuple, the electronic device determines the first probability of each second word-tuple as the first word-tuple in accordance with the first prompt word through the multi-head prediction module, and uses the second word-tuple with the highest first probability among the N second word-tuples as the first word-tuple in the i-th word-tuple.

[0085] In the embodiment of the present application, i is a positive integer.

[0086] In the embodiments of the present application, the above-mentioned token group is a token sequence composed of multiple tokens, that is, the number of tokens that the large model can predict simultaneously.

[0087] Optionally, in the embodiments of the present application, the number of tokens included in each token group in the above-mentioned at least one token group can be determined by the number of heads in the multi-head prediction module. For example, it can be the same as the number of heads in the multi-head prediction module, or it can be a multiple of the number of heads in the multi-head prediction module. Thus, the tokens in each token group are the tokens obtained by the electronic device through the multi-head prediction module an integer number of times.

[0088] Exemplarily, assume that 4 new heads are added to the multi-head prediction module, that is, the multi-head prediction module includes 4 + 1 = 5 heads. Then the number of tokens included in each token group can also be 5.

[0089] Optionally, in the embodiments of the present application, for the first token group in the at least one token group, the electronic device can, through the multi-head prediction module, determine the probability of each second token as the first token in the first token group according to the first prompt word, and use the second token with the highest probability as the first token in the first token group.

[0090] Optionally, in the embodiments of the present application, for other token groups other than the first token group in the at least one token group, the electronic device can, through the multi-head prediction module, determine the first probability of each second token as the first token in this token group according to the last token in the previous token group, and use the second token with the highest probability as the first token in this token group.

[0091] Step 202b2: The electronic device, through the multi-head prediction module, respectively determines the second probability of each second token as the j-th token in the i-th token group according to the first prompt word, and uses the second token with the highest second probability and associated with the (j - 1)-th token in the i-th token group as the j-th token in the i-th token group.

[0092] In the embodiments of the present application, j is a positive integer greater than 1.

[0093] In the embodiments of the present application, the above-mentioned association includes semantic, syntactic structure, context, or logical and other interconnections between the generated tokens. Specifically:

[0094] (1) The generated token should be semantically associated with the previous token. For example, if the previous token is "interview", then the next token of "interview" may be "time" or "of", and these tokens are semantically related to "interview" to form a coherent phrase or sentence.

[0095] (2) The generated lemmas need to follow the grammatical rules of the language to ensure the correctness and fluency of the sentence. For example, if the previous lemma is a noun, the next lemma may be a verb or adjective to maintain the grammatical structure of the sentence.

[0096] (3) The generated token should take into account the previous contextual information to ensure that the generated content is consistent with the previous content. For example, if the previous token mentions a specific topic or object, then the subsequent token should be consistent with that topic or object.

[0097] (4) The generated lemmas may need to be logically related to each other. For example, if the previous lemma is “because”, then the next lemma may need to contain “so” or “therefore” to form a logically coherent sentence.

[0098] In this way, by considering the associations between these word units, electronic devices can generate more natural and coherent text that conforms to the expression habits of human language.

[0099] Optionally, in an embodiment of the present application, after determining the first word in the first word group, the electronic device can continue to determine the probability of each second word being the second word in the first word group based on the first prompt word, and use the second word with the highest probability and associated with the first word in the first word group as the second word in the first word group. At the same time, the electronic device can continue to determine the probability of each second word being the third word in the first word group based on the first prompt word, and use the second word with the highest probability and associated with the second word in the first word group as the third word in the first word group. By analogy, for each word group, the electronic device can use a multi-head prediction module to simultaneously determine all the words in the word group that have the highest probability of appearing and are associated from the N second words.

[0100] Exemplarily, assuming that the number of heads of the multi-head prediction module is 5, and the number of word-grams in each word-gram group is also 5, taking the first prompt word as "self-introduction during an interview" as an example, the electronic device calculates the probability of each second word-gram being the first word-gram in the first word-gram group according to the first prompt word. Assume that the N second word-grams include: "Respect", "of", "everyone", "interviewer", "hello everyone", "I am", "Zhang San", "graduated from", "computer science", "major", "thank you very much", "company", "give me", "this time", "opportunity", "please allow me", "share", "my experience", "background" and other word-grams. The electronic device can use the multi-head prediction module to simultaneously calculate the probability of all word-grams in the N second word-grams, which are each of the 5 word-grams in the first word-gram group, according to the first prompt word "self-introduction during an interview".

[0101] (1) The first token in the first token group:

[0102] The electronic device uses a multi-head prediction module to calculate the probability of each second token as the first token according to the first prompt word "Self-introduction during an interview".

[0103] Suppose the following probabilities are obtained:

[0104] P(Respect) = 0.30

[0105] P(of) = 0.05

[0106] P(ladies and gentlemen) = 0.20

[0107] P(interviewer) = 0.25

[0108] P(Hello everyone) = 0.10

[0109] P(Permit me to) = 0.05

[0110] P(Share) = 0.05

[0111] The electronic device selects the token with the highest probability according to the probability. Among them, P(Respect) = 0.30 is the highest, so the first token is "Respect".

[0112] (2) The second token in the first token group:

[0113] The electronic device uses a multi-head prediction module to calculate the probability of each second token as the second token in the first token group according to the first prompt word "Self-introduction during an interview". At this time, the first token is "interviewer".

[0114] Suppose the following probabilities are obtained:

[0115] P(of) = 0.40

[0116] P(ladies and gentlemen) = 0.25

[0117] P(Hello everyone) = 0.20

[0118] P(Respect) = 0.05

[0119] P(Permit me to) = 0.05

[0120] P(Share) = 0.05

[0121] The electronic device selects the token with the highest probability according to the probability. Among them, P(of) = 0.40 is the highest, so the second token is "of".

[0122] (3) The third token in the first token group:

[0123] The electronic device uses a multi-head prediction module to calculate the probability of each second token as the third token in the first token group according to the first prompt word "self-introduction during an interview". At this time, the second token is "的".

[0124] Suppose the following probabilities are obtained:

[0125] P(各位) = 0.50

[0126] P(大家好) = 0.30

[0127] P(尊敬) = 0.10

[0128] P(请允许我) = 0.05

[0129] P(分享一下) = 0.05

[0130] The electronic device selects the token with the highest probability according to the probability. Among them, in this example, P(各位) = 0.50 is the highest, so the third token is "各位".

[0131] (4) The fourth token in the first token group:

[0132] The electronic device uses a multi-head prediction module to calculate the probability of each second token as the fourth token in the first token group according to the first prompt word "self-introduction during an interview". At this time, the third token is "各位".

[0133] Suppose the following probabilities are obtained:

[0134] P(尊敬) = 0.20

[0135] P(面试官) = 0.40

[0136] P(大家好) = 0.10

[0137] P(请允许我) = 0.10

[0138] P(分享一下) = 0.20

[0139] Select the token with the highest probability according to the probability.

[0140] In this example, P(面试官) = 0.40 is the highest, so the fourth token is "面试官".

[0141] (5) The fifth token in the first token group:

[0142] The electronic device uses a multi-head prediction module to calculate the probability of each second token as the fifth token in the first token group according to the first prompt word "self-introduction during an interview". At this time, the fourth token is "大家好".

[0143] Assume the following probabilities:

[0144] P(please allow me) = 0.20

[0145] P(Share)=0.10

[0146] P(hello everyone) = 0.40

[0147] P(Zhang San)=0.30

[0148] Based on the probability, the word-gram with the highest probability is selected.

[0149] In this example, P(hello everyone) = 0.40 is the highest, so the fifth word is "hello everyone".

[0150] The first word tuple generated by the electronic device is: "Dear", "of", "everyone", "interviewer", "hello everyone", and then the electronic device can use the multi-head prediction module to calculate all the words in the N second words at the same time according to the last word "hello everyone" in the first word tuple, which are the probabilities of each word in the 5 words of the second word tuple. For example, the 5 words in the second word tuple can be "I am", "Zhang San", "graduated from", "computer science", and "major". By analogy, the electronic device can continue to generate multiple word tuples to form at least one first word in order, such as "Dear interviewers, hello everyone. I am Zhang San, graduated from computer science. During college, I participated in many projects and won excellent awards. I believe that my ability can contribute to your company."

[0151] In this way, the electronic device can generate the probability distribution of multiple subsequent word units in parallel through the multi-head prediction module, so that multiple word units can be determined at the same time, thereby improving the efficiency of determining word units.

[0152] Optionally, in the embodiment of the present application, the text generation method provided in the embodiment of the present application also includes the following step 301.

[0153] Step 301: After determining the kth word in the i-th word group, if the number of determined first word-grams is greater than or equal to the quantity threshold, the electronic device determines the third probability of each second word-gram being the k+1th word-gram in the i-th word group according to the first prompt word and the first text through the multi-head prediction module, and takes the second word-gram with the highest third probability and associated with the kth word-gram among the N second word-grams as the k+1th word-gram.

[0154] In the embodiment of the present application, k is a positive integer.

[0155] In the embodiment of the present application, in order for the large model to generate the first text, the electronic device needs to input the token in the first text into the large model, so that the large model generates the specified text segment based on the input token. Figure 5 As shown, in addition to encoding the original input in the model input layer, this method also uses an embedding layer to encode the first text to obtain a vector representation of the first text. Based on the original large model, this method adds an activation classifier C and a connection layer M, where the input of the activation classifier C is the vector of the first text, and the output is 0 or 1. The output is 0, which means that the model does not need the information of the first text when decoding at the current step, and the output is 1, which means that the information of the first text is needed; the input of the connection layer M is the vector of the first text multiplied by the output of the activation classifier C (the dimension is still the dimension of the vector of the first text). The structure of the connection layer M can be a simple matrix or a transformer layer. The appropriate structure can be selected during the training process. It only needs to ensure that the output dimension is the same as the last layer vector of the large model. When the output of the activated classifier C is 0, it means that its input is all 0, and the large model will not use any information related to the first text; when the output of the activated classifier C is 1, the dimension of the vector of the first text can be converted into the dimension of the last layer vector of the large model through the trained connection layer M, and the two are added together to update the last layer vector, so that the large model can generate the first text in the subsequent process of determining the word unit.

[0156] Optionally, in an embodiment of the present application, in the process of the electronic device determining at least one first word element, the electronic device may record the number of determined word elements.

[0157] Optionally, in an embodiment of the present application, when the number of word units determined by the electronic device is less than a quantity threshold, the electronic device may determine that new text content needs to be generated, so that the electronic device may set the output of the activation classifier C to 0, so that the input of the connection layer M is the product of the vector of the first text and 0, and obtains a vector with all elements of the same dimension as the vector of the first text as 0. After the connection layer M obtains a vector of all 0s, when the electronic device uses the connection layer M to add this vector to the last layer vector of the large model, it may not affect the last layer vector of the large model, so that the electronic device can continue with the subsequent steps of text generation and generate new word units according to the new context and input through the large model. At this time, the generated content will be independent of the first text, thereby ensuring that the generated text is novel enough.

[0158] It should be noted that the large model will generate the probability of all vocabulary being sampled at the end of each inference. During inference, you can choose a variety of sampling methods according to the specific task. These methods can be called sampling strategies. For example, Top-K Sampling can only retain the top K most likely tokens, and then randomly sample from these K tokens. For example, Top-p Sampling can only retain tokens with a cumulative probability greater than a certain threshold, and then randomly sample from this set.

[0159] Optionally, in an embodiment of the present application, when the number of word units determined by the electronic device is less than a quantity threshold, the electronic device may determine that new text content needs to be generated, so that the electronic device may calculate the frequency of occurrence of each token in the first answer, and adopt a specific penalty coefficient strategy according to the token frequency distribution when the model is sampled to reduce the probability of the corresponding token being selected (which may not include single-word tokens and punctuation marks). Since the token that appears most frequently is likely to be strongly related to the topic, the electronic device may sort the tokens in the first answer from large to small according to the frequency of occurrence, and the penalty coefficient for the token with the median frequency is the highest, and the penalty coefficients for the remaining tokens decrease according to the frequency change. The penalty coefficient can be used as a hyperparameter for testing and adjustment according to specific tasks, so that the adjusted sampling strategy makes the text generated by the large model more meet the requirements of the difference from the first answer.

[0160] For example, suppose the tokens and corresponding probabilities obtained by the model when determining the token once are [interview: 0.45, at: 0.31, of course: 0.18, respect: 0.08, ...], and suppose the first answer is: "Self-introduction during an interview is the key to leaving a good first impression on the interviewer. A good self-introduction should be concise and to the point, while fully demonstrating your strengths and personality. The following is a basic self-introduction template, which you can adjust according to your actual situation: [Beginning] Dear interviewers, hello everyone! I am [your name], graduated from [your school], majoring in [your major]. Thank you very much [company name] for giving me this interview opportunity, allowing me to have the opportunity to show myself to you." The electronic device can count the first answer. The frequency of the token, where the highest token frequency is 4, the lowest is 1, and the median is 2 and 3. Assuming the penalty coefficient is 0.8, in order to improve the diversity of the newly generated text, the penalty coefficient of the token that appears 2 and 3 times in the frequency table is 0.8*0.8, and the penalty coefficient of the token that appears 1 and 4 times is 0.8. The adjusted token probability distribution is [interview: 0.36, in: 0.31, of course: 0.08, respected: 0.064,...]. The electronic device can then normalize the probability of the token and sample according to the modified probability. Therefore, through the adjustment of the penalty coefficient, the probability of the token "interview" is reduced, and the model is more likely to generate tokens that are different from the previous answer when sampling, such as "in".

[0161] Optionally, in an embodiment of the present application, when the number of word units determined by the electronic device is greater than or equal to a quantity threshold, the electronic device may determine that it is necessary to find an opportunity to insert the first text, so that the electronic device may set the output of the activation classifier C to 1, so that the input of the connection layer M is the product of the vector of the first text and 1, and the vector of the first text is obtained. After the connection layer M obtains the vector of the first text, the electronic device can use the connection layer M to add the vector of the first text to the last layer of the vector of the large model, so that the last layer of the vector of the large model can be affected, so that the electronic device can tend to generate the first text through the large model. At this time, the generated content will be affected by the first text, so that the large model can output the first text at the appropriate location.

[0162] Step 203: The electronic device outputs a second answer according to the at least one first word and the first text through a text generation model.

[0163] In the embodiment of the present application, the second answer is a new answer obtained by combining at least one first word generated by the electronic device with the first text through a text generation model. The second answer retains the part of the text that the user is satisfied with, and generates new content according to the context, so the second answer is an answer that better meets the user's needs.

[0164] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 6 As shown, the above step 203 can be specifically implemented through the following steps 203a and 203b.

[0165] Step 203a: The electronic device merges the generated first word-gram and the first text through the multi-head prediction module of the text generation model to obtain the first text.

[0166] Optionally, in an embodiment of the present application, in the process of generating at least one first word-gram, the electronic device may continuously generate first word-grams through a multi-head prediction module of a text generation model until the generated first word-grams are associated with the text content of the first text, so that the electronic device can determine the position of the first text in the at least one first word-gram.

[0167] Optionally, in an embodiment of the present application, the at least one first word-gram is divided into at least one word-gram group, and each word-gram group includes a part of the word-grams in at least one first word-gram. Figure 6 ,like Figure 7 As shown, the above step 203a can be specifically implemented through the following steps 203a1 and 203a2.

[0168] Step 203a1: The electronic device determines, through the multi-head prediction module, the fourth probability that the first a third word-grams of the first text are the word-grams in the b-th word-gram group.

[0169] In the embodiments of the present application, a is a positive integer, and b is a positive integer.

[0170] Optionally, in the embodiment of the present application, the above a can be the number of heads in the multi-head prediction module, that is, n+1, or the above a can also be less than the number of heads in the multi-head prediction module. In this way, the electronic device can simultaneously predict the probability of the first a third word-grams in the first text being each word-gram in the b-th word-gram group, and then calculate the fourth probability through these probabilities.

[0171] Optionally, in an embodiment of the present application, the electronic device can determine the probability of the N second word-grams being each word-gram in the b-th word-gram group, and then the electronic device can search for the word-grams in the first text from the N second word-grams, thereby obtaining the probability of these word-grams being the word-grams in the corresponding positions of the word-grams in the first text in the b-th word-gram groups, and then multiplying these probabilities to calculate the fourth probability that the first a third word-grams of the first text are word-grams in the b-th word-gram group.

[0172] Optionally, in an embodiment of the present application, for the first word in the bth word group, the electronic device can determine the probability that each word in the N second words is the first word in the bth word group, thereby knowing the probability that the first word in the first text is the first word in the bth word group; for the second word in the bth word group, the electronic device can determine the probability that each word in the N second words is the second word in the bth word group, thereby knowing the probability that the second word in the first text is the second word in the bth word group. Similarly, the electronic device can know the probability that the first a words in the first text are the ath words in the bth word group. Then, the electronic device can multiply these a probabilities to calculate the fourth probability.

[0173] For example, Figure 8 As shown, assuming that a is 5, for the first word in the b-th word tuple, assuming that the first text 12 is "Dear interviewers, hello everyone! I am [your name], graduated from [your school], and my major is [your major]." The electronic device can calculate the probability of the first 5 tokens in the first text 12, namely "Dear", "of", "everyone", "interviewer", and "hello everyone" as the first 5 words in the b-th word tuple respectively. Assume that the electronic device calculates that the probability of "respect" being the first word in the b-th word group is 0.71, the probability of "of" being the second word in the b-th word group is 0.26, the probability of "everyone" being the second word in the b-th word group is 0.67, the probability of "interviewer" being the second word in the b-th word group is 0.43, and the probability of "hello everyone" being the second word in the b-th word group is 0.74, then the electronic device can calculate that the probability of "respect", "of", "everyone", "interviewer", and "hello everyone" being the first 5 tokens in the b-th word group is P("respect")*P("of")*P("everyone")*P("interview")*P("official")=0.71×0.26×0.67×0.43×0.74=0.039, which is the fourth probability.

[0174] Step 203a2: When the fourth probability is greater than or equal to the probability threshold, the electronic device merges the generated first word-element and the first text through the multi-head prediction module to obtain a second text.

[0175] Optionally, in the embodiment of the present application, the above probability threshold may be a default value of the electronic device or a preset value by the user. For example, the above probability threshold may be 0.3, 0.4, 0.5, etc. The specific value may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0176] Optionally, in an embodiment of the present application, when the fourth probability is greater than or equal to the probability threshold, the electronic device determines that the position of the current token is suitable for placing the first text. The electronic device can merge the first text after the generated first word element through a multi-head prediction module.

[0177] For example, assuming that the probability threshold is 0.3, and assuming that the first text is "Dear interviewers, hello everyone! I am [your name], graduated from [your school], and my major is [your major]." Assuming that the electronic device calculates that the probability of the first five tokens "Dear", "of", "everyone", "interviewer", and "hello" in the first text being the first five tokens in the bth word tuple is 0.039, then the electronic device can determine that the current position is suitable for placing the first text. Assuming that the first word tuple that has been generated is: "Self-introduction during an interview is a key step in leaving a good first impression on the interviewer. A good self-introduction It should be concise and to the point, and fully demonstrate your strengths and personality. The following is a basic self-introduction template, which you can adjust according to your actual situation: ", the electronic device can merge the first text after the generated first word element, and get: "The self-introduction during the interview is the key to leaving a good first impression on the interviewer. A good self-introduction should be concise and to the point, and fully demonstrate your strengths and personality. The following is a basic self-introduction template, which you can adjust according to your actual situation: Dear interviewers, hello! I am [your name], graduated from [your school], and my major is [your major]."

[0178] In this way, the electronic device can calculate the probability that the token in the first text is the first word, calculate the probability of the position of each token in the first text, and ensure that the generated text will only be merged when the degree of semantic match with the first text is high, thereby improving the accuracy of the merge position and enhancing the coherence of the merged text.

[0179] Step 203b: The electronic device outputs a second answer based on the second text through the multi-head prediction module.

[0180] Optionally, in an embodiment of the present application, after the user selects the first text in the first answer, the electronic device can detect information in the first answer, including the number of the first text and its corresponding number of tokens and the number of other texts in the first answer other than the first text and the corresponding number of tokens. The electronic device can determine a generation requirement list based on this information and the first prompt word, and the generation requirement list is used by the electronic device to process the first answer. The electronic device can read and judge the generation requirement list in the order of the list, and generate the corresponding text according to the generation requirements in the generation requirement list.

[0181] For example, in combination Figure 2 ,like Fig. 9 As shown, after the user marks the first text 12, the first text 13, and the first text 14 in the first answer 11, the mobile phone can determine to generate a requirement list 15, which includes a first prompt word and 7 generation requirements. The 7 generation requirements are respectively regenerate text segment-1 (token number: 42), retain text segment-1 (token number: 42), regenerate text segment-2 (token number: 53), retain text segment-2 (token number: 31), regenerate text segment-3 (token number: 105), retain text segment-3 (token number: 34), and regenerate text segment-4 (token number: 28). The content of the generation requirement list 15 is as follows:

[0182] {“User input question”: “Self-introduction during interview”,

[0183] "Generate Requirements List":[

[0184] "Regenerate text fragment-1 (token number: 42): The self-introduction during the interview is the key to leaving a good first impression on the interviewer. A good self-introduction should be concise and to the point, while fully demonstrating your strengths and personality. The following is a basic self-introduction template, which you can adjust according to your actual situation: [beginning]",

[0185] "Retained text fragment-1 (token number: 42): Dear interviewers, hello! I am [your name], graduated from [your school], majoring in [your major]. Thank you very much [company name] for giving me this interview opportunity, allowing me to have the opportunity to show myself to you.",

[0186] "Regenerate text fragment-2 (token number: 53): [Educational background] During college, I systematically studied [relevant courses] and actively participated in [relevant projects or activities]. Through these studies and practices, I not only mastered solid professional knowledge, but also developed strong learning ability and teamwork spirit. [Work experience] (if you have relevant work or internship experience)",

[0187] “Retained text fragment-2 (token number: 31): During my internship at [internship company], I was mainly responsible for [specific work content], during which I learned [learned skills] and achieved [specific results].”,

[0188] "Regenerate text fragment-3 (token number: 105): These experiences have given me a deeper understanding of [position name] and have confirmed my career development direction in this field. [Personal advantages] I think I have the following advantages: First, I have strong [certain skills or qualities] and can quickly adapt to new environments and new tasks; second, I am good at communication and teamwork, and can effectively cooperate with colleagues from different backgrounds; finally, I take my work seriously and responsibly, pay attention to details, and pursue excellence. [Expectations for future work] I am full of enthusiasm and expectations for [company name] and [position name]. ",

[0189] “Retained text fragment-3 (token number: 34): I believe that my professional background and work experience can bring new vitality to the team. If I am lucky enough to join [company name], I will work hard, constantly improve myself, and contribute my own strength to the development of the company.”,

[0190] "Regenerate text fragment-4 (token number: 28): [End] Thank you again to all the interviewers for giving me this opportunity, and I look forward to further communication with you. Thank you! I hope this template will be helpful to you. I wish you a good interview! "]

[0191] }

[0192] The mobile phone can read and judge the generation requirement list in the order of the list, and start to regenerate the text according to the regenerated text fragment-1 (token number: 42) in the generation requirement list 15 to obtain the newly generated text 16 "In an interview, self-introduction is a very important part. A decent and wonderful self-introduction can leave a good first impression on the interviewer. The following is a template for a self-introduction, which you can adjust according to your actual situation: ", and the mobile phone can judge that the first text 12 can be placed according to the retained text fragment-1 (token number: 42) in the generation requirement list 15, and then merge the first text 12 after the newly generated text 16.

[0193] The mobile phone can then start regenerating text based on the regenerated text fragment-2 (token number: 53) to obtain the newly generated text 17 "Personal background: I graduated from [your university], majoring in [your major]. I had excellent grades while in school. At the same time, I focused on studying in the field of [field of the interview company] and have internship experience in companies in related industries." In addition, the mobile phone can determine that the first text 13 can be placed based on the retained text fragment-2 (token number: 31) in the generation requirement list 15, and then merge the first text 13 after the newly generated text 17.

[0194] By analogy, the mobile phone generates a new text 18 based on the regenerated text fragment-3 (token number: 105) and merges the first text 14 into the new text 18, and finally generates a new text 19 based on the regenerated text fragment-4 (token number: 28) to obtain the second answer 20.

[0195] In this way, the electronic device can merge the generated first word with the first text through the multi-head prediction module, ensuring that the generated first text not only contains the content that the user wants to retain, but is also consistent with the context, thereby making the generated text more semantically coherent and natural.

[0196] In an embodiment of the present application, the electronic device can adjust the output answer according to the text requirements circled by the user. By adjusting the structure of the large model, the two-stage reasoning decoding strategy and the multi-head generation mechanism, the large model can retain the user-specified text fragments in the corresponding appropriate positions when generating new answers. At the same time, the content generated by the remaining text fragments is diversified, which improves the difference between the answers and the previous answers, thereby improving user satisfaction.

[0197] In the embodiment of the present application, the electronic device can be universally applicable to various other text generation tasks such as text rewriting, creative copywriting, writing, copywriting, etc. The position of the retained text fragment in the scheme is strictly in accordance with its position in the original answer. On this basis, the user can also specify any position of the retained text fragment in the new answer, which can be achieved by adding training data to optimize the activation classifier.

[0198] The embodiment of the present application provides a text generation method. Since the electronic device can generate at least one first word-element according to a first prompt word and a first answer, that is, generate new text content according to the first prompt word and the answer generated based on the first prompt word, and then output a second answer according to the at least one first word-element and the first text through a text generation model, that is, combine the new text content with the part of the answer generated based on the first prompt word that satisfies the user to obtain the second answer, so that while generating a new answer different from the first answer according to the first prompt word, the part that satisfies the user is retained, thereby improving the probability of generating an answer that satisfies the user, thereby improving the flexibility of the text generation method.

[0199] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Fig.10 As shown, before the above step 202, the text generation method provided in the embodiment of the present application also includes the following steps 401 to 404.

[0200] Step 401: The electronic device obtains a first sample prompt word and a first sample answer.

[0201] In an embodiment of the present application, the first sample answer is generated based on a first sample prompt word, and the first sample answer includes a sample first text.

[0202] Optionally, in the embodiment of the present application, the first sample prompt words may be prompt words with a sufficient number of samples, and the first sample answers may be sample answers generated and marked for each prompt word. For example, there are 5,000 first sample prompt words and 5,000 first sample answers, and the first sample prompt words and the first sample answers correspond one to one.

[0203] It should be noted that the specific number of first sample prompt words and first sample answers can be determined according to actual needs in the model training process, and this application does not limit it here.

[0204] Optionally, in an embodiment of the present application, the electronic device may construct a data set to train the parameters of the activation classifier C and the connection layer M. The method for constructing the data set is as follows:

[0205] (1) First, a certain number of prompts are input into the big model, and the output of the big model is recorded to construct a batch of training data. For example, the input for "global climate change" can be obtained ["input": "global climate change", "output": "global climate change refers to long-term changes in the earth's climate system caused by human activities and natural factors. These changes may manifest as rising temperatures, changes in precipitation patterns, and rising sea levels. Climate change has a profound impact on ecosystems, economies, and societies, leading to frequent extreme weather events and affecting agricultural production and water resources management. Countries are working hard to meet the challenges of climate change through emission reduction and sustainable development strategies."]

[0206] (2) The electronic device can then randomly retain some text fragments of the answer output by the large model and randomly change the remaining text fragments of the answer, for example, by using various processing methods such as abbreviation and rewriting, to construct corresponding training data for the above-mentioned examples:

[0207] [“input”: “global climate change”,

[0208] "Number of tokens in each text segment": [Newly generated text segment (number of tokens - 35), (Specified text segment: number of tokens - 35), (Newly generated text segment: number of tokens - 17)],

[0209] "Specify text fragment", "Climate change has profound impacts on ecosystems, economies and societies, leading to frequent extreme weather events and affecting agricultural production and water resource management.",

[0210] Output: "Global climate change refers to long-term changes in the Earth's climate system due to human activities and natural factors. These changes may manifest themselves in rising temperatures, changes in precipitation patterns, and rising sea levels. Climate change has profound impacts on ecosystems, economies, and societies, leading to frequent extreme weather events and affecting agricultural production and water resource management. Countries are working to address the challenges of climate change through emission reduction and sustainable development strategies."]

[0211] Optionally, in an embodiment of the present application, the electronic device may randomly initialize parameters when training the parameters of the activation classifier C and the connection layer M, and determine whether the activation classifier C and the connection layer M need to be used according to the current decoding position.

[0212] Optionally, in the embodiment of the present application, the electronic device may train the activation classifier C and the connection layer M in two stages, and distinguish the text segments by the number of tokens recorded in the generation requirement list. The specific training method is as follows:

[0213] Exemplarily, assume that the token length of the newly generated text segment is n, and assume that the preset threshold is 2. The electronic device needs to determine whether it is necessary to generate a certain length of new text through the model first. Therefore, when the token step corresponding to the model < n / 2, the electronic device can set the output of the activation classifier to 0, so that the output result of the connection layer M is all 0, making the result of the superposition and summation of the output result and the last layer vector of the model unchanged, that is, the current model does not need to specify the information of the token sequence of the text during decoding, so as to directly generate new text by the model. Assume that the number of tokens in the currently newly generated text segment is 35, and the current model input is "Global climate change: Global climate change is". The token length generated by the model ("Global", "climate", "change", "is") is 4, which is less than 35 / 2. Therefore, the electronic device can set all the parameters of the connection layer M to 0, and the model samples and generates the token "impact" at the current step.

[0214] When the token step corresponding to the model >= n / 2, at this time, the electronic device can enable the activation classifier C and the connection layer M through the model, and judge whether to input the vector of the specified text segment at the current position by training the activation classifier C. If the electronic device determines that the vector of the specified text segment is input at the current position, the electronic device can multiply the encoded vector of the specified text segment by the connection layer and then add it to the last layer vector parameter, so that the model can accept the subsequent token sequence information during decoding and output the specified text segment at the appropriate position. Assume that the current input is "Global climate change: Global climate change is caused by long-term changes in the Earth's climate system due to human activities and natural factors. These changes may manifest as rising temperatures, altered precipitation patterns, rising sea levels, etc.". The currently generated token length is 35, which is greater than 35 / 2. Through training iteration, at this time, the electronic device can determine through the activation classifier C that the model can accept new information at this position. Therefore, the electronic device can use the Embedding layer to encode "Climate change has a profound impact on ecosystems, economies and societies, leading to frequent extreme weather events and affecting agricultural production and water resource management." as the vector of the specified text segment, then use the connection layer M to add it to the last layer vector representation, and finally make a prediction through the LM head. Affected by the vector of the specified text segment, the model will tend to output the reserved text segment and output the token "impact" at the current step.

[0215] In this way, the electronic device can update the parameters of the activation classifier C and the connection layer M of the large model. Through such training, the model can learn to retain the given text at the appropriate position and generate new text at the remaining positions when given text information, and the model output is complete and fluent.

[0216] Step 402: The electronic device inputs a first sample prompt word and a first sample answer into a preset model, and generates at least one first sample word unit according to the first sample prompt word and the first sample answer through the preset model.

[0217] Optionally, in an embodiment of the present application, the composition of the above-mentioned preset model is substantially the same as the composition of the above-mentioned text generation model. For details, please refer to the description of the above-mentioned embodiment, and the embodiment of the present application will not be repeated here.

[0218] Optionally, in the embodiment of the present application, combined with Fig.10 ,like Fig.11 As shown, the above step 402 can be implemented through the following steps 402a to 402c.

[0219] Step 402a: The electronic device inputs the first sample prompt word and the first sample answer into a preset module of a preset model, and obtains N second sample word units based on the first sample answer through the preset module.

[0220] In the embodiment of the present application, the above-mentioned N second sample word-grams are N possible candidate word-grams predicted by a preset model based on the text content of the first sample answer.

[0221] Optionally, in an embodiment of the present application, the configuration of the above-mentioned preset module is substantially the same as the configuration of the above-mentioned multi-head prediction module. For details, please refer to the description of the above-mentioned embodiment, and the embodiment of the present application will not be repeated here.

[0222] Step 402b: The electronic device determines at least one first sample word-unit from the N second sample word-units according to the first sample prompt word and the N second sample word-units through a preset module.

[0223] In the embodiment of the present application, the at least one first sample word-unit is a sample word-unit related to the first sample prompt word determined by the electronic device from among the N candidate second sample word-units.

[0224] Step 402c: The electronic device trains a preset module according to at least one first sample word-gram and at least one target word-gram to obtain a multi-head prediction module.

[0225] Optionally, in an embodiment of the present application, the electronic device can add n add heads to predict subsequent tokens respectively on the basis of retaining the original LMHead in the last hidden layer of the large model. The structure and parameters of the newly added head are consistent with the original LM Head. The first add head predicts the next token of the next token, and the subsequent add heads are analogous. During the training process, the original model parameters will be frozen, and only the parameters of the add heads (i.e., the above-mentioned multi-head prediction module) will be trained and updated. Through such changes, the large model can obtain the probability distribution of the subsequent n+1 tokens in one reasoning. For example, the large model has 4 additional add heads. Through model training, the large model can generate the probability distribution of the subsequent 5 tokens as much as possible during each reasoning. For the input of "global climate change: global climate change is", the original LM head predicts the token "influence", and the add heads predict the subsequent "of", "one", "factor" and "cause".

[0226] Optionally, in an embodiment of the present application, the electronic device may calculate a first loss value based on at least one first sample word-gram and at least one target word-gram.

[0227] Optionally, in an embodiment of the present application, the first loss value may represent at least one of the following differences between the first sample word-gram and at least one target word-gram: a difference in semantic features, a difference in contextual consistency, a difference in the degree of matching with the first sample prompt word, etc.

[0228] Optionally, in an embodiment of the present application, the electronic device may adjust module parameters in a preset module based on the first loss value to obtain a multi-head prediction module.

[0229] It should be noted that the smaller the difference between the at least one first sample word-gram and the at least one target word-gram is, the closer the at least one first sample word-gram is to the answer to the user's requirement, and the smaller the first loss value is.

[0230] Optionally, in an embodiment of the present application, the electronic device may use a back propagation algorithm to iteratively update the parameters in the preset module based on the first loss value until convergence, so as to train a multi-head prediction module.

[0231] In this way, the electronic device can continuously calculate the loss value through at least one first sample word and at least one target word, and adjust the module parameters of the preset module according to the loss value, so as to train the preset module into a multi-head prediction module.

[0232] Step 403: The electronic device outputs a second sample answer according to at least one first sample word and a sample first text through a preset model.

[0233] In the embodiment of the present application, the second sample answer is an answer obtained by combining at least one first sample word element generated by the electronic device through a text generation model with the sample first text.

[0234] Step 404: The electronic device trains a preset model according to the second sample answer and the target answer to obtain a text training model.

[0235] Optionally, in an embodiment of the present application, the electronic device may calculate a second loss value based on the second sample answer and the target answer.

[0236] Optionally, in an embodiment of the present application, the second loss value may characterize at least one of the following differences between the second sample answer and the target answer: a difference in semantic features, a difference in contextual consistency, a difference in the degree of matching with the first sample prompt word, etc.

[0237] Optionally, in an embodiment of the present application, the electronic device may adjust model parameters in a preset model based on the second loss value to obtain a text generation model.

[0238] It should be noted that the smaller the difference between the second sample answer and the target answer, the closer the second sample answer is to the answer required by the user, and the smaller the second loss value is.

[0239] Optionally, in an embodiment of the present application, the electronic device may use a back propagation algorithm to iteratively update the model parameters in the preset model based on the second loss value until convergence, so as to train a text generation model.

[0240] Optionally, in an embodiment of the present application, the electronic device can add an activation classifier C, a connection layer M, and an Add heads module to the original structure of the large model, freeze the parameters of the original model when training and updating the model parameters, and only update the parameters of these three modules, thereby decoupling the model structure while avoiding affecting the effect of the original large model.

[0241] In this way, the electronic device can continuously calculate the loss value through the second sample answer and the target answer, and adjust the model parameters of the preset model according to the loss value, so as to train the preset model into a text generation model.

[0242] The present application embodiment provides a text generation method, Fig.12 FIG. 1 is a flowchart of a text generation method provided by an embodiment of the present application, which can be applied to electronic devices. Fig.12 As shown, the text generation method provided in the embodiment of the present application may include the following steps 10 to 16.

[0243] Step 10: The electronic device receives a user input of a first text in a first answer.

[0244] Step 11: The electronic device generates a requirement list.

[0245] Optionally, in an embodiment of the present application, the electronic device may separately mark the first text in the first answer and the non-first text in the first answer other than the first text, and respectively calculate the number of tokens in each text fragment in the first text and the non-first text, and then organize these text fragments in the order in the first answer to generate a requirement list.

[0246] Step 12: The electronic device determines whether the generation requirement list is empty.

[0247] When the generation requirement list is not empty, the electronic device executes the following step 13 ; when the generation requirement list is empty, the electronic device executes the following step 16 .

[0248] Step 13: The electronic device determines whether the current text segment is to be regenerated.

[0249] When the electronic device determines that the current text segment is regenerated, the electronic device executes the following step 14 ; when the electronic device determines that the current text segment is not regenerated, the electronic device executes the following step 15 .

[0250] Optionally, in an embodiment of the present application, the electronic device may process each text segment in sequence according to the order of generating the text segments in the requirement list.

[0251] Optionally, in an embodiment of the present application, for non-first text, the electronic device can generate a new text segment; for the first text, the electronic device can determine a position suitable for splicing the first text during the process of generating the first text, and splice the first text to the end of the newly generated text at this position.

[0252] Step 14: The electronic device generates a new text segment using the text generation model.

[0253] Step 15: The electronic device splices the first text.

[0254] Optionally, in an embodiment of the present application, when the first text segment in the generation requirement list is the first text, the electronic device may directly splice the first text at the position of the first token.

[0255] Step 16: The electronic device outputs the generated text as the second answer.

[0256] Optionally, in an embodiment of the present application, after executing steps 13, 14 and 15 multiple times, the electronic device can process all text fragments in the generation requirement list according to their corresponding processing methods, and finally obtain a second answer containing the first text and the newly generated text.

[0257] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0258] The text generation method provided in the embodiment of the present application can be executed by a text generation device. In the embodiment of the present application, the text generation device provided in the embodiment of the present application is described by taking the text generation method executed by the text generation device as an example.

[0259] Fig.13 A possible structural diagram of a text generation device involved in some embodiments of the present application is shown. Fig.13 As shown, the text generating device 70 may include: an acquisition module 71 and an execution module 72 .

[0260] The acquisition module 71 is used to acquire a first prompt word and a first answer. The first answer is generated based on the first prompt word, and the first answer includes a first text.

[0261] The execution module 72 is used to input the first prompt word and the first answer obtained by the obtaining module 71 into the text generation model, and generate at least one first word-unit according to the first prompt word and the first answer through the text generation model.

[0262] The execution module 72 is further configured to output a second answer according to at least one first word and the first text through a text generation model.

[0263] In a possible implementation, for example, combining Fig.13 ,like Fig.14 As shown, the text generation device provided by the embodiment of the present application also includes: a determination module 73. The execution module 72 is specifically used to input the first prompt word and the first answer obtained by the acquisition module 71 into the multi-head prediction module of the text generation model, and obtain N second word units based on the first answer through the multi-head prediction module, where N is a positive integer. The determination module 73 is used to determine at least one first word unit from the N second word units according to the first prompt word obtained by the acquisition module 71 and the N second word units obtained by the execution module 72 through the multi-head prediction module.

[0264] In a possible implementation, the at least one first word-gram is divided into at least one word-gram group, and each word-gram group includes part of the word-grams in at least one first word-gram. The determination module 73 is specifically used to: for the i-th word-gram group in at least one word-gram group, determine the first probability of each second word-gram as the first word-gram in the i-th word-gram group through the multi-head prediction module according to the first prompt word obtained by the acquisition module 71, and use the second word-gram with the highest first probability among the N second word-grams obtained by the acquisition module 71 as the first word-gram in the i-th word-gram group. And, determine the second probability of each second word-gram as the j-th word-gram in the i-th word-gram group through the multi-head prediction module according to the first prompt word obtained by the acquisition module 71, and use the second word-gram with the highest second probability and associated with the j-1-th word-gram in the i-th word-gram group as the j-th word-gram in the i-th word-gram group. Wherein, i is a positive integer, and j is a positive integer greater than 1.

[0265] In a possible implementation, the determination module 73 is further configured to, after determining the kth word in the ith word group, if the number of determined first words is greater than or equal to the quantity threshold, determine the third probability of each second word being the k+1th word in the ith word group according to the first prompt word and the first text through the multi-head prediction module, and use the second word with the highest third probability and associated with the kth word among the N second words as the k+1th word, where k is a positive integer.

[0266] In one possible implementation, the execution module 72 is specifically used to: merge the generated first word element and the first text acquired by the acquisition module 71 through the multi-head prediction module of the text generation model to obtain a first text; and output a second answer based on the second text through the multi-head prediction module.

[0267] In a possible implementation, the at least one first word-gram is divided into at least one word-gram group, and each word-gram group includes a part of the word-grams in at least one first word-gram. The execution module 72 is specifically used to: determine, through the multi-head prediction module, the fourth probability that the first a third word-grams of the first text acquired by the acquisition module 71 are used as the word-grams in the b-th word-gram group, where a is a positive integer and b is a positive integer; and, when the fourth probability is greater than or equal to the probability threshold, merge the generated first word-gram and the first text through the multi-head prediction module.

[0268] In a possible implementation, the acquisition module 71 is also used to acquire a first sample prompt word and a first sample answer before inputting the first prompt word and the first answer into the text generation model and generating at least one first word-unit according to the first prompt word and the first answer through the text generation model, wherein the first sample answer is generated based on the first sample prompt word, and the first sample answer includes a sample first text. The execution module 72 is also used to: input the first sample prompt word acquired by the acquisition module 71 and the first sample answer acquired by the acquisition module 71 into a preset model, and generate at least one first sample word-unit according to the first sample prompt word and the first sample answer through the preset model; output a second sample answer according to the at least one first sample word-unit and the sample first text through the preset model; and train the preset model according to the second sample answer and the target answer to obtain a text training model.

[0269] In a possible implementation, the device further includes: a determination module 73. The acquisition module 71 is specifically used to input the first sample prompt word and the first sample answer into a preset module of a preset model, and obtain N second sample word-grams based on the first sample answer through the preset module. The determination module 73 is used to determine at least one first sample word-gram from the N second sample word-grams through the preset module according to the first sample prompt word and the N second sample word-grams obtained by the acquisition module 71. The execution module 72 is also used to train the preset module according to the at least one first sample word-gram and the at least one target word-gram determined by the determination module 73 to obtain a multi-head prediction module.

[0270] In an embodiment of the present application, a text generation device is provided. Since the text generation device can generate at least one first word-gram according to a first prompt word and a first answer, that is, generate new text content according to the first prompt word and the answer generated based on the first prompt word, and then output a second answer according to the at least one first word-gram and the first text through a text generation model, that is, combine the new text content with the part of the answer generated based on the first prompt word that satisfies the user to obtain the second answer, so that while generating a new answer different from the first answer according to the first prompt word, the part that satisfies the user is retained, thereby improving the probability of generating an answer that satisfies the user, thereby improving the flexibility of the text generation method.

[0271] The text generation device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtualreality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0272] The text generation device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0273] The text generation device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0274] Alternatively, if Fig.15 As shown, an embodiment of the present application further provides an electronic device 1000, including a processor 1001 and a memory 1002, wherein the memory 1002 stores a program or instruction that can be executed on the processor 1001, and when the program or instruction is executed by the processor 1001, each step of the above-mentioned text generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0275] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0276] Fig.16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0277] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0278] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.16 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0279] The processor 110 is used to obtain a first prompt word and a first answer. The first answer is generated based on the first prompt word, and the first answer includes a first text.

[0280] The processor 110 is further configured to input the first prompt word and the first answer acquired by the processor 110 into a text generation model, and generate at least one first word-gram according to the first prompt word and the first answer through the text generation model.

[0281] The processor 110 is further configured to output a second answer according to at least one first word and the first text through a text generation model.

[0282] Optionally, the processor 110 is specifically configured to input the first prompt word and the first answer into a multi-head prediction module of a text generation model, and obtain N second word-grams based on the first answer through the multi-head prediction module, where N is a positive integer. The processor 110 is configured to determine at least one first word-gram from the N second word-grams according to the first prompt word and the N second word-grams through the multi-head prediction module.

[0283] Optionally, the at least one first word-gram is divided into at least one word-gram group, and each word-gram group includes part of the words in at least one first word-gram. The processor 110 is specifically used to: for the i-th word-gram in at least one word-gram group, determine the first probability of each second word-gram being the first word-gram in the i-th word-gram group according to the first prompt word through the multi-head prediction module, and use the second word-gram with the highest first probability among the N second word-grams as the first word-gram in the i-th word-gram group. And, determine the second probability of each second word-gram being the j-th word-gram in the i-th word-gram group according to the first prompt word through the multi-head prediction module, and use the second word-gram with the highest second probability among the N second word-grams and associated with the j-1-th word-gram in the i-th word-gram group as the j-th word-gram in the i-th word-gram group. Wherein, i is a positive integer, and j is a positive integer greater than 1.

[0284] Optionally, the processor 110 is further configured to, after determining the kth word in the ith word group, if the number of determined first words is greater than or equal to the quantity threshold, determine the third probability of each second word being the k+1th word in the ith word group according to the first prompt word and the first text through the multi-head prediction module, and use the second word with the highest third probability and associated with the kth word among the N second words as the k+1th word, where k is a positive integer.

[0285] Optionally, the processor 110 is specifically used to: merge the generated first word and the first text through the multi-head prediction module of the text generation model to obtain the first text; and output the second answer based on the second text through the multi-head prediction module.

[0286] Optionally, the at least one first word-gram is divided into at least one word-gram group, each word-gram group includes a part of the word-grams in at least one first word-gram. The processor 110 is specifically used to: determine, through a multi-head prediction module, a fourth probability that the first a third word-grams of the first text are used as the word-grams in the b-th word-gram group, where a is a positive integer and b is a positive integer; and, when the fourth probability is greater than or equal to a probability threshold, merge the generated first word-grams and the first text through the multi-head prediction module.

[0287] Optionally, the processor 110 is further used to obtain a first sample prompt word and a first sample answer before inputting the first prompt word and the first answer into the text generation model and generating at least one first word-gram according to the first prompt word and the first answer through the text generation model, wherein the first sample answer is generated based on the first sample prompt word, and the first sample answer includes a sample first text. The processor 110 is further used to: input the first sample prompt word and the first sample answer into a preset model, generate at least one first sample word-gram according to the first sample prompt word and the first sample answer through the preset model; output a second sample answer according to the at least one first sample word-gram and the sample first text through the preset model; train the preset model according to the second sample answer and the target answer to obtain a text training model.

[0288] Optionally, the device further includes: a processor 110. The processor 110 is specifically used to input the first sample prompt word and the first sample answer into a preset module of a preset model, and obtain N second sample word-grams based on the first sample answer through the preset module. The processor 110 is used to determine at least one first sample word-gram from the N second sample word-grams according to the first sample prompt word and the N second sample word-grams through the preset module. The processor 110 is also used to train the preset module according to at least one first sample word-gram and at least one target word-gram to obtain a multi-head prediction module.

[0289] An embodiment of the present application provides an electronic device. Since the electronic device can generate at least one first word-element according to a first prompt word and a first answer, that is, generate new text content according to the first prompt word and the answer generated based on the first prompt word, and then output a second answer according to the at least one first word-element and the first text through a text generation model, that is, combine the new text content with the part of the answer generated based on the first prompt word that satisfies the user to obtain the second answer, so that while generating a new answer different from the first answer according to the first prompt word, the part that satisfies the user is retained, thereby improving the probability of generating an answer that satisfies the user, thereby improving the flexibility of the text generation method.

[0290] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and can achieve the same technical effect. To avoid repetition, it is not repeated here. The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, it is not repeated here.

[0291] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0292] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0293] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0294] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned text generation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0295] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0296] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned text generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0297] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0298] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned text generation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0299] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0300] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0301] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A text generation method, characterized in that: include: Obtaining a first prompt word and a first answer, wherein the first answer is generated based on the first prompt word, and the first answer includes a first text; Inputting the first prompt word and the first answer into a text generation model, and generating at least one first word-gram according to the first prompt word and the first answer through the text generation model; A second answer is outputted according to the at least one first word-gram and the first text by the text generation model.

2. The method according to claim 1, characterized in that The step of inputting the first prompt word and the first answer into a text generation model, and generating at least one first word-gram according to the first prompt word and the first answer by the text generation model, comprises: Inputting the first prompt word and the first answer into the multi-head prediction module of the text generation model, and obtaining N second word-units based on the first answer through the multi-head prediction module, where N is a positive integer; The at least one first word-gram is determined from the N second word-grams by the multi-head prediction module according to the first prompt word and the N second word-grams.

3. The method according to claim 2, characterized in that The at least one first word-gram is divided into at least one word-gram group, each word-gram group includes a part of the word-grams in the at least one first word-gram; The determining, by the multi-head prediction module, the at least one first word-gram from the N second word-grams according to the first prompt word and the N second word-grams comprises: For the i-th word-tuple in the at least one word-tuple, determine, by the multi-head prediction module, a first probability that each second word-tuple is the first word-tuple in the i-th word-tuple according to the first prompt word, and use the second word-tuple with the highest first probability among the N second word-tuples as the first word-tuple in the i-th word-tuple; Determine, by the multi-head prediction module, the second probability of each second word-gram being the j-th word-gram in the i-th word-gram group according to the first prompt word, and use the second word-gram with the highest second probability among the N second word-grams and associated with the j-1-th word-gram in the i-th word-gram group as the j-th word-gram in the i-th word-gram group; Wherein, i is a positive integer, and j is a positive integer greater than 1.

4. The method according to claim 3, characterized in that The method further comprises: After determining the kth word-gram in the i-th word-gram group, if the number of the determined first word-grams is greater than or equal to the number threshold, the multi-head prediction module determines the third probability of each second word-gram being the k+1th word-gram in the i-th word-gram group according to the first prompt word and the first text, and uses the second word-gram with the highest third probability and associated with the kth word-gram among the N second word-grams as the k+1th word-gram; Wherein, k is a positive integer.

5. The method according to claim 1, characterized in that Outputting a second answer according to the at least one first word-gram and the first text by the text generation model includes: Merging the generated first word-gram and the first text by the multi-head prediction module of the text generation model to obtain a second text; The second answer is outputted according to the second text by the multi-head prediction module.

6. The method according to claim 5, characterized in that The at least one first word-gram includes at least one word-gram group, each word-gram group includes a part of the word-grams in the at least one first word-gram; Merging the generated first word-element and the first text by the multi-head prediction module of the text generation model includes: Determine, by the multi-head prediction module, a fourth probability that the first a third word-grams of the first text are word-grams in the b-th word-gram group, where a is a positive integer and b is a positive integer; When the fourth probability is greater than or equal to the probability threshold, the multi-head prediction module merges the generated first word-gram and the first text.

7. The method according to claim 1, characterized in that Before inputting the first prompt word and the first answer into a text generation model and generating at least one first word-gram according to the first prompt word and the first answer through the text generation model, the method further includes: Obtaining a first sample prompt word and a first sample answer, wherein the first sample answer is generated based on the first sample prompt word, and the first sample answer includes a sample first text; Inputting the first sample prompt word and the first sample answer into a preset model, and generating at least one first sample word-unit according to the first sample prompt word and the first sample answer through the preset model; Outputting a second sample answer according to the at least one first sample word and the sample first text through the preset model; The preset model is trained according to the second sample answer and the target answer to obtain the text training model.

8. The method according to claim 7, characterized in that The step of inputting the first sample prompt word and the first sample answer into a preset model, and generating at least one first sample word-unit according to the first sample prompt word and the first answer through the preset model, comprises: Inputting the first sample prompt word and the first sample answer into a preset module of the preset model, and obtaining N second sample word-units based on the first sample answer through the preset module; Determining, by the preset module, the at least one first sample word-gram from the N second sample word-grams according to the first sample prompt word and the N second sample word-grams; The preset module is trained according to the at least one first sample word-gram and the at least one target word-gram to obtain the multi-head prediction module.

9. A text generation device, characterized in that: include: Get module and execute module; The acquisition module is used to acquire a first prompt word and a first answer, wherein the first answer is generated based on the first prompt word, and the first answer includes a first text; The execution module is used to input the first prompt word and the first answer acquired by the acquisition module into a text generation model, and generate at least one first word-unit according to the first prompt word and the first answer through the text generation model; The execution module is further configured to output a second answer according to the at least one first word and the first text through the text generation model.

10. The device according to claim 9, characterized in that The device further comprises: a determination module; The execution module is specifically configured to input the first prompt word and the first answer acquired by the acquisition module into the multi-head prediction module of the text generation model, and acquire N second word-units based on the first answer through the multi-head prediction module, where N is a positive integer; The determination module is used to determine the at least one first word-gram from the N second word-grams according to the first prompt word obtained by the acquisition module and the N second word-grams obtained by the execution module through the multi-head prediction module.

11. The device according to claim 10, characterized in that The at least one first word-gram is divided into at least one word-gram group, each word-gram group includes a part of the word-grams in the at least one first word-gram; The determination module is specifically used for: For the i-th word-tuple in the at least one word-tuple, determine, by the multi-head prediction module, a first probability that each second word-tuple is the first word-tuple in the i-th word-tuple according to the first prompt word obtained by the obtaining module, and use the second word-tuple with the highest first probability among the N second word-tuples obtained by the obtaining module as the first word-tuple in the i-th word-tuple; as well as, Determine, by the multi-head prediction module, the second probability of each second word-gram being the j-th word-gram in the i-th word-gram group according to the first prompt word obtained by the obtaining module, and use the second word-gram with the highest second probability and associated with the j-1-th word-gram in the i-th word-gram group among the N second word-grams obtained by the obtaining module as the j-th word-gram in the i-th word-gram group; Wherein, i is a positive integer, and j is a positive integer greater than 1.

12. The device according to claim 11, characterized in that The determination module is further configured to, after determining the kth word in the i-th word group, if the number of determined first words is greater than or equal to a quantity threshold, determine, through the multi-head prediction module, a third probability of each second word being the k+1th word in the i-th word group according to the first prompt word and the first text, and use the second word with the highest third probability and associated with the kth word among the N second words as the k+1th word; Wherein, k is a positive integer.

13. The device according to claim 9, characterized in that The execution module is specifically used for: In the process of generating the at least one first word-gram acquired by the acquisition module, the multi-head prediction module of the text generation model merges the generated first word-gram and the first text acquired by the acquisition module to obtain a second text; and The second answer is outputted according to the second text by the multi-head prediction module.

14. The device according to claim 13, characterized in that The at least one first word-gram is divided into at least one word-gram group, each word-gram group includes a part of the word-grams in the at least one first word-gram; The execution module is specifically used for: Determine, by the multi-head prediction module, a fourth probability that the first a third word-grams of the first text acquired by the acquisition module are used as the word-grams in the b-th word-gram group, where a is a positive integer and b is a positive integer; as well as, When the fourth probability is greater than or equal to the probability threshold, the multi-head prediction module merges the generated first word-gram and the first text.

15. The device according to claim 9, characterized in that The acquisition module is further configured to acquire a first sample prompt word and a first sample answer before inputting the first prompt word and the first answer into the text generation model and generating at least one first word-gram according to the first prompt word and the first answer through the text generation model, wherein the first sample answer is generated based on the first sample prompt word and includes a sample first text; The execution module is further used for: Inputting the first sample prompt word obtained by the obtaining module and the first sample answer obtained by the obtaining module into a preset model, and generating at least one first sample word-unit according to the first sample prompt word and the first sample answer through the preset model; Outputting a second sample answer according to the at least one first sample word and the sample first text through the preset model; The preset model is trained according to the second sample answer and the target answer to obtain the text training model.

16. The device according to claim 15, characterized in that The device further comprises: a determination module; The acquisition module is specifically used to input the first sample prompt word and the first sample answer into the preset module of the preset model, and acquire N second sample word-units based on the first sample answer through the preset module; The determining module is configured to determine the at least one first sample word-gram from the N second sample word-grams according to the first sample prompt word and the N second sample word-grams acquired by the acquiring module through the preset module; The execution module is further used to train the preset module according to the at least one first sample word-gram and the at least one target word-gram determined by the determination module to obtain the multi-head prediction module.

17. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the text generation method according to any one of claims 1 to 8 are implemented.

18. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the text generation method according to any one of claims 1 to 8 are implemented.

19. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the text generation method according to any one of claims 1 to 8.

Images