Text generation method, electronic device, storage medium and computer program product

Abstract

The present application discloses a text generation method, an electronic device, a storage medium and a computer program product, which relate to the fields of large model technology and natural language processing. The method may include: responding to an input instruction on an operation interface, determining first text data corresponding to the input instruction, wherein the first text data contains multiple first word units; based on the first text data, generating second text data through multiple iterations, wherein in the same iteration process, multiple second word units in the second text data are generated based on the first text data and candidate word units in the first text data, and the candidate word units are used to represent word units corresponding to the same iteration process; and displaying the second text data on the operation interface. The present application solves the technical problem of low accuracy in text generation.

Description

Technical Field

[0001] The present application relates to the fields of large model technology and natural language processing, and more specifically, to a text generation method, an electronic device, a storage medium and a computer program product. Background Art

[0002] At present, with the development of deep learning technology, Large-Scale Language Model for Information Retrieval (LLM) has become a research hotspot in the field of Natural Language Processing (NLP), and has demonstrated powerful capabilities in tasks such as text generation, dialogue systems, machine translation, and reading comprehension.

[0003] In the related art, LLM mainly adopts an autoregressive model architecture when processing sequence data (i.e., text data). In the inference stage (i.e., in reasoning or generating text), when predicting tokens in text data, the above model usually predicts tokens one by one until the preset length of the text data is reached or the end mark is encountered.

[0004] However, as the length of text data increases, the above model only focuses on predicting the next word, and there is a greater possibility that the generated text will deviate from the user's original intention or context, resulting in a decrease in the quality of the generated text, especially in scenarios that require cross-sentence or multi-round dialogues, which may affect the coherence and rationality of the generated text. Therefore, there is still a technical problem of low text generation accuracy.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a text generation method, an electronic device, a storage medium, and a computer program product to at least solve the technical problem of low text generation accuracy.

[0007] According to one aspect of an embodiment of the present application, a text generation method is provided. The method may include: responding to an input instruction acting on an operation interface, determining first text data corresponding to the input instruction, wherein the first text data includes multiple first word units; based on the first text data, generating second text data through multiple iterations, wherein in the same iteration process, multiple second word units in the second text data are generated based on the first text data and candidate word units in the first text data, and the candidate word units are used to represent word units corresponding to the same iteration process; and displaying the second text data on the operation interface.

[0008] According to another aspect of the embodiment of the present application, another text generation method is provided. The method may include: obtaining first text data by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter includes the first text data, and the first text data includes multiple first word units; based on the first text data, generating second text data through multiple iterations, wherein in the same iteration process, multiple second word units in the second text data are generated based on the first text data and candidate word units in the first text data, and the candidate word units are used to represent word units corresponding to the same iteration process; outputting the second text data by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the second text data.

[0009] According to another aspect of an embodiment of the present application, an electronic device is also provided. The electronic device may include a memory and a processor: the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions. When the above-mentioned computer-executable instructions are executed by the processor, any one of the above-mentioned methods is implemented.

[0010] According to another aspect of an embodiment of the present application, a processor is further provided, and the processor is used to run a program, wherein any one of the above methods is executed when the program is running.

[0011] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any of the above methods.

[0012] According to another aspect of the embodiment of the present application, a computer program product is also provided. The computer program product includes a computer program, and when the computer program is executed by a processor, the method of the embodiment of the present application is implemented.

[0013] In an embodiment of the present application, if an input instruction corresponding to the corresponding operation performed on the operation interface is detected, the first text data containing multiple first word units can be analyzed from the input instruction. The second text data can be generated by multiple iterations according to the first text data, wherein the multiple second word units in the second text data in the same iteration process are generated in combination with the first text data and the candidate word units therein. The generated second text data can be displayed on the operation interface. In this embodiment, the first word unit that has been acquired can be used to predict multiple second word units at one time, avoiding the situation where the quality of text generation is reduced due to only being able to predict one word unit each time until the entire text data is obtained. And the second word unit can be generated in combination with the first text data and the candidate word units therein, and the contextual relationship of multiple word units can be considered at the same time through the above method, so that the original intention or context of the user can be grasped more accurately, thereby maintaining longer coherence and logical consistency when generating text. Then the technical effect of text generation accuracy is achieved, and the technical problem of low text generation accuracy is solved.

[0014] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 is a schematic diagram of an application scenario of a text generation method according to an embodiment of the present application;

[0017] Figure 2 is a flowchart of a text generation method according to an embodiment of the present application;

[0018] Figure 3 is a flowchart of another text generation method according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of an optimized large language model according to an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a text generation device according to an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of another text generation device according to an embodiment of the present application;

[0022] Figure 7is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0023] Figure 8 is a block diagram of an electronic device according to a text generation method of an embodiment of the present application;

[0024] Fig. 9 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a text generation method according to an embodiment of the present application;

[0025] Fig.10 It is a structural block diagram of a computing environment of a text generation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] The technical solution provided in this application is mainly implemented using large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as LLM, multi-modal pre-training model, etc.

[0028] It should be noted that when the large model is actually applied, the pre-trained model can be fine-tuned through a small number of samples so that the large model can be applied to different tasks. For example, the large model can be widely used in NLP, computer vision, speech processing and other fields, and can be specifically applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, it is explained as an example that in a dialogue scenario, the LLM is first adjusted to obtain a text generation model, and then the text generation model is used to analyze the first text data output by the user to generate the second text data corresponding to the first text data.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0031] Tokens are the core concept in NLP and text analysis. They refer to the smallest unit in a language or text and are used to convert text data into a form that can be processed by computers. When building language models, especially when processing deep learning models such as LLM, the choice and representation of tokens have an important impact on the performance of the model.

[0032] According to an embodiment of the present application, a text generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Considering the huge number of model parameters of the large model and the limited computing resources of the mobile terminal, the above-mentioned text generation method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 In the application scenario shown, the large model is deployed in a server 30, which can be a cloud. The server 30 can be connected to one or more terminal devices 10 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks, wherein the terminal device 10 can be a client device, and the client device here can include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a vehicle-mounted device, etc., and the client devices together constitute a client relative to the server. An operation interface for obtaining product images can be deployed on the graphical user interface of the client device, and the operation interface can be an operation interface of an e-commerce platform. The terminal device 10 can interact with the user through the graphical user interface to implement the call of the large model, and then implement the text generation method provided in the embodiment of the present application. The server 30 and the terminal device 10 can exchange information through the network 20. A large model can be deployed in the database 40 in the server. In the embodiment of the present application, the database 40 stores a text generation model that can process the first text data to obtain the second text data. It can be used to call through the server 30 to generate the second text data to reply to the first text data.

[0034] In an embodiment of the present application, a system consisting of a terminal device, a network and a server can perform the following steps: if a user needs to reply to a corresponding question through the above text generation, the user can execute a corresponding input instruction on the operation interface of the terminal device and input the first text data to be replied. The terminal device can obtain the input instruction and send it to the server through the network. After the server receives the input instruction, it can perform the following steps: step S102, responding to the input instruction on the operation interface, determining the first text data corresponding to the input instruction, wherein the first text data contains multiple first word units; step S104, based on the first text data, generating the second text data through multiple iterations, wherein in the same iteration process, multiple second word units in the second text data are generated based on the first text data and the candidate word units in the first text data, and the candidate word units are used to represent the word units corresponding to the same iteration process; the server generates the second text data, which can be sent to the terminal device through the network. After the terminal device receives the second text data, it can perform the following steps: step S106, displaying the second text data on the operation interface. To reply to the first text data for the user to view.

[0035] In this embodiment, the first word-gram that has been acquired can be used to predict multiple second word-grams at one time, avoiding the situation where the quality of text generation is reduced due to the fact that only one word-gram can be predicted each time until the entire text data is obtained. In addition, the second word-gram can be generated by combining the first text data and the candidate word-grams therein. The above method can simultaneously consider the contextual relationship of multiple word-grams, so that the original intention or context of the user can be grasped more accurately, thereby maintaining longer coherence and logical consistency when generating text. In this way, the technical effect of text generation accuracy is achieved, and the technical problem of low text generation accuracy is solved.

[0036] It should be noted that, when the operating resources of the client device can meet the deployment and operating conditions of the large model, the embodiments of the present application can be carried out in the client device.

[0037] Under the above operating environment, this application provides Figure 2 It should be noted that the text generation method of this embodiment can be Figure 1 The illustrated embodiment is executed by a mobile terminal. Figure 2 is a flowchart of a method for generating text according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0038] Step S202: responding to an input instruction applied to the operation interface, determining first text data corresponding to the input instruction.

[0039] In the technical solution provided in the above step S202 of the present application, the first text data can be a prompt information (prompt) input by the user, that is, the initial text to be replied to by the user, which serves as the guiding information for the server to generate a corresponding reply response. The first text data can include multiple first tokens. The first text data can be a sentence, a question, a subject description, a conversation history or any other form of text. The first token can be the smallest unit after the first text data is decomposed, for example, it can be a word, a subword, a character or a special question mark, a word, etc. The first token can also be called an input token.

[0040] It should be noted that the above-mentioned first text data and the form and content of the first word contained therein are only for illustration and are not specifically limited here.

[0041] Optionally, the operation interface can be a web form, an input box of an application, or a dialogue window of a chat robot, etc. It should be noted that the application scenarios and types of the above-mentioned operation interfaces are only examples, and are not specifically limited here. As long as the operation interface corresponding to the text generation method of the embodiment of the application can be applied, it is within the protection scope of the embodiment of the application.

[0042] In this embodiment, when an input instruction is detected on the operation interface, the first text data input by the user can be determined according to the input instruction.

[0043] Optionally, the user sends an input instruction to the corresponding server through the operation interface. The above input instruction is transmitted to the server. If the server receives the user's input instruction, it can parse and understand the input instruction and convert the input instruction into the first text data that can be processed by the text generation model of the embodiment of the present application, that is, prompt, to guide the text generation process of text generation.

[0044] For example, a user may input on the operation interface: "Hello, what is 1+1?", and the above content may be an input instruction sent by the user, and its intention is to ask for the answer to a math problem. For the above user input "Hello, what is 1+1?", the server may recognize its complete content and convert it into a set of first word units, and the first word units may be "you", "good", "1+1", "equal to" and "how much". The set composed of the above first word units forms the first text data. It should be noted that the above-mentioned first text data input and the first word units contained therein are only for illustration and are not specifically limited here.

[0045] Step S204: generating second text data through multiple iterations based on the first text data.

[0046] In the technical solution provided in the above step S204 of the present application, the second text data can be the output information (output) fed back to the user by analyzing the first text data, which is used to reply to the first text data. That is, the second text data is the response information generated after in-depth understanding and analysis of the first text data input by the user, and includes a new word sequence generated based on the context and requirements of the prompt, which is used to answer questions, complete tasks, supplement text or continue the conversation. The second text data can be obtained by combining multiple second word units generated separately in multiple iterations. Multiple second word units in the same iteration process can be generated based on the first text data and candidate word units (candidate token) in the first text data, that is, the second word unit can be generated once for understanding the contextual relationship of the first text data. The second word unit can be an element such as a word, a phrase, a number or a special symbol, which is only used as an example here without specific limitation.

[0047] Optionally, the candidate word-gram may be a continuous or related word-gram sequence selected from the first word-gram, which is used to more accurately generate the subsequent second word-gram in the prediction stage, and may also be referred to as a candidate word-gram sequence. For example, if the first word-gram includes n word-grams C0, C1, C2, ...n-1, and if the second word-gram is predicted based on C0 in the first text data during this iteration, the candidate word-gram for this iteration may be the m first word-grams after C0, that is, C1~m. Each iteration process corresponds to a corresponding candidate word-gram, that is, each iteration process has a corresponding current prediction point (the first word-gram used to predict the second word-gram in the prompt), and a corresponding number of first word-grams are selected after the current prediction point as candidate word-grams. m is the sequence length, which may be a preset fixed value or dynamically adjusted according to actual needs.

[0048] For example, if the prompt is "Hello, what is 1+1?", m=4 is preset, and the current prediction point is "1", then the candidate word (C1~4) can be the sequence of "1" followed by "+1 equals".

[0049] It should be noted that the above-mentioned setting method and size of m are only for example illustration and are not specifically limited here. As long as a method can be used to select a continuous or related word sequence in the first text data to analyze the context information and facilitate a better understanding of the context, it is within the protection scope of the embodiments of the present application.

[0050] In this embodiment, in response to the input instruction on the operation interface, after the corresponding first text data is determined, multiple second word units can be generated in the same iteration process based on the first text data and part of the word units in the first word unit, thereby generating the second text data through multiple iterations.

[0051] Optionally, at the initial stage of text generation, the server may receive and parse the prompt input by the user, and may convert each first word in the prompt into an internally processable representation, such as a word embedding vector, to establish a preliminary understanding of the prompt context.

[0052] Optionally, in the process of iteratively generating the second text data, the iterative generation of the second text data output can be started based on the parsed prompt. Different from the autoregressive model in the related art that predicts each word one by one, the embodiment of the present application adopts a more efficient generation mechanism. Specifically, multiple second word-grams are predicted based on the current prediction point and candidate word-grams (C1~m) in the first text data in a single iteration. When generating the second word-gram, not only the prompt is considered, but also the candidate word-gram sequence C1~m that immediately follows the prompt is additionally used as auxiliary information for prediction. This mechanism can consider the contextual relationship of multiple first word-grams at one time during the generation process, thereby improving the generation speed and the accuracy of the prediction.

[0053] Optionally, if one iteration process is set to predict four word-grams, then in a certain iteration process, for the "1" in the prompt "Hello, what is 1+1?" as a prediction, the subsequent candidate word-grams C1~4 "+1 equals" can be considered at the same time to predict the second word-gram. In a certain iteration process, for "1+", the subsequent candidate word-grams C1~4 "1 equals more" can be considered at the same time to predict the second word-gram. Similarly, in multiple iteration processes, the second word-gram is predicted for different first word-grams in the first text data, and the corresponding candidate word-grams. Until the first word-grams in the first text data are all used as the current prediction point to predict the second word-gram. The second word-grams generated by the multiple iteration processes can be combined to obtain the second text data. Through the above prediction method, the number of predictions required is reduced, especially in the generation of long texts, avoiding the slow generation speed and long-term dependence problems caused by the one-by-one prediction method in the related technology, thereby achieving the technical effect of significantly improving the efficiency of the text generation process.

[0054] In an embodiment of the present application, multiple word units can be predicted in a single iteration, which significantly reduces the number of iterations required to generate text of the same length, thereby greatly improving the generation speed and reducing waiting time. Using C1~m as a candidate word unit, the prediction can be based on richer contextual information, reducing the uncertainty of the search, and improving the coherence and prediction accuracy of the generated text. Predicting multiple word units at one time helps the model maintain coherence and logical consistency during the generation process, especially in tasks that require understanding complex contexts and long-term logical relationships, such as multi-round conversations or cross-sentence understanding. The user experience is improved by the above-mentioned faster and more accurate text generation, especially in scenarios that require real-time response or processing of large amounts of text data, users can get high-quality responses faster. By adjusting the length of C1~m, it can flexibly adapt to different types of generation tasks, from simple questions and answers to complex text creation, and can provide more efficient and intelligent solutions.

[0055] In summary, text generation based on generating second text data (output) through multiple iterations based on first text data (prompt) not only optimizes the generation efficiency of large-scale language models, but also improves the quality and coherence of generated text, providing more powerful and flexible generation capabilities for a wide range of application scenarios.

[0056] Step S206: display the second text data on the operation interface.

[0057] In the technical solution provided in the above step S206 of the present application, after the second text data is generated based on the first text data through multiple iterations, the second text data can be displayed on the operation interface.

[0058] Optionally, after the second text data is generated, it can be sent to the operation interface through the network for display. Once the second text data is displayed on the operation interface, the user can immediately see the response of the server and provide feedback or further interaction according to their own needs. For example, the user can evaluate the quality of the generated text, request the model to modify or expand the text, save or share the generated text, etc. User feedback not only affects the further adjustment of the text generation model, but may also be used to train the text generation model and optimize its generation strategy through reinforcement learning and other methods.

[0059] Through the above steps S202 to S206 of the present application, if an input instruction corresponding to the corresponding operation performed on the operation interface is detected, the first text data containing multiple first word units can be analyzed from the input instruction. The second text data can be generated by multiple iterations according to the first text data, wherein the multiple second word units in the second text data in the same iteration process are generated in combination with the first text data and the candidate word units therein. The generated second text data can be displayed on the operation interface. In this embodiment, the first word unit that has been acquired can be used to predict multiple second word units at one time, avoiding the situation where the quality of text generation is reduced due to only being able to predict one word unit each time until the entire text data is obtained. And the second word unit can be generated in combination with the first text data and the candidate word units therein, and the contextual relationship of multiple word units can be considered at the same time through the above method, so that the original intention or context of the user can be grasped more accurately, thereby maintaining longer coherence and logical consistency when generating text. Then the technical effect of text generation accuracy is achieved, and the technical problem of low text generation accuracy is solved.

[0060] The above method of this embodiment is further introduced below.

[0061] As an optional implementation, step S104, based on the first text data, generates second text data through multiple iterations, including: inputting the first text data into a text generation model, and generating the second text data through multiple iterations using the text generation model.

[0062] In this embodiment, in the process of generating second text data based on first text data through multiple iterations, the first text data can be input into a text generation model, and the text generation model can be used to generate the second text data through multiple iterations, wherein the text generation model is obtained by optimizing an ordinary LLM.

[0063] Optionally, the ordinary LLM can only predict the next second word-gram in each iteration, and then add the second word-gram to the sequence composed of the pre-predicted second word-grams (the second word-grams in the historical iteration process), and then predict the next second word-gram. For example, in the nth iteration of the ordinary LLM, the sequence T0~n-1 composed of the pre-predicted second word-grams can be used to predict Tn. The text generation model in the embodiment of the present application can be obtained by adjusting the above-mentioned LLM. Specifically, in the iteration process, not only can multiple second word-grams be predicted at one time, but also under the premise of the sequence composed of the pre-predicted second word-grams, the candidate word-grams corresponding to this iteration process in the first text data and the current prediction point can be used to predict multiple second word-grams. For example, in the nth iteration of the text generation model, T0~n-1 and C1~ml can be combined to predict Tn, Tn+1, Tn+2, and Tn+3.

[0064] Optionally, the user's input instructions are parsed and converted into first text data, and the first text data can be input into a text generation model. The above text generation model is different from the traditional LLM. It is optimized and more efficient in processing prompts. It can generate high-quality second text data by utilizing the word unit information that has appeared in the prompt and the candidate word units C1~m.

[0065] Optionally, the optimized text generation model generates the second text data in multiple iterations. In each iteration, the model not only considers all the word units in the prompt, but also pays special attention to the partial word unit sequence C1~m, and predicts multiple subsequent word units at one time based on this information, instead of one by one like the traditional LLM. The above text generation mechanism significantly reduces the time and steps required to generate long texts, and improves the overall generation efficiency.

[0066] In the embodiment of the present application, due to the adoption of a mechanism for predicting multiple word units at one time based on C1~m, the number of iterations required by the optimized text generation model when generating text of the same length is far less than that of the traditional LLM, thereby significantly speeding up the speed of text generation. For example, if C1~m covers 4 word units, then theoretically the model generation speed can be 4 times faster than the LLM model predicted word by word. Using C1~m as additional prediction auxiliary information, the text generation model can better understand the context of the word unit sequence when generating, reduce the scope of blind search, and thus improve the accuracy of predicting word units. This is particularly important for tasks that rely on accurate context understanding, such as text generation for professional fields or complex dialogue systems. The strategy of predicting multiple word units at one time helps the model to consider the context of multiple word units at the same time when generating text, reducing the long-term dependency problem caused by predicting word units one by one. This makes the generated text more logically coherent, especially in scenarios that require cross-sentence or multi-round dialogues, to maintain the coherence and rationality of the text. Faster generation speed and higher text quality directly improve the user experience, allowing users to obtain accurate and coherent text responses more quickly. It is suitable for scenarios that require real-time interaction or high-frequency text generation, and enhances the value of the model in practical applications.

[0067] In summary, by optimizing the generation mechanism of LLM, not only the generation speed is improved, but also the prediction accuracy is enhanced and the processing ability of long-term dependencies is optimized. At the same time, these technical effects are achieved without significantly increasing the model parameters, which greatly improves the performance and practicality of large-scale language models in text generation tasks.

[0068] As an optional implementation, first text data is input into a text generation model, and the text generation model is used to generate second text data corresponding to the first text data through multiple iterations, including: obtaining a second word element corresponding to a historical iteration process before a current iteration process, wherein the output word element corresponding to the historical iteration process is a word element generated by the text generation model in the historical iteration process; determining a candidate word element to be input into the text generation model in the current iteration process from multiple first word elements; inputting the candidate word element, the second word element corresponding to the historical iteration process, and the first text data into the text generation model, and using the text generation model to generate a second word element corresponding to the current iteration process; and summarizing the second word elements corresponding to multiple iteration processes to obtain second text data.

[0069] In this embodiment, in the process of inputting the first text data into the text generation model and generating the second text data by multiple iterations using the text generation model, the second word element corresponding to the historical iteration process before the current iteration process can be obtained. The candidate word element to be input into the text generation model in the current iteration process is determined from multiple first word elements. The candidate word element, the second word element in the historical iteration process and the first text data can be input into the text generation model. The text generation model is used to generate the second word element corresponding to the current iteration process. The second word elements corresponding to multiple iteration processes can be summarized to obtain the second text data. Among them, the word element generated in the historical iteration process is all the results predicted before. If this iteration is the nth time, the word element generated in the historical iteration process is T0~n-1.

[0070] Optionally, in the process of generating the second text data, the text generation model generates a certain number of word-grams after each iteration. Before the current iteration (nth iteration), the second word-grams generated in the historical iteration process (T0~n-1) can be collected, that is, all prediction results from the first iteration to the current iteration. The above historical word-gram sequence (the second word-gram generated in the historical iteration process) can be used as information input into the text generation model when generating new word-grams (the second word-gram to be generated in the current iteration), which is crucial to maintaining the coherence and logic of the generated text.

[0071] Optionally, in addition to the second word T0~n-1 in the historical iteration process, the text generation model can also determine the candidate word (C1~m) to be input in the current iteration process. The candidate word can be the part of the prompt that follows the generated word, or it can be a continuous word sequence. The role of C1~m is to provide the text generation model with direct context information for the current word generation, helping the model to more accurately predict subsequent words.

[0072] Optionally, in the current iteration, the candidate word (C1~m), the second word in the historical iteration (T0~n-1), and the prompt are input into the text generation model together. The above input strategy is different from the single prompt input method of the original LLM. It integrates multi-source information, so that the text generation model can simultaneously consider the original context of the prompt, the context information of the generated word sequence, and the direct guidance of the candidate word when generating the second word.

[0073] In the embodiments of the present application, C1~m is used as candidate tokens, enabling the text generation model to more accurately understand the context during prediction, thereby improving the coherence and rationality of the generated text. Since the text generation model can refer to the historically generated token sequence (T0~n-1) in each iteration, it performs better in handling long-term dependencies in the text, which is particularly beneficial for text generation tasks across sentences or multi-turn dialogues.

[0074] In summary, by combining the historical iteration tokens (T0~n-1), candidate tokens, and the original prompt information in the text generation model, an all-round improvement in generation speed, prediction accuracy, and long-term dependency handling ability is achieved, providing strong technical support for the efficient and high-quality text generation of large language models in various application scenarios. Thus, the technical effect of text generation accuracy is realized.

[0075] As an alternative implementation, determining the candidate tokens to be input into the text generation model in the current iteration process from multiple first tokens includes: determining the target token corresponding to the second token corresponding to the historical iteration process from the multiple first tokens; and obtaining a preset number of tokens after the target token from the multiple first tokens to obtain the candidate tokens.

[0076] In this embodiment, in the process of determining the candidate tokens to be input into the text generation model in the current iteration process from multiple first tokens, the target token corresponding to the second token of the historical iteration process can be determined from the multiple first tokens. Then, a preset number of tokens after the target token are obtained from the multiple first tokens to obtain the candidate tokens. In the embodiments of the present application, the preset number can be set to 4. This is only an example for illustration and is not subject to specific limitations. The target token can be the first first token that the current iteration process is focusing on and predicting when generating or predicting the token sequence, that is, it can be the first word in the prompt that this iteration process focuses on. It is located at the end of the generated token sequence and is also the starting point of the token sequence to be generated. If the preset number is 4, the corresponding second tokens in this iteration process can be Tn, Tn+1, Tn+2, and Tn+3.

[0077] For example, if the prompt is "Hello, what is 1+1 equal to?", and the text generation model has analyzed the two tokens "Hello" and "good", then in the subsequent analysis process, the first word is "1". If the current prediction point is "1", the text generation model will use the four tokens "+1 equal to" immediately following it as candidate tokens (C1~4) to assist in predicting and generating multiple second tokens. The above "1" is the first word in the current iteration process, which is the starting point for the text generation model to predict the next token sequence.

[0078] Optionally, during the text generation process, the text generation model will iteratively generate word-meta sequences multiple times to form a complete second text tree. The target word-meta in the current iteration is the first word-meta that the text generation model needs to predict at the current moment. It is located at the end of the generated word-meta sequence and is also the starting point of the word-meta sequence to be generated. The target word-meta can be determined based on the prompt and the second word-meta of the historical iteration process (T0~n-1). At the beginning of each iteration, the next word-meta that is most directly related to the generated word-meta can be identified from the prompt and used as the target word-meta. The target word-meta is the focus of the model prediction process and the basis for the subsequent selection of candidate word-meta.

[0079] Optionally, the preset number can be set to 4, that is, the model will consider the 4 consecutive word-grams following the target word-gram as candidate information. Through the above method, more accurate contextual guidance is provided to the text generation model, so that it not only relies on the sequence of generated word-grams when predicting, but also can directly use the word-gram information following the target word-gram in the prompt. For example, when predicting "1", C1~4 is "+1 equals". This sequence provides contextual information directly related to "1", helping the text generation model to more accurately predict the next word-gram sequence.

[0080] In an embodiment of the present application, the candidate word (C1~m) is determined by the above method, thereby narrowing the scope of prediction so that the text generation model can focus more on the word that is most relevant to the current context, reduce the possibility of blind search, and improve the accuracy of predicted word. Due to the introduction of candidate word, the text generation model can predict multiple word in one iteration. Compared with word-by-word prediction, the speed of text generation is significantly accelerated, especially in scenarios where long text or quick response needs to be generated, the user experience is significantly improved. The C1~m mechanism can help the model take into account the contextual relationships of longer sequences when predicting, which is particularly beneficial for text generation tasks across sentences or multi-round dialogues, and can generate more logically coherent text.

[0081] In summary, by determining the target word from the prompt and obtaining candidate word based on it (C1~m), an optimized prediction strategy is provided for the text generation model, which helps to improve the quality and efficiency of generated text and achieve fast and accurate text generation, which is of great value for various natural language-based task processing.

[0082] As an optional implementation, the number of candidate word-grams is the same as the number of second word-grams corresponding to the current iteration process.

[0083] In this embodiment, the number of candidate word-grams may be the same as the number of second word-grams corresponding to the current iteration process.

[0084] It should be noted that the setting of the preset number is flexible and can be adjusted according to specific application scenarios and requirements. In different types of text generation tasks, the number of subsequent word units that may need to be considered may vary. For example, in scenarios that require more detailed context understanding, the value of m can be set higher.

[0085] In the LLM in the related art, usually one word-gram is generated at a time, and then the generated word-gram is used as a new input for the next prediction. However, in the embodiment of the present application, the text generation model generates multiple word-grams in each iteration, and the number of these word-grams is the same as the number of candidate word-grams (C1~m). If the number of candidate word-grams is set to 4, then in each iteration, the text generation model will predict and generate 4 word-grams, and will also consider the 4 candidate word-grams (C1~4) immediately following the starting point of the word-gram sequence to be generated. Since the number of the above two word-grams is the same, the text generation model can simultaneously consider the same number of subsequent word-grams as context information when predicting the word-gram sequence, which can help the text generation model generate word-gram sequences that not only consider the direct context of the current word-gram, but also foresee the potential context of subsequent word-grams, thereby generating more coherent and reasonable text.

[0086] Optionally, the number of candidate lemmas is set to be the same as the number of second lemmas in the current iteration process. When the text generation model generates multiple lemmas in one iteration, it can also make full use of the candidate lemma information immediately following the generated lemma sequence in the prompt. This reduces the number of iterations required by the model and significantly speeds up text generation. In a multi-round dialogue scenario, the text generation model can predict and generate replies for multiple lemmas at one time based on the previous dialogue history, while considering subsequent candidate lemma information, making the dialogue smoother and more natural, reducing the waiting time for each reply, and enhancing the user experience. For example, in the task of continuing an article or story, setting the above two numbers to be the same allows the text generation model to generate subsequent content for multiple lemmas at one time based on the understanding of the existing text, while being able to foresee the next lemma sequence, which helps maintain the style of the article and the coherence of the narrative.

[0087] In summary, by setting the number of candidate word units (C1~m) to be the same as the number of second word units generated by the current iteration process, the coherence of the text generation process is enhanced, the generation efficiency is improved, and the resource utilization is optimized, providing a more efficient and intelligent solution for a variety of natural language-based task processing. The above settings are particularly suitable for scenarios that require real-time response or generation of long texts, improving the performance and user experience of large-scale language models in practical applications.

[0088] As an optional implementation, the text generation model includes: an embedding layer, a feature processing layer and multiple output layers; the candidate word-grams, the second word-grams corresponding to the historical iteration process and the first text data are input into the text generation model, and the second word-grams corresponding to the current iteration process are generated by the text generation model, including: encoding the candidate word-grams by using the embedding layer to obtain candidate encoding vectors, encoding the second word-grams corresponding to the historical iteration process by using the embedding layer to obtain historical encoding vectors, and encoding the first text data by using the embedding layer to obtain text encoding vectors; inputting the candidate encoding vectors, the historical encoding vectors and the text encoding vectors into the feature processing layer, and obtaining the hidden state vector output by the feature processing layer; inputting the hidden state vectors into multiple output layers respectively, and obtaining the second word-grams corresponding to the current iteration process output by the multiple output layers, wherein the second word-grams corresponding to the current iteration process correspond one-to-one to the multiple output layers.

[0089] In this embodiment, in the process of inputting the candidate word-gram, the second word-gram in the historical iteration process and the first text data into the text generation model to generate the second word-gram corresponding to the current iteration process, the embedding layer in the text generation model can be used to encode the candidate word-gram to obtain a candidate encoding vector, and the embedding layer can also be used to encode the second word-gram corresponding to the historical iteration process to obtain a historical encoding vector, and the embedding layer can also be used to encode the first text data to obtain a text encoding vector. The above-mentioned candidate encoding vector, historical encoding vector and text encoding vector can be input into the feature processing layer in the text generation model, and the feature processing layer is used to analyze the above-mentioned encoding vector to obtain a hidden state vector. The hidden state vectors can be respectively input into multiple output layers in the text generation model, and the output layers are used to output the second word-gram corresponding to the current iteration process respectively.

[0090] Optionally, the text generation model of the embodiment of the present application may include an embedding layer, a feature processing layer, and multiple output layers. The embedding layer may also be referred to as a word embedding layer. The feature processing layer may also be a transformer layer, or a hidden layer. The input layer may be a language model output (LanguageModel Head, LM Head for short) layer. The number of input layers may be the same as the number of second word units corresponding to the current iteration process, that is, the two correspond one to one. For example, if the number of second word units corresponding to the current iteration process is 4, there may be 4 LM Heads, namely LM Head, LM Head 2, LM Head 3, and LM Head 4, which are only used as examples here, and the number and name of the input layer are not specifically limited. The hidden state vector may also be referred to as the final hidden state (Last HiddenStates) or the hidden layer result, which may be the vector representation of the output of the last layer of the network in the Transformer Layers layer. The hidden state contains the text generation model's deep understanding and processing results of the above three aspects of the input data, and can capture the long-term dependencies and rich semantic information of the elements in the sequence.

[0091] Optionally, the embedding layer is the first step for the text generation model to process the above three aspects of input data. Whether it is a candidate word, the second word in the historical iteration process or the prompt, they can be sent to the embedding layer for encoding. The embedding layer converts the above three types of input data into corresponding high-dimensional vectors, namely candidate encoding vectors, historical encoding vectors and text encoding vectors, which can capture the semantics and contextual information of the word. For example, for the candidate word "+1 equals", the embedding layer converts it into a corresponding vector representation, so that the model can understand the meaning of these words in the context.

[0092] Optionally, Transformer Layers are responsible for deep analysis and processing of the above-mentioned vectors encoded by the embedding layer. This layer can capture and understand the long-term dependencies and semantic structures between word units through multi-layer attention mechanisms and feedforward neural networks. During the processing, Transformer Layers will comprehensively consider candidate encoding vectors, historical encoding vectors, and text encoding vectors to generate hidden state vectors. The hidden state vector contains the Transformer Layers layer's deep understanding of the input data. They can effectively integrate information from different inputs and provide rich semantic and contextual information for the subsequent output layer.

[0093] Optionally, the output layer is responsible for converting the hidden state vector into a probability distribution of a word unit, and then predicting the next word unit sequence. In an embodiment of the present application, each LM Head can correspond to a word unit in the generated sequence. If it is set to generate 4 words at a time, there will be 4 LM Heads. Each LM Head receives the hidden state vector as input, generates a probability distribution of the word unit, and then samples the next word unit according to the probability size. In this way, the text generation model can predict multiple words at the same time, which significantly improves the speed of text generation.

[0094] In an embodiment of the present application, by optimizing and adjusting the LLM, a text generation model is formed, which can efficiently process and generate text. Specifically, different input data are encoded into corresponding vectors through the embedding layer in the text generation model to ensure that the text generation model can understand the semantics and contextual information of the word unit. By deeply analyzing the above vectors using Transformer Layers, the complex structure of the word unit sequence can be captured. Using multiple LM Heads to predict multiple word units at one time based on the hidden state vector greatly speeds up the generation process. At the same time, because more contextual information can be considered, the quality of the generated text is also improved.

[0095] In summary, the various layers of the text generation model in the embodiment of the present application work closely together to achieve efficient processing of input data and rapid generation of high-quality text. The embedding layer encodes word-unit information, the feature processing layer analyzes and integrates this information, and the output layer predicts the word-unit sequence based on the analysis results. The whole process reflects the powerful ability of deep learning models in natural language processing tasks, especially for application scenarios that require fast and smooth text generation, such as dialogue systems, text generation and translation, etc., through the optimization of LLM, the user experience and generation efficiency can be significantly improved.

[0096] As an optional implementation, the feature processing layer includes multiple sub-layers; the candidate coding vector, the historical coding vector and the text coding vector are input into the feature processing layer, and the hidden state vector output by the feature processing layer is obtained, including: the historical coding vector and the text coding vector are input into the first sub-layer of the multiple sub-layers, and the candidate coding vector is input into the last sub-layer of the multiple sub-layers, and the hidden state vector output by the feature processing layer is obtained.

[0097] In this embodiment, when the candidate coding vector, the historical coding vector and the text coding vector are input into the feature processing layer, and the feature processing layer is used to process the above three vectors, the historical coding vector and the text coding vector can be input into the first sublayer of multiple sublayers, and the candidate coding vector can be input into the last sublayer of multiple sublayers, and each sublayer is used to perform corresponding processing on the received vectors to obtain a hidden state vector. Transformer Layers can be the main model structure layer of the text generation model, and the feature processing layer can include multiple sublayers, that is, it includes several layers, such as 16, 32, 40 or 64 layers. This is only for example and no specific limitation is made to the number of sublayers.

[0098] Optionally, the feature processing layer, as a key component of the text generation model, is composed of a series of sub-layers, and the number of sub-layers depends on the complexity and performance requirements of the model. The feature processing layer is responsible for deep feature extraction and fusion of the input vector data to generate high-quality text.

[0099] Optionally, the historical encoding vector (containing the word-unit information generated by the previous iteration) and the text encoding vector (the word-unit information of the prompt) are jointly input to the first sublayer of the feature processing layer. This method can integrate the contextual information of the sequence as early as possible, so that the feature processing layer can consider the output of the historical iteration and the context of the prompt, and build a comprehensive understanding of the sequence at the early stage of processing.

[0100] Optionally, unlike the initial processing of the historical encoding vector and the text encoding vector, the candidate encoding vector (containing a series of word-unit information immediately following the predicted word-unit sequence) is directly fed into the last sublayer of the feature processing layer. This ensures that the contextual information of the candidate word-unit can be directly and deeply integrated into the final output (hidden state vector) of the model after deep feature processing by the previous sublayer. The above processing of the candidate encoding vector helps to directly utilize relevant contextual information during prediction, reduce the scope of blind search in the entire vocabulary, and thus improve the accuracy and efficiency of prediction.

[0101] Optionally, in the feature processing layer, each sublayer extracts and transforms deep features based on the output of the previous sublayer. Through multi-layer self-attention mechanisms and feedforward neural networks, the sublayers can gradually deepen their understanding of word-meta sequences and capture long-term dependencies between words. From the first sublayer to the last sublayer, the data stream undergoes a series of feature enhancements and information fusions until the final hidden state vector is generated. After multi-sublayer deep processing in the feature processing layer, a hidden state vector that integrates history, prompt, and candidate word-meta information is obtained. The hidden state vector contains the text generation model's comprehensive understanding of the current predicted word-meta sequence, including semantics, grammar, and complex relationships between word-meta, providing a rich intermediate representation for subsequent word-meta generation.

[0102] In an embodiment of the present application, a specific input strategy is adopted in the feature processing layer, that is, the history and text encoding vectors are input into the first sublayer, and the candidate encoding vectors are input into the last sublayer. The direct input of candidate word units into the last sublayer reduces the computational burden of the model when generating each word unit and improves the generation speed. The deep integration of candidate encoding vectors ensures that the model can generate word units using the most direct contextual information, thereby improving the accuracy and coherence of the generated text. The deep feature interaction and transmission of multiple sublayers in the feature processing layer help capture the long-term dependencies in the word unit sequence and improve the performance of the model when processing long texts or multi-round conversations. By implementing the above-mentioned specific input and processing strategies in the feature processing layer, the performance of the text generation model is effectively optimized, the generation speed is improved, the high quality of the generated text is guaranteed, and significant advantages are also shown in processing long-term dependencies and contextual information.

[0103] As an optional implementation, the hidden state vector is input into multiple output layers respectively, and the second word element corresponding to the current iteration process output by the multiple output layers is obtained, including: using multiple output layers to map the hidden state vector to a preset vocabulary, and obtaining probability values ​​of multiple preset word elements contained in the preset vocabulary; based on the probability values ​​of the multiple preset word elements, determining the second word element corresponding to the current iteration process output by the multiple output layers.

[0104] In this embodiment, in the process of inputting the hidden state vector into multiple output layers respectively, so as to obtain the corresponding second word element in the current iteration process, the multiple output layers can be used to map the hidden state vector to the preset word list, so as to obtain the probability values ​​of multiple preset word elements contained in the preset word list. The corresponding second word element can be determined based on the probability values ​​of multiple preset word elements. Among them, the preset word list can be a full word list, that is, it contains all possible word elements in the model dictionary, or a smaller word list can be selected according to the task requirements. For example, in the generation of professional field text, the word list can only contain word elements related to the field, which can reduce the computational burden and improve the generation efficiency and accuracy. It should be noted that the above-mentioned preset word list is only for example and is not specifically limited here.

[0105] Optionally, in the process of obtaining the second word element corresponding to the current iteration process, the above-mentioned hidden state vector can be input into multiple output layers of the text generation model. The task of these output layers is to map the vector to each word element in the full vocabulary and generate a probability value distribution of the corresponding word element.

[0106] Optionally, each output layer is responsible for predicting a set of probability values ​​for mapping the hidden state vector to a word in a preset vocabulary. For example, if the current iteration is to generate four words, there will be four output layers, each of which will generate a probability distribution vector corresponding to the preset vocabulary, and the length of each vector is equal to the size of the vocabulary. The above probability values ​​reflect the probability of each word being selected as the next generated word given the hidden state vector.

[0107] Optionally, after obtaining the probability values ​​of multiple word-grams in a preset vocabulary, the second word-gram corresponding to the current iteration process is determined based on the above probability values. For example, based on the probability distribution of preset word-grams, a word-gram with a high probability can be selected as the output by random sampling. However, the above strategy can generate diverse and innovative texts, but may introduce some uncertainty. It is also possible to directly select a word-gram with a high probability as the output. The above strategy ensures that the word-gram generated in each iteration is statistically most likely to be correct, but it may limit the diversity of the text, especially when the probability value distribution is relatively flat.

[0108] It should be noted that the above-mentioned strategy and process for determining the second word-unit based on the probability values ​​of multiple preset word-units are only for illustration and are not specifically limited here. As long as the process and method can determine the second word-unit of the current iteration process based on the probability value, it is within the protection scope of the embodiments of the present application.

[0109] Optionally, the mapping process of the output layer involves a fully connected layer and a softmax function. The fully connected layer converts the hidden state vector into a vector equal to the size of the preset vocabulary, with each element corresponding to a word in the vocabulary. The softmax function then converts these elements into probability values, ensuring that the sum of the probability values ​​of all word units is 1, which is essential for sampling or deterministic selection of word units. Since the number of output layers is the same as the number of word units to be generated in the current iteration, the output layers can be processed in parallel to generate probability distributions for multiple word units at the same time. The above parallel processing capability not only increases the speed of text generation, but also allows the model to take into account other word units generated in parallel when generating each word unit, further optimizing the coherence and rationality of the generated text.

[0110] In an embodiment of the present application, from the hidden state vector to the probability value of the word in the preset vocabulary, and then to the process of determining the second word corresponding to the current iteration based on the probability value, the hidden state vector is mapped to the preset vocabulary through multiple parallel output layers, the probability value distribution of the word is generated, and then the word to be output is determined based on these probability values, thereby achieving fast and accurate text generation, and providing efficient and high-quality text generation capabilities for various applications such as dialogue systems, automatic text summarization, and intelligent writing assistance.

[0111] As an optional implementation, the method also includes: outputting multiple preset generation models, wherein different generation models generate different numbers of word units in the same iteration process; responding to a selection instruction on an operation interface, determining a preset generation model corresponding to the selection instruction from multiple preset generation models to obtain a text generation model.

[0112] In this embodiment, different preset generation models can be formulated according to the different numbers of second word units to be generated in the iteration process, and the plurality of preset generation models formulated above can be deployed in the database of the server, and the corresponding controls of the different preset generation models can be deployed on the operation interface of the user's client device. The user can perform corresponding selection operations on the controls corresponding to the corresponding preset generation models on the operation interface according to his own needs, for example, he can click on the corresponding control to generate the corresponding selection instruction, and determine the preset generation model corresponding to the selected control as the text generation model.

[0113] For example, among multiple preset generation models, one may generate 2 tokens in one iteration, while another model may generate 4 or 6 tokens. By setting the above multiple preset generation models, a variety of generation speeds and generated text granularities can be provided to meet the needs of different scenarios and user preferences.

[0114] Optionally, during the interaction between the user and the server, the user can perform selection operations on the operation interface based on his or her own needs. For example, if the user needs to generate a reply quickly, he or she may prefer a preset generation model that can generate more words at a time; and if the user cares about the precision and accuracy of the generated text, he or she may be more inclined to choose a preset generation model that generates fewer words at a time. The user's selection operation will generate a corresponding selection instruction, and the preset generation model to be used can be dynamically adjusted according to this instruction, that is, the operation interface responds to the user's selection instruction and determines a model for subsequent text generation from multiple preset generation models.

[0115] In the embodiment of the present application, a variety of text generation model selections can be provided so that different user needs and application scenarios can be flexibly adapted. The user can select a suitable preset generation model as a text generation model for text generation according to the urgency of the task, the complexity of the text or personal preference. The user can dynamically select a model during the interaction process, which not only improves the flexibility of selection, but also significantly improves the user experience. The user can adjust the generation speed and text quality according to the real-time requirements of the task, so that the system can better meet the diverse needs of the user and promote a more efficient and personalized human-computer dialogue and text generation experience.

[0116] The present application embodiment also provides another text generation method, Figure 3 is a flowchart of a text generation method according to an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps:

[0117] Step S302: acquiring first text data by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes first text data, and the first text data includes a plurality of first word units.

[0118] In the technical solution provided in the above step S302 of the present application, the first text data can be obtained through the first interface.

[0119] Optionally, when an input instruction is detected on the operation interface, the first interface may be called to determine the first text data input by the user according to the input instruction.

[0120] Step S304, based on the first text data, generate second text data through multiple iterations, wherein in the same iteration process, multiple second word-grams in the second text data are generated based on the first text data and candidate word-grams in the first text data, and the candidate word-grams are used to represent word-grams corresponding to the same iteration process.

[0121] In the technical solution provided in the above step S304 of the present application, after calling the first interface to obtain the first text data, multiple second word-grams can be generated in the same iteration process based on the first text data and the candidate word-grams in the first word-gram, so that the second text data can be combined based on the multiple second word-grams generated in multiple iteration processes.

[0122] Optionally, at the initial stage of text generation, the prompt input by the user can be parsed, and each first word in the prompt can be converted into an internally processable representation, such as a word embedding vector, to establish a preliminary understanding of the prompt context.

[0123] Optionally, based on the parsed prompt, iterative generation of the second text data output is started. A more efficient generation mechanism is adopted. Specifically, multiple second word-grams are predicted based on the first text data and partial word-grams (C1~m) in a single iteration. When generating each word-gram, not only the prompt is considered, but also the candidate word-gram sequence C1~m that immediately follows the prompt is additionally used as auxiliary information for prediction. This mechanism can consider the contextual relationship of multiple word-grams at one time during the generation process, thereby improving the generation speed and prediction accuracy. In each iteration, a subsequent word-gram sequence can be generated based on the current prompt state and the previous predicted word-gram (if any).

[0124] Step S306: outputting the second text data by calling the second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the second text data.

[0125] In the technical solution provided in the above step S306 of the present application, after the second text data is generated based on the first text data through multiple iterations, the second text data can be output through the second interface and displayed on the operation interface.

[0126] In an embodiment of the present application, the first text data is obtained by calling the first interface; based on the first text data, the second text data is generated by multiple iterations; and the second text data is output by calling the second interface. The first word element that has been obtained can be used to predict multiple second word elements at one time, avoiding the situation where the text generation efficiency is low because only one word element can be predicted each time until the entire text data is obtained. And the second word element can be generated by combining the first text data and some of the word elements therein. The above method can simultaneously consider the contextual relationship of multiple word elements, thereby maintaining longer coherence and logical consistency when generating text. Thereby achieving the technical effect of text generation accuracy and solving the technical problem of low text generation accuracy.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application, such as the data for verification, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0128] In the related art, during the training phase, LLM uses Teacher Forcing to use previous true words as input for subsequent word predictions, allowing the model to process in parallel during training and increase the training speed. However, a side effect of this approach is that the model only focuses on predicting the next word during training, ignoring the long-term dependencies of word sequences, which may reduce the performance of the model when dealing with complex tasks that require understanding the context. During the inference phase, LLM usually adopts an autoregressive model to predict words one by one, which means that after each prediction, the new word must be re-input into the model for the next prediction. Although the above method can produce high-quality text, due to its serial nature, the inference speed is slow and the accuracy is low, especially when generating long texts.

[0129] Specifically, a common large language model may include an embedding layer, a converter layer, and a language model output layer. The input of the large language model is the serialized word unit identifier before this iteration, represented as T0~n-1. The embedding layer is the first layer of the model, which converts the input T0~n-1 into a vector of fixed size and inputs the vector into the converter layer. The converter layer constitutes the main part of the model, and each layer contains a self-attention mechanism and a feedforward neural network. The self-attention mechanism allows attention to each word unit in the sequence when processing the sequence, thereby capturing the dependencies between word units, especially long-term dependencies. The feedforward neural network is used to perform nonlinear transformations on the representation of word units to enhance the expressive power of the model. As the last part of the model, the language model output layer has the task of converting the output of the last layer in the converter layer (that is, Last HiddenStates) back to the word unit probability distribution in the vocabulary. The above conversion process is completed through the weight matrix in the LM Head, which maps the high-dimensional hidden state of the model back to the word unit space of the vocabulary. The output of the converter layer is a conditional probability distribution of a word, where each word has a corresponding probability value. By sampling from this probability distribution, the next word Tn predicted by the model can be obtained. The above text generation process is not only slow, but also cannot consider the overall contextual relationship because it needs to predict word by word. Therefore, there is still a technical problem of low text generation accuracy.

[0130] The following is a further explanation of the embodiments of the present application.

[0131] In an embodiment of the present application, LLM is improved. At the input end of the model, in addition to using the traditional sequence of T0~n-1 words as input, candidate word sequence C1~m and prompt are also introduced. When predicting, the model not only considers historical words, but also considers words that have appeared in prompt, which helps the model to understand the context more accurately, especially in scenarios such as question answering and reading comprehension, which can effectively reduce the search space during prediction and improve prediction accuracy. According to the number of candidate words input, a corresponding number of sub-layers are set in the LM Head layer, and multiple subsequent words can be predicted at one time, which not only greatly improves the reasoning speed, but also because the above sub-layers share the same hidden state vector, the actual increase in the number of parameters is not much, and the complexity and time of training will not be significantly increased. The use of candidate word sequence can enhance the model's understanding of local context, thereby improving prediction accuracy. During reasoning, the use of candidate words can narrow the prediction range from the full vocabulary to the words that appear in prompt, greatly reducing the search space and improving prediction efficiency. Although additional prediction heads are added, the mechanism of sharing hidden layers means that the increase in parameters is limited and will not significantly affect the training speed. The ability to predict multiple words at once can significantly speed up the reasoning process, especially when generating long texts, which can achieve exponential speed increases. This achieves the technical effect of text generation accuracy and solves the technical problem of low text generation accuracy.

[0132] The above method of this embodiment is further introduced below.

[0133] In the embodiments of the present application, Figure 4 is a schematic diagram of an optimized large language model according to an embodiment of the present application, such as Figure 4As shown, the model may include an embedding layer 401, a converter layer 402, and m output layers, wherein the converter layer 403 may include m sublayers. The optimized large language model may have three inputs, that is, prompt, T0~n-1 in the historical iteration process before this iteration, and candidate word C1~m may be input into the embedding layer 401 of the optimized large language model. The above prompt, T0~n-1, and C1~m may be converted by using the embedding layer 401 to obtain the encoding vectors corresponding to the above inputs, that is, the encoding vector of prompt, the encoding vector of T0~n-1, and the encoding vector of C1~m. The encoding vector of prompt and the encoding vector of T0~n-1 may be input into the sublayer 1 of the converter layer 402 for processing, and the encoding vector of C1~m may be input into the sublayer m (that is, the last sublayer) of the converter layer 402 for processing, and finally the hidden state vector may be output. The hidden state vector can be input into output layer 1 to output layer m for processing respectively to predict the next word respectively. For example, output layer 1 predicts Tn and output layer m predicts Tn+m-1.

[0134] For example, in the embodiment of the present application, m=4 can be set. At this time, the corresponding converter layer 502 can include four sublayers, namely sublayer 1, sublayer 2, sublayer 3 and sublayer 4, and the corresponding output layers can also be four, namely output layer 1, output layer 2, output layer 3 and output layer 4. The word element output by output layer 1 is Tn, the word element output by output layer 2 is Tn+1, the word element output by output layer 3 is Tn+2, and the word element output by output layer 4 is Tn+3.

[0135] Optionally, the large language model structure proposed in the embodiment of the present application is an innovative optimization of the LLM of the related art, which aims to improve the efficiency and performance of the model in sequence generation tasks, especially when dealing with long-term dependencies. At the input end, in addition to the traditional T0~n-1 word sequence, additional candidate words C1~m and C1~m are added. The above-mentioned candidate words are part of the word sequence that is most likely to appear in the prompt, and are used to predict whether the m words after the currently generated word hit the C1~m sequence.

[0136] For example, when m=4, if the generated word sequence is "Hello, 1+1", then the C1~4 sequence is the four words after "equal to", that is, "how much".

[0137] In an embodiment of the present application, by considering the word units that have appeared in the prompt, the above model can better understand the possible semantic and grammatical environment of the currently generated word unit. During the generation process, the candidate word units C1~m can significantly reduce the number of subsequent word units that the model needs to consider, especially when the prompt provides a specific context, the model can predict more accurately and improve the generation efficiency and quality. The output layer is increased to 4, so that the model can predict the subsequent 4 word units at one time, instead of predicting one by one like the LLM in the related art. This means that when the model generates "1+1 equals", it can simultaneously predict and generate the two word units of "how much", even if they do not appear completely in the prompt. Predicting multiple word units at one time can significantly reduce the calculation steps when generating long texts, and theoretically can increase the reasoning speed to four times the original. Predicting subsequent word units at the same time helps the model generate more coherent sequences and reduce the context incoherence problem that may be caused by serial prediction.

[0138] Optionally, in the LM Head layer, the number of prediction heads can be increased and the weight matrix can be adjusted so that the model can output the probability distribution of multiple tokens at the same time. This usually involves innovations in the model architecture, such as sharing parameters between multiple heads to control the growth of the number of parameters.

[0139] In the embodiments of the present application, by introducing candidate word units C1~m and a mechanism for predicting multiple word units at one time, the performance and efficiency of large language models in sequence generation tasks are optimized, especially for tasks that require processing large amounts of text and long sequences.

[0140] In this embodiment, by introducing innovations in the structure and prediction mechanism of LLM, it is intended to solve the limitations of LLM in the related art in terms of accuracy, prediction probability optimization, training efficiency and reasoning speed. By taking the token that has appeared in the prompt (i.e., the C1~m sequence) as an additional input, the model can consider more contextual information when predicting. The LLM in the related art needs to select the next word from the full vocabulary (which may contain tens of thousands to hundreds of thousands of words) when predicting. In the embodiment of the present application, by considering the limited C1~m that appears in the prompt, the search range during prediction can be significantly reduced. The above method can effectively improve the generation speed of the model. At the same time, due to the reduction of the search space, the probability of prediction will be more concentrated, which improves the confidence and quality of the prediction.

[0141] Optionally, although additional prediction heads are added to the output layer (for example, from 1 head to 4 heads), the number of parameters actually added is not large because these heads share the hidden layer results of the last layer of Transformer. This means that the complexity and computational requirements of the model have not increased significantly, and the training speed and efficiency can be guaranteed. The design of shared parameters not only maintains the training speed of the model, but also utilizes the information of the existing hidden layer and avoids redundant calculations. It is an effective strategy to improve model performance while controlling computational costs. In actual text generation, the model can predict the subsequent 4 tokens at a time, instead of predicting them one by one like the related art LLM model. The above changes are particularly effective when generating long texts, which can reduce the inference time to a quarter of the original, greatly improve the generation efficiency, and reduce the delay of generating text. This mechanism is of great significance for application scenarios that require fast response, such as real-time dialogue systems and automatic summary generation.

[0142] In summary, the embodiments of the present application improve the accuracy, efficiency and speed of large language models in sequence generation tasks by optimizing input information utilization, reducing the prediction range, sharing parameters to control computational complexity, and predicting multiple words at one time. The above improvements not only help improve the performance of the model in specific application scenarios, but also provide a more efficient technical solution for large-scale text generation and processing.

[0143] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 2 A text generation device according to the text generation method shown.

[0144] Figure 5 is a schematic diagram of a text generation device according to an embodiment of the present application, such as Figure 5 As shown, the text generating device 500 may include: a determining unit 502 , a first generating unit 504 and a display unit 506 .

[0145] The determination unit 502 is configured to respond to an input instruction applied to the operation interface and determine first text data corresponding to the input instruction, wherein the first text data includes a plurality of first word units.

[0146] The first generating unit 504 is used to generate second text data based on the first text data through multiple iterations, wherein in the same iteration process, multiple second word-grams in the second text data are generated based on the first text data and candidate word-grams in the first text data, and the candidate word-grams are used to represent word-grams corresponding to the same iteration process.

[0147] The display unit 506 is used to display the second text data on the operation interface.

[0148] Here, the first determination unit 502, the first generation unit 504, and the first display unit 506 correspond to steps S202 to S206 in the embodiment, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 704) and processed by one or more processors (e.g., processors 702a, 702b..., 702n), and the above units can also be run as part of the device in the computer terminal A provided in the following embodiment.

[0149] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 3 A text generation device according to the text generation method shown.

[0150] Figure 6 is a schematic diagram of another text generation device according to an embodiment of the present application, such as Figure 6 As shown, the text generating device 600 may include: a first calling unit 602 , a second generating unit 604 and a second calling unit 606 .

[0151] The first calling unit 602 is used to obtain first text data by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes first text data, and the first text data includes a plurality of first word units.

[0152] The second generating unit 604 is used to generate second text data based on the first text data through multiple iterations, wherein in the same iteration process, multiple second word-grams in the second text data are generated based on the first text data and candidate word-grams in the first text data, and the candidate word-grams are used to represent word-grams corresponding to the same iteration process.

[0153] The second calling unit 606 is configured to output the second text data by calling the second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the second text data.

[0154] Here, the first calling unit 602, the second generating unit 604, and the second calling unit 606 correspond to steps S302 to S306 in the above embodiment, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 704) and processed by one or more processors (e.g., processors 702a, 702b..., 702n), and the above units can also be run as part of the device in the computer terminal A provided in the following embodiment.

[0155] In the text generation device, if an input instruction corresponding to the corresponding operation performed on the operation interface is detected, the first text data containing multiple first word units can be analyzed from the input instruction. The second text data can be generated by multiple iterations according to the first text data, wherein the multiple second word units in the second text data in the same iteration process are generated by combining the first text data and the candidate word units therein. The generated second text data can be displayed on the operation interface. In this embodiment, the first word unit that has been acquired can be used to predict multiple second word units at one time, avoiding the situation where the quality of text generation is reduced due to only being able to predict one word unit each time until the entire text data is obtained. And the second word unit can be generated by combining the first text data and the candidate word units therein. The contextual relationship of multiple word units can be considered at the same time through the above method, so that the original intention or context of the user can be grasped more accurately, thereby maintaining longer coherence and logical consistency when generating text. Then the technical effect of text generation accuracy is achieved, and the technical problem of low text generation accuracy is solved.

[0156] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0157] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0158] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the text generation method: responding to an input instruction acting on an operation interface, determining the first text data corresponding to the input instruction; based on the first text data, generating second text data through multiple iterations; and displaying the second text data on the operation interface.

[0159] Optionally, Figure 7 is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 7 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 702 , a memory 704 and a transmission device 706 .

[0160] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the text generation method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned text generation method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computer terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] Using the embodiment of the present application, a text generation method is provided. In the embodiment of the present application, the first word element that has been obtained can be used to predict multiple second word elements at one time, avoiding the situation where the quality of text generation is reduced due to only being able to predict one word element each time until the entire text data is obtained. And the second word element can be generated in combination with the first text data and the candidate word elements therein. The above method can simultaneously consider the contextual relationship of multiple word elements, so that the original intention or context of the user can be grasped more accurately, thereby maintaining longer coherence and logical consistency when generating text. Thereby achieving the technical effect of text generation accuracy and solving the technical problem of low text generation accuracy.

[0162] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID for short), a PAD, or other terminal devices. Figure 7 It does not limit the structure of the above-mentioned computer terminal A. For example, the computer terminal A may also include Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 7 Different configurations shown.

[0163] A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0164] The embodiment of the present application also provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the text generation method provided in the first embodiment.

[0165] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0166] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the above steps of the embodiment of the present application.

[0167] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0168] Figure 8 It is a block diagram of an electronic device according to a text generation method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0169] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0170] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 804, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0171] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the text generation method. For example, in some embodiments, the text generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the text generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the text generation method in any other appropriate manner (e.g., by means of firmware).

[0172] The embodiment of the present application further provides a computer program product. Optionally, in the present embodiment, the computer program product may include a computer program, and when the computer program is executed by a processor, the text generation method of the embodiment of the present application is implemented.

[0173] According to an embodiment of the present application, a text generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0174] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Fig. 9 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a text generation method according to an embodiment of the present application, such as Fig. 9As shown, the computer terminal 90 (or mobile device) may include one or more (902a, 902b, ..., 902n are used to illustrate in the figure) processors 902 (the processor 902 may include but is not limited to a processing device such as a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA)), a memory 904 for storing data, and a transmission device 906 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig. 9 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig. 9 More or fewer components as shown, or with Fig. 9 Different configurations shown.

[0175] Fig. 9 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned computer terminal 90 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Fig.10 The block diagram shows the use of the above Fig. 9 The computer terminal 90 (or mobile device) is shown as an embodiment of a computing node in the computing environment 1001 .

[0176] Fig.10 is a structural block diagram of a computing environment of a text generation method according to an embodiment of the present application, such as Fig.10 As shown, computing environment 1001 includes multiple computing nodes (such as servers) running on a distributed network (shown as 1010-1, 1010-2, ...) in the figure. The computing nodes all contain local processing and memory resources, and end users 1002 can remotely run applications or store data in computing environment 1001. Applications can be provided as multiple services 1020-1, 1020-2, 1020-3 and 1020-4 in computing environment 1601, representing services "F", "G", "I" and "H" respectively.

[0177] The end user 1002 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 1002's provision and / or request can be provided to the entry gateway 1030. The entry gateway 1030 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 1001).

[0178] Services are provided or deployed based on various virtualization technologies supported by the computing environment 1001. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system so that multiple workloads can run on a single operating system instance.

[0179] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Fig.10 As shown, service 1020-2 can be equipped with one or more Pods 1040-1, 1040-2, ..., 1040-N (collectively referred to as Pods). Pods can include a proxy 1045 and one or more containers 1042-1, 1042-2, ..., 1042-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 1045 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with Pods similar to Pods.

[0180] During operation, executing a user request from end user 1002 may require invoking one or more services in computing environment 1001, and executing one or more functions of a service may require invoking one or more functions of another service. Fig.10 As shown, service "F" 1020-1 receives a user request from end user 1002 from ingress gateway 1030, service "F" 1020-1 may call service "G" 1020-2, and service "G" 1020-2 may request service "I" 1020-3 to perform one or more functions.

[0181] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0182] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0184] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0186] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory for short), an optical fiber, a portable compact disk read-only memory (CD-ROM for short), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD), a monitor for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a path ball), through which the user may provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0189] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0190] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0191] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0193] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory, random access memory, mobile hard disk, disk or optical disk, etc., which can store program code.

[0196] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A text generation method, characterized in that: include: In response to an input instruction acting on the operation interface, determining first text data corresponding to the input instruction, wherein the first text data includes a plurality of first word units; Based on the first text data, second text data is generated in a multiple-iteration manner, wherein in a same iterative process, a plurality of second word-grams in the second text data are generated based on the first text data and candidate word-grams in the first text data, the candidate word-grams are used to represent word-grams corresponding to the same iterative process, and the candidate word-grams are a continuous segment of word-grams selected from the first word-grams; Displaying the second text data on the operation interface; Among them, based on the first text data, the second text data is generated through multiple iterations, including: based on the candidate word elements of the current iteration process, the second word elements corresponding to the historical iteration process before the current iteration process and the first text data, the second word elements corresponding to the current iteration process are generated; the second word elements corresponding to multiple iteration processes are summarized to obtain the second text data.

2. The method according to claim 1, characterized in that: The generating second text data based on the first text data by multiple iterations includes: The first text data is input into a text generation model, and the second text data is generated by using the text generation model through multiple iterations.

3. The method according to claim 2, characterized in that The step of generating the second word-gram corresponding to the current iteration process based on the candidate word-gram of the current iteration process, the second word-gram corresponding to the historical iteration process before the current iteration process, and the first text data includes: Obtaining a second word-gram corresponding to a historical iteration process before a current iteration process, wherein the second word-gram corresponding to the historical iteration process is a word-gram generated by the text generation model in the historical iteration process; Determining, from the plurality of first word-grams, candidate word-grams to be input into the text generation model in the current iteration process; The candidate word-gram, the second word-gram corresponding to the historical iteration process and the first text data are input into the text generation model, and the second word-gram corresponding to the current iteration process is generated by using the text generation model.

4. The method according to claim 3, characterized in that The step of determining, from the plurality of first word-grams, a candidate word-gram to be input into the text generation model in the current iteration process comprises: Determining, from the plurality of first word-grams, a target word-gram corresponding to the second word-gram corresponding to the historical iteration process; A preset number of word-grams located after the target word-gram are obtained from the plurality of first word-grams to obtain the candidate word-grams.

5. The method according to claim 3 or 4, characterized in that: The number of the candidate word-grams is the same as the number of the second word-grams corresponding to the current iteration process.

6. The method according to claim 3, characterized in that The text generation model comprises: an embedding layer, a feature processing layer and a plurality of output layers; the step of inputting the candidate word-gram, the second word-gram corresponding to the historical iteration process and the first text data into the text generation model, and using the text generation model to generate the second word-gram corresponding to the current iteration process comprises: Encoding the candidate word using the embedding layer to obtain a candidate encoding vector, encoding the second word corresponding to the historical iteration process using the embedding layer to obtain a historical encoding vector, and encoding the first text data using the embedding layer to obtain a text encoding vector; Inputting the candidate encoding vector, the historical encoding vector and the text encoding vector into the feature processing layer, and obtaining the hidden state vector output by the feature processing layer; The hidden state vectors are respectively input into the multiple output layers, and second word units corresponding to the current iteration process output by the multiple output layers are obtained, wherein the second word units corresponding to the current iteration process correspond to the multiple output layers one by one.

7. The method according to claim 6, characterized in that The feature processing layer includes a plurality of sub-layers; the step of inputting the candidate encoding vector, the historical encoding vector, and the text encoding vector into the feature processing layer, and obtaining the hidden state vector output by the feature processing layer includes: The historical encoding vector and the text encoding vector are input into the first sublayer of the multiple sublayers, and the candidate encoding vector is input into the last sublayer of the multiple sublayers, and the hidden state vector output by the feature processing layer is obtained.

8. The method according to claim 6, characterized in that The step of inputting the hidden state vectors to the multiple output layers respectively and obtaining the second word units corresponding to the current iteration process output by the multiple output layers includes: Mapping the hidden state vector to a preset vocabulary using the multiple output layers to obtain probability values ​​of multiple preset word units included in the preset vocabulary; Based on the probability values ​​of the plurality of preset word-grams, a second word-gram corresponding to the current iteration process output by the plurality of output layers is determined.

9. The method according to claim 2, characterized in that: The method further comprises: Output multiple preset generation models, where different generation models generate different numbers of tokens in the same iteration process; In response to a selection instruction acting on the operation interface, a preset generation model corresponding to the selection instruction is determined from the multiple preset generation models to obtain the text generation model.

10. A text generation method, characterized in that: include: Acquire first text data by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes the first text data, and the first text data includes a plurality of first word units; Based on the first text data, second text data is generated in a multiple-iteration manner, wherein in a same iterative process, a plurality of second word-grams in the second text data are generated based on the first text data and candidate word-grams in the first text data, the candidate word-grams are used to represent word-grams corresponding to the same iterative process, and the candidate word-grams are a continuous segment of word-grams selected from the first word-grams; Outputting the second text data by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the second text data; Among them, based on the first text data, the second text data is generated through multiple iterations, including: based on the candidate word elements of the current iteration process, the second word elements corresponding to the historical iteration process before the current iteration process and the first text data, the second word elements corresponding to the current iteration process are generated; the second word elements corresponding to multiple iteration processes are summarized to obtain the second text data.

11. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 10 when running.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images