Text generation method and apparatus, electronic device, and storage medium

Abstract

The present application is suitable for the technical field of natural language processing, and provides a text generation method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a predicted token and a first hidden representation corresponding to text to be predicted, wherein the predicted token includes a new token obtained by performing inference on the text to be predicted, and the first hidden representation is an embedding representation obtained by performing, by means of a transformer layer, feature extraction on the text to be predicted; using the predicted token and the first hidden representation as an input of a trained acceleration network to obtain a candidate token outputted by the acceleration network, wherein the acceleration network is used for inferring the new token on the basis of the first hidden representation and the predicted token to obtain the candidate token; and on the basis of the predicted token and the candidate token, determining first target text corresponding to the text to be predicted. The present application can improve the quality of generated text.

Description

Text generation method and device, electronic equipment and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202411000687.9, filed on July 24, 2024, and entitled "Text generation method and device, electronic equipment and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application belongs to the technical field of natural language processing, and particularly relates to a text generation method and device, an electronic equipment and a computer readable storage medium. BACKGROUND

[0003] Text generation technology is an important branch of the field of natural language processing (NLP), which can predict the next word or short sentence that the user may want to input according to the current input text of the user, thereby improving the efficiency of text input and reducing human errors. Among them, speculation sampling is a reasoning acceleration technology for accelerating text generation. In the text generation process, after generating the next word by the generation model, an additional draft model is used to directly predict the word that the generation model is likely to generate next according to the word generated by the generation model, and the word is used as a candidate word. Finally, the generation model selects the acceptable word from the candidate words without generating the word again.

[0004] However, when accelerating text generation by the method of speculation sampling, the draft model only generates candidate words according to the last word generated by the generation model, and the acceptance rate of the candidate words depends heavily on the accuracy of the draft model. When the accuracy of the draft model is limited, it may lead to a decrease in the efficiency of text generation or a decrease in the quality of the text.

[0005] SUMMARY

[0006] The embodiments of the present application provide a text generation method, device, electronic equipment and storage medium, which can improve the quality of the generated text.

[0007] In a first aspect, the embodiments of the present application provide a text generation method, comprising:

[0008] obtaining a prediction word and a first hidden representation corresponding to a to-be-predicted text, the prediction word comprising a new word obtained by reasoning the to-be-predicted text, and the first hidden representation being an embedding representation obtained by extracting features of the to-be-predicted text through a transformer layer;

[0009] input the predicted wordpiece and the first hidden representation into a trained acceleration network to obtain a candidate wordpiece output by the acceleration network, the acceleration network being configured to infer a new wordpiece based on the first hidden representation and the predicted wordpiece to obtain the candidate wordpiece;

[0010] determine the first target text corresponding to the text to be predicted based on the predicted wordpiece and the candidate wordpiece.

[0011] In a second aspect, an embodiment of the present application provides a text generation apparatus, comprising:

[0012] a prediction module configured to obtain a predicted wordpiece and a first hidden representation corresponding to a text to be predicted, the predicted wordpiece comprising a new wordpiece inferred from the text to be predicted, and the first hidden representation being an embedding representation obtained by performing feature extraction on the text to be predicted through a transformer layer;

[0013] a candidate wordpiece generation module configured to input the predicted wordpiece and the first hidden representation into a trained acceleration network to obtain a candidate wordpiece output by the acceleration network, the acceleration network being configured to infer a new wordpiece based on the first hidden representation and the predicted wordpiece to obtain the candidate wordpiece;

[0014] a first target text generation module configured to determine a first target text corresponding to the text to be predicted based on the predicted wordpiece and the candidate wordpiece.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the text generation method of the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer storage medium storing a computer program, the computer program being executable by a processor to implement the steps of the text generation method of the first aspect.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, when the computer program product is executed on an electronic device, causing the electronic device to execute the text generation method of the first aspect.

[0018] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0019] In the embodiment of the present application, in the process of text generation, the predicted word element corresponding to the to-be-predicted text and the first hidden representation are input into the trained acceleration network. Since the predicted word element is a new word element inferred from the to-be-predicted text, and the first hidden representation is an embedding representation obtained by feature extraction of the to-be-predicted text by the transformer layer, it can reflect the semantic information and context dependency of the word element of the to-be-predicted text. Therefore, in the process of inferring the candidate word element according to the predicted word element, the acceleration network can better infer the candidate word element by combining the semantic information and context dependency of the word element of the to-be-predicted text, and obtain a candidate word element with higher accuracy. That is, by increasing the first hidden representation, the dependence on the accuracy of the acceleration network is reduced. At the same time, since the above processing can generate a candidate word element with higher accuracy, and the receiving rate of the candidate word element with higher accuracy is higher, the generation efficiency of the finally generated first target text can be better guaranteed and the text quality can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0021] FIG. 1 is a flow diagram of a text generation method according to an embodiment of the present application;

[0022] FIG. 2 is a structural diagram of a large language model and an acceleration network according to an embodiment of the present application;

[0023] FIG. 3 is a flow diagram of another text generation method according to an embodiment of the present application;

[0024] FIG. 4 is a structural diagram of a text generation device according to an embodiment of the present application;

[0025] FIG. 5 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0027] It should be understood that the term "comprising" as used in the specification and in the claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] It should also be understood that the term "and / or" as used in the specification and in the claims indicates any combination of one or more of the associated listed items and all possible combinations thereof.

[0029] In addition, in the description of the specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0030] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized.

[0031] Embodiment I:

[0032] Figure 1 shows a flowchart of a text generation method according to an embodiment of the present application, which is described in detail as follows:

[0033] In step S101, a prediction word and a first hidden representation corresponding to a to-be-predicted text are obtained. The prediction word includes a new word obtained by reasoning the to-be-predicted text. The first hidden representation is an embedding representation obtained by feature extraction of the to-be-predicted text through a transformer layer.

[0034] The transformer layer refers to a network layer composed of two key parts, a multi-head self-attention layer (Multi-Head Self-Attention Layer) and a position-wise feed-forward network (FFN).

[0035] It can be understood that the specific manner of obtaining the above-mentioned to-be-predicted text can be determined according to actual application scenarios. For example, when the above-mentioned text generation method is applied to literary creation, the author can input a beginning or a plot fragment, etc., the server obtains the information input by the author as the to-be-predicted text, predicts and generates a possible development path according to the to-be-predicted text, obtains the first target text, and returns it to the author, which can better stimulate the author's inspiration and help the author explore the unexpected story direction.

[0036] Optionally, when the prediction word unit corresponding to the to-be-predicted text is obtained, the to-be-predicted text can be taken as the input of the pre-trained generation model, and the new word unit of the to-be-predicted text is inferred through the generation model to automatically generate the prediction word unit with good accuracy. The above-mentioned generation model can be a recurrent neural network, a rule-based model, a statistical language model or a generative adversarial network, etc., and the embodiments of the present application do not make specific limitations thereto.

[0037] In some embodiments, only the next word unit of the last word unit in the to-be-predicted text can be predicted, and the predicted word unit is taken as the prediction word unit.

[0038] In the embodiments of the present application, the multi-head self-attention mechanism and the position feed-forward network in the transformer layer perform feature extraction processing on the to-be-predicted text. In the feature extraction processing, each word unit in the to-be-predicted text can be converted into an embedding representation considering the information of other word units in the to-be-predicted text, i.e., each word unit can be converted into an embedding representation obtained by adding the vector representation and the position representation, and finally, the first hidden representation corresponding to the to-be-predicted text can be obtained according to the embedding representation of each word unit. Since the vector representation can reflect the semantic information of the word unit, and the position representation can reflect the relative position of the word unit in the to-be-predicted text, the first hidden representation obtained according to the embedding representation of each word unit can better reflect the semantic information and context dependency of the to-be-predicted text, so that the subsequent acceleration model can effectively understand and process the to-be-predicted text when generating the candidate word unit according to the first hidden representation.

[0039] In the embodiments of the present application, the embedding representation of each word unit in the to-be-predicted text is extracted through the transformer layer to obtain the first hidden representation corresponding to the to-be-predicted text. The embedding representation of the word unit can reflect the semantic information of the word unit and the relative position of the word unit in the to-be-predicted text, so that the obtained first hidden representation can better reflect the semantic information and context dependency of the to-be-predicted text, and can be directly used as the input of the acceleration network for subsequent prediction of the candidate word unit. The acceleration network does not need to process long text, and the requirement for the text processing capability of the acceleration network is reduced.

[0040] Step S102, taking the predicted word and the first hidden representation as inputs of the trained acceleration network, obtaining candidate words output by the acceleration network, the acceleration network being used to infer a new word based on the first hidden representation and the predicted word, and obtaining the candidate words.

[0041] Optionally, when inferring a new word based on the first hidden representation and the predicted word, the acceleration network can infer only the next word of the predicted word (i.e., the word at the next position of the predicted word), or can infer multiple words subsequent to the predicted word (i.e., multiple words at consecutive positions after the predicted word).

[0042] For example, assuming that one word occupies one word position, the word position of the last word in the text to be predicted is 5, and the word position of the predicted word is 6, the acceleration network can infer the words at word positions 7, 8, and 9 based on the predicted word and the text to be predicted, and obtain the candidate words at the three word positions. For each word position, multiple candidate words corresponding to the word position can be inferred, and the first target text is determined by selecting from the multiple candidate words.

[0043] In some embodiments, the acceleration network is a lightweight network, such as an autoregressive model including an encoding layer, three attention layers, and a prediction layer. Since the first hidden representation corresponding to the text to be predicted is obtained by a transformer layer, the input of the acceleration model is a vector form of the first hidden representation, and therefore, the lightweight acceleration network can quickly and accurately generate candidate words based on the first hidden representation, without complex processing of long texts, effectively avoiding problems such as calculation bottlenecks when a lightweight acceleration network processes long texts, and facilitating practical applications in resource-limited scenarios such as edge devices.

[0044] In the embodiments of the present application, since the first hidden representation is an embedding representation embedding the vector representation and the position representation of each word in the text to be predicted, it can reflect the semantic information and context dependency of each word in the text to be predicted, and therefore, when the first hidden representation and the predicted word are input into the trained acceleration network, the acceleration network can infer the new word of the predicted word based on the semantic information and context dependency of each word before the predicted word, thereby obtaining candidate words with high accuracy, i.e., the acceleration network can generate candidate words with high accuracy without relying on an acceleration network with high accuracy, reducing the dependence on the accuracy of the acceleration network.

[0045] Step S103, determining the first target text corresponding to the text to be predicted based on the predicted word and the candidate words.

[0046] After the candidate word pieces are obtained, the candidate word pieces that can be accepted can be verified through a generation model for generating predicted word pieces or other manners, and then the first target text corresponding to the text to be predicted is determined according to the predicted word pieces and the word pieces that can be accepted.

[0047] For example, it is assumed that the acceleration network generates candidate word pieces according to a probability distribution of predicting new word pieces from the predicted probability distribution when generating candidate word pieces, and a word piece is randomly selected from the predicted probability distribution as a candidate word piece.

[0048] When verifying the candidate word pieces that can be accepted (referred to as target candidate word pieces) in the obtained candidate word pieces, the probability of the candidate word piece can be compared with a randomly generated random number. If the probability of the candidate word piece is greater than the random number, the candidate word piece can be accepted, otherwise, the candidate word piece is rejected. It can be understood that the rejected candidate word piece will not usually appear in the finally determined first target text.

[0049] In the embodiments of the present application, the predicted word pieces corresponding to the text to be predicted and the first hidden representation are input into the trained acceleration network. Since the predicted word pieces are new word pieces inferred from the text to be predicted, and the first hidden representation is an embedding representation extracted from the text to be predicted by the transformer layer, it can reflect the semantic information and context dependency of each word piece in the text to be predicted. Therefore, the acceleration network can better infer the candidate word pieces by combining the semantic information and context dependency of the word pieces in the text to be predicted during the process of inferring the candidate word pieces from the predicted word pieces, and obtain candidate word pieces with high accuracy. That is, by increasing the first hidden representation as input, the dependence on the accuracy of the acceleration network is reduced. At the same time, since the above processing can generate candidate word pieces with high accuracy, and the acceptance rate of the candidate word pieces with high accuracy is high, the generation efficiency of the finally generated first target text can be better guaranteed and the text quality can be improved.

[0050] It should be noted that after the candidate word pieces that can be accepted (referred to as target candidate word pieces) are determined, if the target candidate word piece and the predicted word piece cannot determine a complete text (for example, the last sentence of the text is a complete sentence), the target candidate word piece can be used as a new predicted word piece to continue generating candidate word pieces through the acceleration network until a complete text is obtained, so as to obtain the required first target text.

[0051] In some embodiments, the above acceleration network includes an encoding layer, an attention layer and a prediction layer, and the above predicted word pieces and the above first hidden representation are input into the acceleration network to obtain the candidate word pieces output by the acceleration network, which includes:

[0052] A1, performing feature extraction processing on the predicted token by the encoding layer to obtain a token feature representation.

[0053] A2, performing attention fusion on the token feature representation by the attention layer according to the first hidden representation to obtain a fused token feature representation.

[0054] A3, processing the fused token feature representation output by the attention layer by the prediction layer to obtain the candidate token.

[0055] Specifically, after the encoding layer outputs the extracted token feature representation of the predicted token, the token feature vector and the first hidden representation are input into the attention layer together. The attention layer can perform attention fusion on the token feature representation according to the first hidden representation, so that the token feature representation can interact with the embedding representation of each token in the first hidden representation, and receive the prediction information of the new token expressed by the first hidden representation. Furthermore, when the prediction layer processes the fused token feature representation, it can better understand the internal relationship and dependency between the predicted token and each token in the text to be predicted, thereby effectively improving the accuracy of the obtained candidate token.

[0056] In some embodiments, the attention layer can use a causal-based cross-attention mechanism to perform attention fusion. In the causal attention mechanism, only the information of the current position and the previous positions can be considered when generating the information of the new position, which can ensure that the generated information of the new position has correct temporal dependency. The cross-attention mechanism can notice different parts of the entire input sequence when generating the information of the new position. In the decoding process, it can dynamically select relevant information from the input sequence according to the context. However, it also considers the output information when generating the information of the new position. Therefore, in the embodiments of the present application, the cross-attention mechanism is improved based on the causal attention mechanism to obtain a causal-based cross-attention mechanism. When the attention layer performs attention fusion on the token feature vector according to the first hidden representation, it only dynamically selects relevant information from the first hidden representation according to the context relationship of each token in the first hidden representation to fuse with the token feature representation. At the same time, it can consider the information interaction between different tokens, improve the accuracy of the fused token feature representation, and ensure that the fused token feature representation has correct temporal dependency.

[0057] In the embodiments of the present application, by attention fusion processing, the semantic information and context dependency of each word in the first hidden representation are fused in the word feature representation of the predicted word, so that when the prediction layer predicts according to the fused word feature representation, it can effectively understand the internal relationship and context dependency between the predicted word and each word in the text to be predicted, improve the accuracy and reliability of the prediction layer decision, that is, improve the accuracy and reliability of the generated candidate word.

[0058] In some embodiments, the number of the first hidden representations is M, the acceleration network includes M attention layers connected in series, one of the first hidden representations corresponds to one of the attention layers, and M is an integer greater than 1.

[0059] Since different transformer layers in the series of transformer layers usually have their specific functions and focus points, the hidden representations output by different transformer layers are different hidden representations extracted based on different angles and abstraction levels, therefore, in the embodiments of the present application, in order to further improve the accuracy of the candidate word, the feature extraction processing of the text to be predicted can be performed through a plurality of transformer layers connected in series, and then M first hidden representations extracted by M different transformer layers are obtained, so that the acceleration network can better predict the candidate word according to the M first hidden representations reflecting different information.

[0060] In the embodiments of the present application, M is the number of attention layers in the acceleration network, that is, the acceleration network includes M attention layers connected in series, and the M first hidden representations correspond to the M attention layers one by one. It should be pointed out that the deeper the first hidden representation of the corresponding transformer layer, the deeper the corresponding attention layer.

[0061] It should be noted that the number of transformer layers connected in series can be greater than or equal to M, and when the number of transformer layers is greater than M, the first hidden representations extracted by M transformer layers can be obtained. For example, assuming that 4 first hidden representations need to be extracted by model A, and the model A includes 12 transformer layers connected in series, the 12 transformer layers can be evenly divided into 4 parts (i.e., the first to third transformer layers are the first part, the fourth to sixth transformer layers are the second part, the seventh to ninth transformer layers are the third part, and the tenth to twelfth transformer layers are the fourth part).

[0062] When the four first hidden representations are obtained through the model A, the first hidden representation output by the third transformer layer is obtained as the first first hidden representation, the first hidden representation output by the sixth transformer layer is obtained as the second first hidden representation, the first hidden representation output by the ninth transformer layer is obtained as the third first hidden representation, and the first hidden representation output by the twelfth transformer layer is obtained as the fourth first hidden representation, thereby obtaining four different first hidden representations.

[0063] As shown in FIG. 2, assuming that the acceleration network sequentially includes attention layer 1, attention layer 2, attention layer 3, and attention layer 4, the first first hidden representation corresponds to attention layer 1, the second first hidden representation corresponds to attention layer 2, the third first hidden representation corresponds to attention layer 3, and the fourth first hidden representation corresponds to attention layer 4.

[0064] Correspondingly, the above step A2 includes:

[0065] The above word feature representation is subjected to attention fusion according to the corresponding first hidden representation through the first attention layer, thereby obtaining the fused word feature representation.

[0066] For each of the attention layers after the first attention layer, the fused word feature representation output by the previous attention layer is subjected to attention fusion according to the corresponding first hidden representation, until the fused word feature representation output by the Mth attention layer is obtained.

[0067] The above step A3 includes:

[0068] The fused word feature representation output by the Mth attention layer is processed through the prediction layer, thereby obtaining the candidate word.

[0069] Since in the multiple transformer layers in series, the output (first hidden representation) of the transformer layer gradually accumulates more context information as the depth increases, so that the deeper transformer layer can capture more semantic information and context dependency, therefore, when the M attention layers in the acceleration network are attention fused according to the M first hidden representations on the word feature representation, each attention layer will be attention fused according to the first hidden representation corresponding to the word feature representation output by the previous attention layer in turn, that is, the semantic information and context dependency in different first hidden representations are gradually fused in the word feature representation, so that the finally obtained fused word feature representation can fuse multiple levels of semantic information and context dependency, help the prediction layer better understand the complex semantic relationship and context dependency between the predicted word and the predicted text, and is beneficial to improve the accuracy of the candidate word.

[0070] As shown in FIG. 2, the acceleration network includes 4 attention layers, in which the attention layer 1 is attention fused according to the first hidden representation 1 on the word feature representation output by the encoding layer, to obtain the fused word feature representation and output; the attention layer 2 is attention fused according to the second hidden representation 2 on the fused word feature representation output by the attention layer 1, to obtain the fused word feature representation and output; and so on, until the fused word feature representation output by the attention layer 4 is obtained.

[0071] The fused word feature representation output by the fourth attention layer is taken as the input of the prediction layer, and the prediction layer processes the fused word feature representation to obtain the required candidate word.

[0072] In the embodiment of the application, since the M first hidden representations obtained are extracted by different transformer layers in the multiple transformer layers in series, which reflect different levels of semantic information and context dependency, therefore, according to the M first hidden representations, the word feature representation of the predicted word is attention fused in turn, which can better fuse the semantic information and context dependency in different first hidden representations into the word feature representation, help the prediction layer better understand the complex semantic relationship and context dependency between the predicted word and the predicted text, and is beneficial to improve the accuracy of the candidate word.

[0073] In some embodiments, after the above-mentioned processing of the fused word feature representation output by the attention layer by the prediction layer to obtain the candidate word, the method further includes:

[0074] The steps of A1 to A3 are re-executed K-1 times based on the latest candidate word unit as a new predicted word unit and the first hidden representation, K being an integer greater than 1.

[0075] Correspondingly, determining the first target text corresponding to the to-be-predicted text based on the predicted word unit and the candidate word unit comprises:

[0076] Determining the first target text corresponding to the to-be-predicted text based on the predicted word unit and the K candidate word units.

[0077] To improve the efficiency of text generation, the acceleration network can predict K word units after the predicted word unit based on the predicted word unit and the first hidden representation, obtain K candidate word units, and then determine the first target text corresponding to the to-be-predicted text based on the predicted word unit and the K candidate word units.

[0078] In the process of predicting K candidate word units based on the predicted word unit and the first hidden representation, the predicted layer processes the fused word unit feature representation output by the attention layer to obtain a candidate word unit (assuming to be candidate word unit 1), and the candidate word unit 1 is the latest candidate word unit. At this time, the latest candidate word unit is taken as a reference predicted word unit, and the steps of A1 to A3 are re-executed according to the reference predicted word unit, and the steps are repeated K-1 times to obtain K-1 candidate word units, and K candidate word units are obtained in combination with the candidate word units obtained in the first round.

[0079] That is, after feature extraction processing is performed on the reference predicted word unit to obtain a new word unit feature representation, the attention layer performs attention fusion on the new word unit feature representation based on the first hidden representation to obtain the fused word unit feature representation, and the predicted layer processes the fused word unit feature representation output by the attention layer to obtain a candidate word unit (candidate word unit 2).

[0080] By analogy, the steps of A1 to A3 are re-executed based on the candidate word unit 2, and the steps are executed K-1 times to obtain K candidate word units.

[0081] It should be noted that in some embodiments, at least two candidate word units corresponding to each word unit position can be generated when predicting and generating candidate word units. At this time, at least two candidate word unit sequences with a length of K can be obtained.

[0082] For example, it is assumed that two candidate word units are generated for each word unit position, and it is assumed that one word corresponds to one word unit position. The value of K is set to 9, that is, nine words after the predicted word unit are generated.

[0083] Assuming that the text to be predicted is "On a distant continent, there is a mysterious forest, " and the predicted word is "legend, " the candidate word for the first word position can be generated according to the predicted word "legend, " which is candidate word M1 "inside" and candidate word N1 "this."

[0084] After obtaining two candidate words for the first word position, the word for the next word position is predicted according to the two candidate words, and the two candidate words for the next word position are obtained, that is, candidate word M2 is predicted according to candidate word M1 and the first hidden representation, and candidate word N2 is predicted according to candidate word N1 and the first hidden representation.

[0085] By analogy, two candidate word sequences of length 9 are obtained, such as candidate word sequence M "inside the life tree is hidden, " and candidate word sequence N "the plants and animals in this forest, " and when determining the first target text, the user can select from the two candidate word sequences, and the first target text is determined according to the user's selection of candidate words.

[0086] In the embodiments of the present application, the K candidate words after the predicted word are generated by accelerating the network, and K candidate words are obtained, so that when the candidate words that can be received are determined among the obtained candidate words, K candidate words can be verified at the same time, and one or more target candidate words are obtained, which can better accelerate the generation of the first target text and achieve good acceleration effect.

[0087] In some embodiments, the number of the above-mentioned first hidden representations is greater than 1, and the above-mentioned obtaining the predicted word and the first hidden representation corresponding to the text to be predicted comprises:

[0088] The above-mentioned text to be predicted is taken as the input of the large language model, and the above-mentioned predicted word and the above-mentioned first hidden representation output by the above-mentioned large language model are obtained, the above-mentioned large language model is used to perform feature extraction processing on the above-mentioned text to be predicted through a plurality of above-mentioned transformer layers connected in series, and the above-mentioned first hidden representation extracted by the last above-mentioned transformer layer is used for reasoning to obtain the above-mentioned predicted word.

[0089] The above-mentioned large language model (Large Language Model, LLM) refers to an artificial intelligence model trained on a large amount of data, which usually contains tens of billions or even more parameters, and can better understand and generate human language, and efficiently perform various natural language processing (Natural Language Processing, NLP) tasks such as text generation, question answering, translation, text summarization and sentiment analysis. The large language model in the embodiments of the present application comprises a plurality of transformer layers connected in series.

[0090] In some embodiments, the large language model described above can be constructed according to a transformer-based decoder-only structure. The large language model based on the transformer-based decoder-only structure is composed of a series of decoder layers, each of which includes a self-attention module and a feedforward neural network, i.e., the decoder layer is equivalent to a transformer layer. Based on a series of decoder components, combined with a large-scale training data set and a complex pre-training strategy, the large language model can achieve excellent performance on a series of language understanding and generation tasks.

[0091] After inputting the to-be-predicted text into the large language model, the large language model will first perform feature extraction processing on the to-be-predicted text in sequence through a plurality of transformer layers, i.e., after the first transformer performs feature extraction processing on the to-be-predicted text to obtain the first hidden representation corresponding to the first transformer layer, the first hidden representation output by the first transformer layer will be used as the input of the second transformer layer, and the second transformer layer will continue to perform feature extraction processing on the transformer layer to obtain the first hidden representation corresponding to the second transformer layer and output it. In this way, the first hidden representation output by the last transformer layer is obtained.

[0092] After obtaining the first hidden representation output by the last transformer layer, the large language model will perform reasoning according to the first hidden representation to obtain the required predicted word token. Since the first hidden representation output by the last transformer layer is the deepest transformer layer in the large language model, it contains the most semantic information and context dependency, therefore, reasoning according to the first hidden representation output by the transformer layer can maximize the accuracy of the obtained predicted word token.

[0093] In the embodiments of the present application, the large language model with good performance on natural language processing tasks is used to automatically generate the predicted word token corresponding to the to-be-predicted text, and in the process of reasoning to obtain the predicted word token, the transformer layer is used to perform feature extraction processing on the to-be-predicted text to obtain the first hidden representation needed subsequently, without the need for additional transformer layers to perform feature extraction processing on the to-be-predicted text, which can reduce the resources required and improve the efficiency of text generation.

[0094] In some embodiments, before the above-mentioned predicted word piece and the above-mentioned first hidden representation are input into the acceleration network as inputs of the acceleration network, the method further comprises:

[0095] The to-be-predicted text in the training data set is input into the large language model to obtain a second target text output by the large language model.

[0096] The second hidden representation corresponding to the first N-1 word pieces in the second target text is obtained by the large language model, and a target word piece is determined according to the word pieces other than the first N word pieces in the second target text, N being less than the number of word pieces in the second target text.

[0097] The Nth word piece in the second target text and the second hidden representation are input into the initial acceleration network to obtain a reference candidate word piece output by the initial acceleration network.

[0098] The initial acceleration network is updated according to the difference between the reference candidate word piece and the target word piece until the updated initial acceleration network meets the requirements, and the trained acceleration network is obtained.

[0099] The training data set refers to a pre-constructed data set containing a large number of examples, which include to-be-predicted texts and corresponding target texts, and is used to guide the pre-training model to better understand and generate human language. Optionally, the training data set can be a public data set such as GLUE, SQuAD, MNLI, etc.

[0100] To further improve the accuracy of the generated candidate word piece, when training the acceleration network, the to-be-predicted text in the training data set obtained can be input into the large language model, and each word piece of the reference text generated by the large language model is obtained to obtain the corresponding second target text, and then the second target text generated by the large language model is used as a training sample of the acceleration network to train the acceleration network. Since the second target text is generated by the large language model, using the second target text to train the acceleration network can make the candidate word piece generated by the acceleration network more consistent with the output distribution of the large language model, can improve the acceptance rate of the candidate word piece, and is conducive to improving the efficiency and quality of text generation.

[0101] Wherein, before training the acceleration network according to the second target text, the transformer layer in the large language model is used to perform feature extraction processing on the first N-1 word pieces in the second target text to obtain the second hidden representation corresponding to the first N word pieces in the second target text, and the target dimension is determined according to the word piece other than the first N word pieces in the second target text. At this time, the Nth word piece in the second target text is equivalent to a predicted word piece obtained by reasoning according to the first N-1 word pieces. Therefore, when training the acceleration network, the Nth word piece and the second hidden representation can be used as the input of the initial acceleration network. The initial acceleration network reasons according to the Nth word piece and the second hidden representation to obtain the reference candidate word piece.

[0102] After obtaining the reference candidate word piece, the initial acceleration network can be updated according to the difference between the reference candidate word piece and the target word piece as the correct value, so as to reduce the difference between the reference candidate word piece generated by the updated initial acceleration network and the target word piece, until the updated initial acceleration network meets the requirements (such as the accuracy reaching the threshold 0.95). The updated initial acceleration network that meets the requirements can be used as the trained acceleration network.

[0103] In some embodiments, a next-prediction loss function can be used as the loss function of the acceleration network. By minimizing the next-prediction loss, the acceleration network can gradually learn how to effectively predict the next word of the reference predicted word piece based on the context information in the second hidden representation, improve the coherence and accuracy of the generated reference candidate word piece with the previous text, and improve the coherence and accuracy of the finally generated text.

[0104] In some embodiments, the attention mechanism used by the attention layer of the acceleration network is a causal cross-attention mechanism. Through the causal cross-attention mechanism, the attention layer only interacts information between the reference predicted word piece and the second hidden representation during the training process, and does not focus on the reference candidate word piece that needs to be output by the acceleration network, so as to ensure that the acceleration network only needs to output the new word piece of the predicted word piece according to the predicted word piece and the hidden representation corresponding to the text before the predicted word piece when reasoning the new word piece of the predicted word piece. At the same time, the acceleration network can perform autoregressive decoding, taking the current output reference candidate word piece as input to obtain the next reference candidate word piece.

[0105] In the embodiments of the present application, since the second target text is generated by a large language model used for generating predicted word pieces in the future, the second target text conforms to the output distribution of the large language model. Therefore, the acceleration network is trained by using the second target text, so that the candidate word pieces generated by the trained acceleration network are more consistent with the output distribution of the large language model, that is, the accuracy of the generated candidate word pieces is improved, thereby improving the probability of the candidate word pieces being accepted and reducing the probability of the candidate word pieces needing to be regenerated, which is beneficial to improving the generation efficiency of the first target text.

[0106] In some embodiments, before determining the first target text corresponding to the to-be-predicted text based on the predicted word piece and the candidate word piece, the method further comprises:

[0107] Determining a target candidate word piece from the candidate word pieces and judging whether the target candidate word piece includes an end identifier.

[0108] In the case that the target candidate word piece does not include the end identifier, determining a new predicted word piece and a new first hidden representation based on the target candidate word piece.

[0109] Taking the new predicted word piece and the new first hidden representation as inputs of the acceleration network, obtaining a new candidate word piece output by the acceleration network.

[0110] Returning to the step of determining a target candidate word piece from the candidate word pieces and judging whether the target candidate word piece includes an end identifier based on the new candidate word piece, until the latest target candidate word piece includes the end identifier.

[0111] Correspondingly, the determining the first target text corresponding to the to-be-predicted text based on the predicted word piece and the candidate word piece comprises:

[0112] Determining the first target text according to all target candidate word pieces and the predicted word piece.

[0113] The end identifier refers to a symbol used to mark the end of a sequence in natural language processing. Optionally, the end identifier can be a special character, symbol or code value, etc., which can be set according to actual application scenarios, and is not limited specifically herein.

[0114] To ensure the integrity of the obtained first target text, after obtaining the candidate word piece generated by the acceleration network, the candidate word piece that can be accepted is verified first, and after obtaining the target candidate word piece, it is judged whether the target candidate word piece includes an end identifier.

[0115] If the end identifier is included in the target candidate word item, it indicates that the complete text can be obtained by combining the predicted word item, the text to be predicted, and the determined target candidate word item. At this time, the first target text can be directly determined according to the predicted word item and the target candidate word item.

[0116] If the end identifier is not included in the target candidate word item, it indicates that the complete text cannot be obtained according to the current target candidate word item, and the word item after the target candidate word item needs to be continuously generated.

[0117] As shown in FIG. 3, in the case where the end identifier is not included in the target candidate word item and the subsequent word item needs to be continuously generated, a new predicted word item can be determined according to the determined target candidate word item, wherein the last target candidate word item (i.e., the target candidate word item at the last position, referred to as the terminal target candidate word item) can be taken as the new predicted word item, and a new first hidden representation can be generated according to each target candidate word item except the terminal target candidate word item, the previous predicted word item, and the text to be predicted, so that the new first hidden representation can reflect the semantic information and context dependency of each known word item except the new predicted word item.

[0118] Then, the new predicted word item and the new first hidden representation are input into the acceleration network, the new candidate word item is generated through the acceleration network, the target candidate word item (i.e., the new target candidate word item) of this round is determined from the new candidate word item, and it is judged whether the new target candidate word item includes the end identifier, until the end identifier is included in the latest obtained target candidate word item. At this time, the first target text corresponding to the text to be predicted can be obtained according to all the determined target candidate word items (i.e., including the target candidate word item determined in each round) and the predicted word item.

[0119] In the embodiment of the present application, when the target candidate word item determined from the candidate word item does not include the end identifier, the new predicted word item is directly determined according to the target candidate word item, and the first hidden representation is updated dynamically in combination with the target candidate word item and the previous predicted word item to obtain the new first hidden representation. Since the new first hidden representation can reflect the semantic information and context dependency of each known word item except the new predicted word item, the generation of the candidate word item according to the new predicted word item and the new first hidden representation can guarantee the accuracy of the obtained new candidate word item. Moreover, the new predicted word item is directly determined according to the target candidate word item, and then the new candidate word item is generated through the acceleration network. Through such a cyclic process, the final first target text is obtained, without the need to infer the new predicted word item through the large language model, which can improve the efficiency of text generation.

[0120] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] Embodiment two:

[0122] Corresponding to the text generation method described in the above embodiments, FIG. 4 shows the structure block diagram of the text generation device provided by the embodiments of the present application. For the convenience of description, only the part related to the embodiments of the present application is shown.

[0123] Referring to FIG. 4, the device comprises a prediction module 41, a candidate word element generation module 42 and a first target text generation module 43. Among them,

[0124] The prediction module is configured to obtain a prediction word element corresponding to the to-be-predicted text and a first hidden representation, wherein the prediction word element comprises a new word element obtained by reasoning the to-be-predicted text, and the first hidden representation is an embedding representation obtained by feature extraction of the to-be-predicted text through a transformer layer.

[0125] The candidate word element generation module is configured to take the prediction word element and the first hidden representation as inputs of a trained acceleration network, to obtain a candidate word element output by the acceleration network, wherein the acceleration network is configured to reason a new word element based on the first hidden representation and the prediction word element, to obtain the candidate word element.

[0126] The first target text generation module is configured to determine a first target text corresponding to the to-be-predicted text based on the prediction word element and the candidate word element.

[0127] In the embodiments of the present application, the prediction word element and the first hidden representation corresponding to the to-be-predicted text are input into the trained acceleration network together. Since the prediction word element is a new word element obtained by reasoning the to-be-predicted text, and the first hidden representation is an embedding representation obtained by feature extraction of the to-be-predicted text through a transformer layer, it can reflect the semantic information and context dependency of each word element in the to-be-predicted text. Therefore, the acceleration network can better reason based on the semantic information and context dependency of the word elements of the to-be-predicted text in the process of reasoning the candidate word element based on the prediction word element, to obtain a candidate word element with high accuracy. That is, by increasing the first hidden representation as an input to reduce the dependence on the accuracy of the acceleration network. At the same time, since the above processing can generate a candidate word element with high accuracy, and the receiving rate of the candidate word element with high accuracy is high, the generation efficiency of the finally generated first target text can be better guaranteed and the text quality can be improved.

[0128] In some embodiments, the acceleration network comprises an encoding layer, an attention layer, and a prediction layer, and the candidate token generation module 42 comprises:

[0129] The extraction unit is configured to perform feature extraction processing on the predicted token by using the encoding layer to obtain a token feature representation.

[0130] The fusion unit is configured to perform attention fusion on the token feature representation according to the first hidden representation by using the attention layer to obtain a fused token feature representation.

[0131] The first processing unit is configured to perform processing on the fused token feature representation output by the attention layer by using the prediction layer to obtain the candidate token.

[0132] In some embodiments, the number of first hidden representations is M, the acceleration network comprises M attention layers connected in series, one first hidden representation corresponds to one attention layer, and M is an integer greater than 1.

[0133] Correspondingly, the candidate token generation module 42 further comprises:

[0134] The attention fusion unit is configured to perform attention fusion on the token feature representation according to the corresponding first hidden representation by using the first attention layer to obtain a fused token feature representation.

[0135] The continuous fusion unit is configured to perform attention fusion on the fused token feature representation output by the previous attention layer according to the corresponding first hidden representation for each attention layer after the first attention layer, until the fused token feature representation output by the Mth attention layer is obtained.

[0136] The second processing unit is configured to perform processing on the fused token feature representation output by the Mth attention layer by using the prediction layer to obtain the candidate token.

[0137] In some embodiments, the candidate token generation module 42 further comprises:

[0138] The execution unit is configured to repeatedly perform the following steps K-1 times to obtain K candidate tokens: taking the latest candidate token as a new predicted token, and re-executing the method as claimed in claim 2 based on the new predicted token and the first hidden representation, and K is an integer greater than 1.

[0139] Correspondingly, the first target text generation module 43 comprises:

[0140] The first target text determination unit is configured to determine the first target text corresponding to the to-be-predicted text based on the predicted word item and the K candidate word items.

[0141] In some embodiments, the number of the first hidden representations is greater than 1, and the prediction module 41 includes:

[0142] The prediction unit is configured to input the to-be-predicted text into a large language model to obtain the predicted word item and the first hidden representation output by the large language model, where the large language model is configured to perform feature extraction on the to-be-predicted text through a plurality of transformer layers connected in series, and perform reasoning based on the first hidden representation extracted by the last transformer layer to obtain the predicted word item.

[0143] In some embodiments, the text generation apparatus further includes:

[0144] The second target text generation module is configured to input the to-be-predicted text in the training data set into the large language model to obtain a second target text output by the large language model.

[0145] The training data acquisition module is configured to acquire, by the large language model, a second hidden representation corresponding to the first N word items in the second target text, and determine a target word item based on the word items other than the first N word items in the second target text, where N is less than the number of word items in the second target text.

[0146] The candidate reference word item acquisition module is configured to input the Nth word item in the second target text and the second hidden representation into an initial acceleration network to obtain a reference candidate word item output by the initial acceleration network.

[0147] The network updating module is configured to update the initial acceleration network based on the difference between the reference candidate word item and the target word item until the updated initial acceleration network meets the requirement, to obtain a trained acceleration network.

[0148] In some embodiments, the text generation apparatus further includes:

[0149] The target candidate word item determination module is configured to determine a target candidate word item from the candidate word items and judge whether the target candidate word item includes an end identifier.

[0150] The re-prediction module is configured to determine a new predicted word item and a new first hidden representation based on the target candidate word item in a case where the target candidate word item does not include the end identifier.

[0151] The new candidate word generation module is configured to take the new predicted word and the new first hidden representation as inputs of the acceleration network, and obtain a new candidate word output by the acceleration network.

[0152] The cycle module is configured to return a target candidate word determined from the candidate word based on the new candidate word, and determine whether the target candidate word includes an end identifier until the latest target candidate word includes the end identifier.

[0153] Correspondingly, the first target text generation module 43 includes:

[0154] The generation unit is configured to determine the first target text according to all the target candidate words and the predicted word.

[0155] It should be noted that the information interaction and execution process between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and the specific functions and technical effects brought by them can be referred to the method embodiments part, which will not be repeated here.

[0156] Embodiment three:

[0157] FIG. 5 is a structural schematic diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 5, the electronic device 5 according to the embodiment includes at least one processor 50 (only one processor is shown in FIG. 5), a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, wherein the processor 50 implements the steps in any of the method embodiments described above when executing the computer program 52.

[0158] The electronic device 5 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device can include, but is not limited to, the processor 50 and the memory 51. Those skilled in the art can understand that FIG. 5 is only an example of the electronic device 5, and does not limit the electronic device 5, which can include more or fewer components than those shown, or combine certain components, or different components, for example, can also include an input / output device, a network access device, and the like.

[0159] The processor 50 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0160] The memory 51 can be an internal storage unit of the electronic device 5, such as a hard disk or a memory of the electronic device 5 in some embodiments. The memory 51 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5 in other embodiments. Further, the memory 51 can include both the internal storage unit and the external storage device of the electronic device 5. The memory 51 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, etc. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for convenient distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0162] The embodiments of the present application further provide a network device, comprising at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above method embodiments when executing the computer program.

[0163] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps in any of the above method embodiments.

[0164] The embodiments of the present application provide a computer program product, which, when running on an electronic device, enables the electronic device to implement the steps in any of the above method embodiments.

[0165] The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above embodiments, which can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program, when executed by a processor, can implement the steps in each of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0166] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0167] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0168] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0169] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0170] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A text generation method characterized by, The method comprises: obtaining a prediction word and a first hidden representation corresponding to a text to be predicted, the prediction word comprising a new word inferred from the text to be predicted, and the first hidden representation being an embedding representation obtained by performing feature extraction on the text to be predicted through a transformer layer; inputting the prediction word and the first hidden representation into a trained acceleration network as inputs of the acceleration network, to obtain a candidate word output by the acceleration network, the acceleration network being configured to infer a new word based on the first hidden representation and the prediction word, and obtain the candidate word; determining a first target text corresponding to the text to be predicted based on the prediction word and the candidate word.

2. The text generation method of claim 1, wherein, The acceleration network comprises an encoding layer, an attention layer and a prediction layer, and the inputting of the prediction word and the first hidden representation into the acceleration network as inputs of the acceleration network to obtain the candidate word output by the acceleration network comprises: performing feature extraction processing on the prediction word through the encoding layer to obtain a word feature representation; performing attention fusion on the word feature representation according to the first hidden representation through the attention layer to obtain the fused word feature representation; processing the fused word feature representation output by the attention layer through the prediction layer to obtain the candidate word.

3. The text generation method of claim 2, wherein, The number of the first hidden representations is M, the acceleration network comprises M attention layers connected in series, one first hidden representation corresponds to one attention layer, and M is an integer greater than 1. The performing of the attention fusion on the word feature representation according to the first hidden representation through the attention layer to obtain the fused word feature representation comprises: performing attention fusion on the word feature representation according to the corresponding first hidden representation through a first attention layer to obtain the fused word feature representation; for each attention layer after the first attention layer, performing attention fusion on the fused word feature representation output by the previous attention layer according to the corresponding first hidden representation, until the fused word feature representation output by the Mth attention layer is obtained; The processing of the fused word feature representation output by the attention layer through the prediction layer to obtain the candidate word comprises: processing the fused word feature representation output by the Mth attention layer through the prediction layer to obtain the candidate word.

4. The text generation method of claim 2, wherein, After the processing of the fused word feature representation output by the attention layer through the prediction layer to obtain the candidate word, the method further comprises: repeating the following steps K-1 times to obtain K candidate words: taking the latest candidate word as a new prediction word, and re-executing the method according to claim 2 based on the new prediction word and the first hidden representation, K being an integer greater than 1; correspondingly, the determining of the first target text corresponding to the text to be predicted based on the prediction word and the candidate word comprises: determine the first target text corresponding to the to-be-predicted text based on the predicted word item and the K candidate word items.

5. The text generation method of claim 1, wherein, The number of the first hidden representations is greater than 1, and the obtaining of the predicted word item and the first hidden representation corresponding to the to-be-predicted text comprises: taking the to-be-predicted text as input of a large language model to obtain the predicted word item and the first hidden representation output by the large language model, the large language model being configured to perform feature extraction processing on the to-be-predicted text through a plurality of transformer layers connected in series, and perform reasoning based on the first hidden representation extracted by the last transformer layer to obtain the predicted word item.

6. The text generation method of claim 5, wherein, Before the taking of the predicted word item and the first hidden representation as input of the acceleration network to obtain the candidate word item output by the acceleration network, the method further comprises: taking to-be-predicted text in a training data set as input of the large language model to obtain a second target text output by the large language model; obtaining, by the large language model, a second hidden representation corresponding to the first N word items in the second target text, and determining a target word item based on word items other than the first N word items in the second target text, N being less than the number of word items in the second target text; inputting the Nth word item in the second target text and the second hidden representation into an initial acceleration network to obtain a reference candidate word item output by the initial acceleration network; updating the initial acceleration network based on the difference between the reference candidate word item and the target word item until the updated initial acceleration network meets the requirement to obtain the trained acceleration network. Before the determining of the first target text corresponding to the to-be-predicted text based on the predicted word item and the candidate word item, the method further comprises:

7. The text generation method of any one of claims 1 to 6, wherein, determining a target candidate word item from the candidate word items and judging whether the target candidate word item includes an end identifier; in a case where the target candidate word item does not include the end identifier, determining a new predicted word item and a new first hidden representation based on the target candidate word item; taking the new predicted word item and the new first hidden representation as input of the acceleration network to obtain a new candidate word item output by the acceleration network; returning to the step of determining a target candidate word item from the candidate word items and judging whether the target candidate word item includes an end identifier based on the new candidate word item until the latest target candidate word item includes the end identifier; correspondingly, the determining of the first target text corresponding to the to-be-predicted text based on the predicted word item and the candidate word item comprises: determining the first target text based on all the target candidate word items and the predicted word item. comprise:

8. A text generating apparatus characterized by comprising: a prediction module configured to obtain a predicted word item and a first hidden representation corresponding to to-be-predicted text, the predicted word item comprising a new word item obtained by reasoning on the to-be-predicted text, and the first hidden representation being an embedding representation obtained by performing feature extraction on the to-be-predicted text through a transformer layer; ​ The candidate word piece generation module is configured to take the predicted word piece and the first hidden representation as inputs of a trained acceleration network, to obtain a candidate word piece output by the acceleration network, and the acceleration network is configured to infer a new word piece based on the first hidden representation and the predicted word piece, to obtain the candidate word piece. The first target text generation module is configured to determine a first target text corresponding to the to-be-predicted text based on the predicted word piece and the candidate word piece.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images