Text generation method and device based on large model, electronic equipment and storage medium

Abstract

The invention provides a text generation method and device based on a large model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence such as large models and natural language processing. According to the specific implementation scheme, a matched prefix is obtained; the matching prefix comprises at least one continuous lexical element; based on the matching prefix, according to a pre-configured draft lemma sequence length, obtaining a draft lemma sequence; the draft lexical element sequence comprises at least one lexical element; performing validity verification on the draft lexical element sequence based on a speculation decoding algorithm by adopting a pre-trained large model; and in response to verification passing, taking the draft lexical element sequence as a generated text. According to the technology disclosed by the invention, the text generation efficiency of the large model can be effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the field of artificial intelligence technology such as big models and natural language processing, and more particularly to a text generation method, device, electronic device and storage medium based on a big model. Background Art

[0002] A large model is a deep learning model with a large parameter scale and a very complex structure. It usually refers to a neural network model with millions to billions of parameters.

[0003] In the existing technology, large models have achieved remarkable results in a series of downstream tasks. They provide many convenient services and help for human life through real-time human-computer interaction. Summary of the invention

[0004] The present invention provides a large model-based text generation method, device, electronic device and storage medium.

[0005] According to one aspect of the present disclosure, a text generation method based on a large model is provided, comprising:

[0006] Obtain a matching prefix; the matching prefix includes at least one continuous word element;

[0007] Based on the matching prefix, a draft word unit sequence is obtained according to a pre-configured draft word unit sequence length; the draft word unit sequence includes at least one word unit;

[0008] Using a pre-trained large model, based on a speculative decoding algorithm, the validity of the draft word sequence is verified;

[0009] In response to the verification being successful, the draft word sequence is used as the generated text.

[0010] According to another aspect of the present disclosure, a text generation device based on a large model is provided, comprising:

[0011] A prefix acquisition module, used to acquire a matching prefix; the matching prefix includes at least one continuous word element;

[0012] A sequence acquisition module, configured to acquire a draft word unit sequence based on the matching prefix and according to a pre-configured draft word unit sequence length; the draft word unit sequence includes at least one word unit;

[0013] A verification module, for verifying the validity of the draft word sequence using a pre-trained large model based on a speculative decoding algorithm;

[0014] The generation module is used for, in response to passing the verification, using the draft word sequence as the generated text.

[0015] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0019] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0020] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and any possible implementation method.

[0021] According to the technology disclosed in the present invention, the text generation efficiency of large models can be effectively improved.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0025] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0027] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0028] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0029] Figure 6 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] Obviously, the described embodiments are only part of the embodiments of the present disclosure, but not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure.

[0032] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0033] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0034] At present, the high inference latency of large models has become the main reason hindering their wider application. The increase in inference latency of large models is not only due to their huge number of parameters and high computing cost, but also mainly due to the generation method of their autoregressive decoding. Autoregressive decoding needs to be generated word by word, which causes the inference latency to increase with the length of the generated sequence.

[0035] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure; Figure 1 As shown, this embodiment provides a text generation method based on a large model, which may specifically include the following steps:

[0036] S101, obtaining a matching prefix; the matching prefix includes at least one continuous word element;

[0037] S102, based on the matching prefix, according to the pre-configured draft word unit sequence length, obtaining a draft word unit sequence; the draft word unit sequence includes at least one word unit;

[0038] The text generation method based on a big model of this embodiment is applied to a scenario in which text is generated based on a big model. The executor of the method is a text generation device based on a big model. The device may be an electronic entity, or may be an application or intelligent agent implemented by software, etc., which can realize text generation based on a big model.

[0039] The matching prefix of this embodiment is the basis for obtaining the draft word-gram sequence. The draft word-gram sequence includes at least one word-gram, and if it includes more than two word-grams, the sequence itself also limits the order of the two or more word-grams. The matching prefix can also be considered as a word-gram sequence, including at least one continuous word-gram. The word-gram of this embodiment can be understood as a word unit. The word-gram unit of this embodiment can be a character granularity or a word granularity, which is set according to the scenario or demand and is not limited here.

[0040] In this embodiment, the pre-configured draft word-gram sequence length is used to limit the number of words included in the draft word-gram sequence.

[0041] In this embodiment, the draft word sequence is based on prefix matching instead of model generation, which can improve the efficiency of obtaining the draft word sequence.

[0042] S103, using a pre-trained large model and a speculative decoding algorithm to verify the validity of the draft word sequence;

[0043] S104. In response to the verification being passed, the draft word sequence is used as the generated text.

[0044] Since the draft word sequence can be based on prefix matching and truncated according to the pre-configured draft word sequence length. Therefore, the draft word sequence cannot be directly used as the generated text, and a pre-trained large model is required to verify the validity of the draft word sequence. Therefore, in this embodiment, since the large model does not need to use autoregressive decoding to generate each word in the draft word sequence, it is only necessary to verify the validity of each word in the draft word sequence.

[0045] If the draft word-unit sequence includes more than two words, the validity of more than two words can be verified in parallel during the verification process using the speculative decoding algorithm, which can effectively improve efficiency.

[0046] In addition, optionally, in actual application scenarios, if the draft word sequence fails verification, the draft word sequence cannot be used as the text generated by the large model.

[0047] The text generation method based on the large model of this embodiment can effectively improve the efficiency of obtaining the draft word unit sequence by obtaining the matching prefix and obtaining the draft word unit sequence according to the pre-configured draft word unit sequence length; further, the large model is used to verify the validity of the draft word unit sequence based on the speculative decoding algorithm. After verification, the draft word unit sequence can be used as the generated text. In this technical solution, the large model does not need to use the autoregressive decoding method to generate text, but only needs to verify the validity of the matched draft word unit sequence. Compared with the use of autoregressive decoding to generate text, it can effectively shorten the time consumption of text generation and improve the efficiency of text generation.

[0048] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure; the text generation method based on the large model of this embodiment, in the above Figure 1 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 2 As shown, the text generation method based on the large model of this embodiment may specifically include the following steps:

[0049] S201, obtaining the last preset number of tokens from the generated text or input prompt information as a matching prefix;

[0050] S202, based on the matching prefix, obtaining a draft word unit sequence from at least one of a reference, a generated text, and input prompt information according to a pre-configured draft word unit sequence length;

[0051] In text generation scenarios such as text summarization, multi-round dialogue, and retrieval enhancement, the generation results and input information of the large model will have a high degree of phrase repetition. In this embodiment, this information repetition can be used to obtain a draft word sequence.

[0052] Specifically, since the large model does not generate text in one step, it can be continuously generated through multiple iterations. If the large model generates text for the first time, the input information of the model only includes the input prompt information, that is, prompt. At this time, the last preset number of words can be obtained from the input prompt information as a matching prefix. If the model does not generate text for the first time, the input information of the model can include the input prompt information and the generated text. Specifically, the input prompt information and the generated text can be spliced ​​and input into the large model as input information. At this time, the last preset number of words can be obtained from the generated text as a matching prefix.

[0053] In this embodiment, the method of obtaining the matching prefix from the generated text or input prompt information is very accurate and efficient.

[0054] Then, the same matching prefix is ​​searched from the previous text. After the match is successful, the fragment of the draft word sequence length after the matching prefix is ​​taken as the draft word sequence. The previous text of this embodiment may also be referred to as the above text, which may include the generated text or the input prompt information. If the input information also includes references, the previous text may also include the references. Based on this, the draft word sequence can be obtained from the references, the generated text or the input prompt information according to the pre-configured draft word sequence length based on the matching prefix.

[0055] In this embodiment, the draft word sequence can be obtained from reference documents, generated texts or input prompt information, which provides effective support for the accurate acquisition of the draft word sequence.

[0056] In order to improve the matching success rate and the effectiveness of the draft word sequence, this embodiment proposes a priority matching algorithm. The draft word sequence can be preferentially searched for prefix matching in the generated text. When the search in the generated text fails, the match is further searched in the input prompt information. In the retrieval enhancement scenario, the input prompt information will also be accompanied by the content of the reference, and at this time, the match is searched in the reference first. In other words, the pre-configured priority strategy can be reference>generated text>input prompt information, that is, the reference has the highest priority and the input prompt information has the lowest priority. This priority strategy not only guarantees the matching success rate, but also improves the quality of the draft word sequence, and can potentially increase the subsequent verification pass rate.

[0057] S203, for each draft word in the draft word sequence, using a large model to predict multiple candidate words at the position of the draft word and the probability of each candidate word;

[0058] S204, based on multiple candidate word-grams and the probability of each candidate word-gram, based on a speculative decoding algorithm, verify the validity of the draft word-gram;

[0059] For each draft word in the draft word sequence, the large model can perform verification of step S203 and step S204 based on the speculative decoding algorithm. Specifically, verification can be performed on multiple draft words in parallel. That is, multiple candidate words at the position of each draft word and the probability of each candidate word are predicted in parallel, and validity verification is performed on multiple draft words in parallel based on the predicted information.

[0060] Specifically, when predicting the candidate word and probability of each draft word position, the prediction can be made based on the previous text of the draft word. For example, the previous text may include input information and the draft word before the draft word in the draft word sequence. If the draft word is the first draft word in the draft word sequence, the corresponding previous text may include input prompt information and input information spliced ​​with generated text. If the text is generated for the first time and has not yet been generated, the previous text may only include input prompt information.

[0061] In this embodiment, based on multiple candidate word units and the probability of each candidate word unit, the validity verification of the draft word unit can be implemented by using a speculative decoding algorithm. Specifically, when the step S204 is implemented, it can include the following steps:

[0062] (1) sorting multiple candidate word-grams based on the probability of each candidate word-gram to obtain a first sorting;

[0063] (2) Based on the first sorting and the speculative decoding algorithm, the validity of the draft word is verified.

[0064] For example, the specific implementation of step (2) may include two methods:

[0065] The first implementation method can be referred to as the topN verification method.

[0066] Specifically, it is detected whether the draft word is in the top N with the highest probability in the first sorting, where N is a positive integer, which can be 1, 2, or other values ​​according to the requirements, and is not limited here. If so, it is determined that the draft word is valid, that is, the draft word passes the verification.

[0067] In this embodiment, when verifying each draft word in the draft word sequence, in order to ensure that the draft word sequence is the same as the word obtained by autoregressive decoding of the large model, when the topN verification method is used for verification, N can be 1, that is, top1 verification is used, that is, only when the draft word is the same as the maximum probability word generated by the large model, the verification is passed and the draft word is adopted. Although this strict verification strategy ensures the consistency of the speculative decoding generation result and the autoregressive decoding, it also brings a lot of resource waste.

[0068] For example, in a search enhancement scenario, suppose the original large model is verifying six draft words in parallel. These six draft words are searched from the reference through prefix matching. Only one of them fails to be verified. Although the subsequent draft words in the draft word sequence can pass the verification, they are discarded because the previous draft word fails to be verified. This example is common under strict verification strategies. In order to avoid the waste of draft words in this situation, the topN verification method can be used. That is, if the draft word is ranked in the top N of the multiple candidates predicted by the large model for this position, it will also pass the verification.

[0069] This method can effectively improve the verification pass rate of draft words, thereby improving the acceleration effect of speculative decoding, and thus effectively improving the efficiency of text generation.

[0070] The second implementation method can be referred to as the topP verification method.

[0071] Specifically, based on the first sorting, the cumulative sum of the probability of the draft word and the probability of the candidate word with a greater probability than the draft word can be obtained; for example, if the first sorting is arranged in descending order of probability, the target sorting identifier of the draft word in the first sorting can be determined first; then the cumulative sum of the probabilities of all candidate word units corresponding to the target sorting identifier before (including the target sorting identifier) ​​in the first sorting can be calculated. Then, it is detected whether the cumulative sum reaches a preset probability value, such as p; if so, it is determined that the draft word is valid, that is, the draft word has passed the verification. P in this embodiment can be set according to actual needs, for example, it can be 0.95, 0.9 or 0.85, etc., which is not limited here.

[0072] This approach can also effectively improve the verification pass rate of draft words, thereby improving the acceleration effect of speculative decoding, and thus effectively improving the efficiency of text generation.

[0073] S205, output verification results;

[0074] For example, the verification result may include a draft word that has passed verification.

[0075] Specifically, according to the verification in step S203 and step S204, the draft word that has passed the verification is determined, the corresponding verification result is obtained, and outputted.

[0076] S206: Based on the verification result, adjust the length of the pre-configured draft word sequence.

[0077] Considering the verification pass rate of speculative decoding, it is also affected by the length of the draft word sequence for each verification, that is, the number of draft words included in the draft word sequence. When the number of draft words is too small, the number of draft words that can be adopted each time is limited; when the number of draft words is too large, the time cost of parallel decoding of large models will increase. Both of these affect the actual acceleration benefits of speculative decoding.

[0078] Based on the above considerations, in this embodiment, the length of the draft word sequence can be dynamically adjusted, which can specifically include the following adjustment methods:

[0079] (a) If all draft words in the draft word sequence pass verification, the preconfigured length of the draft word sequence is lengthened; for example, 1, 2 or other values ​​may be added to the length of the existing preconfigured draft word sequence, which is not limited here.

[0080] (b) If some draft words in the draft word sequence pass the verification, the length of the preconfigured draft word sequence is shortened. For example, the length of the existing preconfigured draft word sequence can be subtracted by 1, 2 or other values, which are not limited here.

[0081] Through the above method, the length of the draft word sequence can be dynamically adjusted according to the verification results of each time, thereby effectively improving the efficiency of each verification, reducing the time consumption of parallel decoding of large models, and effectively improving the efficiency of text generation.

[0082] In this embodiment, when the verification of the draft word unit sequence fails, that is, not all draft word units in the draft word unit sequence are verified, at this time, the autoregressive decoding method of the large model is used for decoding to generate text to ensure the realization of the text generation function.

[0083] The large model-based text generation method of this embodiment, by using a search and matching strategy, can effectively shorten the time consumption of obtaining the draft word unit sequence, improve the efficiency of obtaining the draft word unit sequence, and reduce the cost of using speculative decoding; at the same time, based on the topN or topP verification method, it can effectively improve the verification pass rate of the draft word units, and ultimately improve the acceleration effect of speculative decoding, thereby improving the text generation efficiency of the large model.

[0084] Figure 3 is a schematic diagram according to the third embodiment of the present disclosure; Figure 2 On the basis of the embodiment shown, the verification methods of topN and topP are combined to verify the draft word sequence, which may specifically include the following steps:

[0085] S301, obtaining the i-th draft word in the draft word sequence; executing step S302;

[0086] In the specific verification process, the i-th draft word is taken in order from front to back. The minimum value of i is 1, and the maximum value is the length of the draft word sequence, for example, in this embodiment, it can be recorded as K. That is, in the initial verification, i=1 is taken first.

[0087] S302, perform TopP verification on the i-th draft word; if the verification fails, execute step S303; if the verification passes, execute step S304;

[0088] For details, please refer to the above Figure 2 The implementation method of the TopP verification recorded in the illustrated embodiment. The TopP verification may refer to the verification being passed if the cumulative probability of the draft word element in the word element set decoded by the large model exceeds the p value.

[0089] S303, perform TopN verification on the i-th draft word; if the verification passes, execute step S304; if the verification fails, execute step 305;

[0090] S304, record that the draft word element verification has passed; execute step S305;

[0091] For details, please refer to the above Figure 2 The implementation method of the TopN verification recorded in the illustrated embodiment. The TopN verification may refer to the verification being passed if the draft word appears in the word set with the top N probability values ​​of the large model decoding.

[0092] S305, check whether the current i is equal to the length K of the draft word sequence; if not, execute step S306; if equal, execute step S307;

[0093] S306, add 1 to i; return to step S301 to continue verification;

[0094] S307. Output all draft word units that have passed verification, and end.

[0095] In the output result of this embodiment, it can be further indicated whether all draft word sequences have passed the verification.

[0096] The verification of the draft word in this embodiment can combine the two verification methods to realize the verification of the draft word in the draft word sequence based on the soft matching window, which can effectively improve the verification pass rate of the draft word and improve the acceleration effect of speculative decoding, thereby effectively improving the efficiency of text generation by large models.

[0097] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure; Figure 4 As shown, this embodiment provides a text generation device 400 based on a large model, including:

[0098] The prefix acquisition module 401 is used to acquire a matching prefix; the matching prefix includes at least one continuous word element;

[0099] A sequence acquisition module 402 is used to acquire a draft word unit sequence based on the matching prefix and according to a pre-configured draft word unit sequence length; the draft word unit sequence includes at least one word unit;

[0100] Verification module 403, used to verify the validity of the draft word sequence based on a speculative decoding algorithm using a pre-trained large model;

[0101] The generation module 404 is used for, in response to the verification being passed, using the draft word sequence as the generated text.

[0102] The large model-based text generation device 400 of this embodiment implements the implementation principle and technical effect of large model-based text generation by adopting the above-mentioned modules, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0103] Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure; Figure 5 As shown, the text generation device 500 based on the large model of this embodiment, in the above Figure 4 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 5 As shown, the text generation device 500 based on the large model of this embodiment includes the above Figure 4 The modules with the same name and function are as follows: a prefix acquisition module 501 , a sequence acquisition module 502 , a verification module 503 and a generation module 504 .

[0104] In this embodiment, the prefix acquisition module 501 is used to:

[0105] The last preset number of word units are obtained from the generated text or input prompt information as the matching prefix.

[0106] Further optionally, in one embodiment of the present disclosure, the sequence acquisition module 502 is used to:

[0107] Based on the matching prefix, a draft word sequence is obtained from references, generated text or input prompt information according to a preconfigured draft word sequence length.

[0108] Further optionally, in one embodiment of the present disclosure, the sequence acquisition module 502 is used to:

[0109] Based on the matching prefix, according to a preconfigured priority strategy, the draft word sequence is obtained from references, generated text or input prompt information based on a preconfigured draft word sequence length.

[0110] Further optionally, if Figure 5 As shown, in one embodiment of the present disclosure, the verification module 503 includes:

[0111] The prediction unit 5031 is used to predict, for each draft word in the draft word sequence, a plurality of candidate words at the position of the draft word and the probability of each candidate word by using the large model;

[0112] The verification unit 5032 is used to verify the validity of the draft word-gram based on the multiple candidate word-grams and the probability of each candidate word-gram and based on a speculative decoding algorithm.

[0113] Further optionally, in one embodiment of the present disclosure, the verification unit 5032 is used to:

[0114] Sorting the plurality of candidate word-grams based on the probability of each candidate word-gram to obtain a first sorting;

[0115] Based on the first sorting and a speculative decoding algorithm, the validity of the draft word is verified.

[0116] Further optionally, in one embodiment of the present disclosure, the verification unit 5032 is used to:

[0117] Detect whether the draft word is located in the top N words with the highest probability in the first sorting, wherein N is a positive integer;

[0118] If so, it is determined that the draft word is valid.

[0119] Further optionally, in one embodiment of the present disclosure, the verification unit 5032 is used to:

[0120] Based on the first ranking, obtaining a cumulative sum of the probability of the draft word-gram and the probabilities of candidate word-grams greater than the probability of the draft word-gram;

[0121] Detecting whether the cumulative sum reaches a preset probability value;

[0122] If so, it is determined that the draft word is valid.

[0123] Further optionally, if Figure 5 As shown, in one embodiment of the present disclosure, the text generation device 500 based on the large model further includes:

[0124] The adjustment module 505 is used to adjust the length of the pre-configured draft word sequence based on the verification result.

[0125] Further optionally, in one embodiment of the present disclosure, the adjustment module 505 is used to:

[0126] If all draft word units in the draft word unit sequence pass the verification, lengthening the pre-configured draft word unit sequence length;

[0127] If some draft word units in the draft word unit sequence pass the verification, the length of the preconfigured draft word unit sequence is shortened.

[0128] The large model-based text generation device 500 of this embodiment, by adopting the above-mentioned modules, realizes the implementation principle and technical effect of large model-based text generation, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0129] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0130] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0131] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0132] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0133] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0134] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the above-mentioned methods of the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the above-mentioned methods of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0135] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0137] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0140] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0142] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A text generation method based on a large model, comprising: Get the matching prefix; The matching prefix includes at least one continuous word element; Based on the matching prefix, a draft word unit sequence is obtained according to a pre-configured draft word unit sequence length; the draft word unit sequence includes at least one word unit; Using a pre-trained large model, based on a speculative decoding algorithm, the validity of the draft word sequence is verified; In response to the verification being successful, the draft word sequence is used as the generated text.

2. The method according to claim 1, wherein: Get matching prefixes, including: The last preset number of word units are obtained from the generated text or input prompt information as the matching prefix.

3. The method according to claim 1, wherein: Based on the matching prefix, according to the pre-configured draft word unit sequence length, a draft word unit sequence is obtained, including: Based on the matching prefix, a draft word sequence is obtained from references, generated text or input prompt information according to a preconfigured draft word sequence length.

4. The method according to claim 3, wherein: Based on the matching prefix, a draft word sequence is obtained from a reference, a generated text, or an input prompt information according to a preconfigured draft word sequence length, including: Based on the matching prefix, according to a preconfigured priority strategy, the draft word sequence is obtained from references, generated text or input prompt information based on a preconfigured draft word sequence length.

5. The method according to claim 1, wherein: Using a pre-trained large model and based on a speculative decoding algorithm, the draft word sequence is validated, including: For each draft word in the draft word sequence, using the large model, predicting multiple candidate words at the position of the draft word and the probability of each candidate word; Based on the multiple candidate word-grams and the probability of each candidate word-gram, and based on a speculative decoding algorithm, the validity of the draft word-gram is verified.

6. The method according to claim 5, wherein: Based on the multiple candidate word-grams and the probability of each candidate word-gram, and based on a speculative decoding algorithm, the validity of the draft word-gram is verified, including: Sorting the plurality of candidate word-grams based on the probability of each candidate word-gram to obtain a first sorting; Based on the first sorting and a speculative decoding algorithm, the validity of the draft word is verified.

7. The method according to claim 6, wherein: Based on the first sorting and the speculative decoding algorithm, the validity of the draft word is verified, including: Detect whether the draft word is located in the top N words with the highest probability in the first sorting, wherein N is a positive integer; If so, it is determined that the draft word is valid.

8. The method according to claim 6, wherein: Based on the first sorting and the speculative decoding algorithm, the validity of the draft word is verified, including: Based on the first ranking, obtaining a cumulative sum of the probability of the draft word-gram and the probabilities of candidate word-grams greater than the probability of the draft word-gram; Detecting whether the cumulative sum reaches a preset probability value; If so, it is determined that the draft word is valid.

9. The method according to claim 6, wherein: After verifying the validity of the draft word sequence using a pre-trained large model based on a speculative decoding algorithm, the method further includes: Based on the verification results, the pre-configured draft word sequence length is adjusted.

10. The method according to claim 9, wherein: Based on the verification results, the pre-configured draft word sequence length is adjusted, including: If all draft word units in the draft word unit sequence pass the verification, lengthening the pre-configured draft word unit sequence length; If some draft word units in the draft word unit sequence pass the verification, the length of the preconfigured draft word unit sequence is shortened.

11. A text generation device based on a large model, comprising: Prefix acquisition module, used to obtain matching prefixes; The matching prefix includes at least one continuous word element; A sequence acquisition module, configured to acquire a draft word unit sequence based on the matching prefix and according to a pre-configured draft word unit sequence length; the draft word unit sequence includes at least one word unit; A verification module, for verifying the validity of the draft word sequence using a pre-trained large model based on a speculative decoding algorithm; The generation module is used for, in response to passing the verification, using the draft word sequence as the generated text.

12. The device according to claim 11, wherein The prefix acquisition module is used to: The last preset number of word units are obtained from the generated text or input prompt information as the matching prefix.

13. The device according to claim 11, wherein: The sequence acquisition module is used to: Based on the matching prefix, a draft word sequence is obtained from references, generated text or input prompt information according to a preconfigured draft word sequence length.

14. The device according to claim 13, wherein: The sequence acquisition module is used to: Based on the matching prefix, according to a preconfigured priority strategy, the draft word sequence is obtained from references, generated text or input prompt information based on a preconfigured draft word sequence length.

15. The device according to claim 11, wherein The verification module comprises: A prediction unit, configured to predict, for each draft word in the draft word sequence, a plurality of candidate words at the position of the draft word and a probability of each candidate word by using the large model; The verification unit is used to verify the validity of the draft word-gram based on the multiple candidate word-grams and the probability of each candidate word-gram and based on a speculative decoding algorithm.

16. The device according to claim 15, wherein: The verification unit is used to: Sorting the plurality of candidate word-grams based on the probability of each candidate word-gram to obtain a first sorting; Based on the first sorting and a speculative decoding algorithm, the validity of the draft word is verified.

17. The device according to claim 16, wherein: The verification unit is used to: Detect whether the draft word is located in the top N words with the highest probability in the first sorting, wherein N is a positive integer; If so, it is determined that the draft word is valid.

18. The device according to claim 16, wherein: The verification unit is used to: Based on the first ranking, obtaining a cumulative sum of the probability of the draft word-gram and the probabilities of candidate word-grams greater than the probability of the draft word-gram; Detecting whether the cumulative sum reaches a preset probability value; If so, it is determined that the draft word is valid.

19. The device according to claim 6, wherein: The device also includes: The adjustment module is used to adjust the length of the pre-configured draft word sequence based on the verification result.

20. The device according to claim 19, wherein The adjustment module is used for: If all draft word units in the draft word unit sequence pass the verification, lengthening the pre-configured draft word unit sequence length; If some draft word units in the draft word unit sequence pass the verification, the length of the preconfigured draft word unit sequence is shortened.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images