Multi-token generation in autoregressive generative artificial intelligence models

Abstract

Techniques and apparatus for generating responses to input prompts using a generative artificial intelligence model. Example methods generally include: generating a lexicalized input comprising multiple input lexical units having a length corresponding to a maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the generated output includes at least two lexical units; generating an output comprising multiple output lexical units based on the lexicalized input and the generative artificial intelligence model, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; identifying a set of valid lexical units from the output based on a comparison of the multiple input lexical units and the multiple output lexical units, and outputting the set of valid lexical units as a response to the input prompt.

Description

Background Technology

[0001] Various aspects of this disclosure relate to generative artificial intelligence models, and more specifically to generating multiple lexical units in each round of inference in response to an input query fed into the generative artificial intelligence model.

[0002] Generative AI models can be used in a variety of environments to generate responses to input prompts (also known as queries or inputs). For example, generative AI models can be used in chatbot applications that use large language models (LLMs) to generate answers or at least responses to input prompts. Other examples where generative AI models can be used include latent diffusion models, in which the model generates an image from an input text description of the content of the desired image; decision transformers, in which future actions are predicted based on a sequence of previous actions within a given environment; and so on.

[0003] Typically, generating responses to queries using generative AI models can be computationally expensive. For example, in a chatbot deployment where a large language model is used to generate responses to queries formatted as text, the response can be generated by passing through the large language model for each lexical unit (e.g., a word or part of a word) generated as part of the response. The output of each pass can be a probability distribution over a set of lexical units (e.g., words or parts of words), from which the next lexical unit (e.g., a word or part of a word) can be selected, for example, by sampling or based on maximum likelihood. Because passes through the large language model are used to generate each word (or (multiple) lexical units) in the response to the query, the computational overhead can be modeled as the product of the number of words included in the response and the computational resource overhead of performing the passes through the large language model (e.g., in terms of processing power, memory bandwidth, and / or other computational resources used), which typically increases with the number of parameters within the large language model. Summary of the Invention

[0004] Certain aspects of this disclosure provide a method for generating responses to input prompts using a generative artificial intelligence model. The method generally includes: generating a lexicalized input comprising multiple input lexical units having a length corresponding to a maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the generated output includes at least two lexical units; generating an output comprising multiple output lexical units based on the lexicalized input and the generative artificial intelligence model, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; identifying a set of valid lexical units from the output based on a comparison of the multiple input lexical units and the multiple output lexical units, and outputting the set of valid lexical units as a response to the input prompt.

[0005] Other aspects include: processing systems configured to perform the foregoing methods and those methods described herein; a non-transitory computer-readable medium including instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the foregoing methods and those methods described herein; a computer program product embodied on a computer-readable storage medium including code for performing the foregoing methods and those methods further described herein; and a processing system including components for performing the foregoing methods and those methods further described herein.

[0006] The following description and related figures illustrate certain exemplary features of one or more aspects. Attached Figure Description

[0007] The accompanying drawings illustrate only certain aspects of this disclosure and are therefore not intended to limit the scope of this disclosure.

[0008] Figure 1 An example is shown where a generative artificial intelligence model sequentially generates individual lexical units in response to an input query.

[0009] Figure 2 Examples are illustrated of using generative artificial intelligence models to generate multiple output lexical units in each round of inference in response to an input query, according to various aspects of this disclosure.

[0010] Figure 3 Example operations are illustrated according to various aspects of this disclosure for generating responses to input prompts using a generative artificial intelligence model that generates multiple output lexical units in each round of inference.

[0011] Figure 4 Example processing systems configured to perform various aspects of this disclosure are described.

[0012] For ease of understanding, the same reference numerals are used where possible to designate common elements shared by all figures. Elements and features conceived in one aspect can be beneficially incorporated into other aspects without further elaboration. Detailed Implementation

[0013] This disclosure provides apparatus, methods, processing systems, and computer-readable media for efficiently generating responses to input queries using generative artificial intelligence models.

[0014] Typically, generative AI models generate responses to queries input into the model. For example, a large language model (LLM) deployed within a chatbot can generate a response to a query using multiple passes through the LLM, where each successive pass is based on the query (which can be lexicalized for processing) and uses lexical units (e.g., words or parts of words) generated through previous passes through the LLM. These LLMs can typically include a large number (e.g., billions or even trillions) of weights or parameters within the model. Due to the size of these models and the operations performed on each lexical unit to predict what the next lexical unit should be in response to the query and previously generated lexical units, deploying large language models on a variety of devices with limited memory and / or processing power relative to the cloud computing instances on which the large language models typically operate. Furthermore, in some cases, the memory bandwidth involved in generating responses to queries provided as input to the model may prevent computational resources from being used for other tasks.

[0015] Generative AI models typically generate responses to input queries sequentially. That is, for a given input cue or query, a generative AI model can generate a response using multiple rounds of inference, where each round generates a single new lexical based on the input cue and previously generated lexicals (if any) for the input cue. Because each round of inference performed by the generative AI model incurs processing overhead, generating a response to an input query can be a computationally expensive process using processing cycles, memory, power, and other resources. For generative AI models deployed, for example, in cloud computing instances or other computing systems where significant computing resources are available, the computational overhead of generating responses to input cuees may not be a limiting factor. However, when generative AI models are deployed on edge devices in a network (e.g., smartphones, tablets, desktop computers, laptops, etc.), the inefficiency of sequentially generating lexicals to be included in the response to the input cue can utilize the significant computing resources on these devices and may consume substantial power, potentially reducing the battery life of battery-powered edge devices.

[0016] Certain aspects of this disclosure provide techniques and apparatus for generating responses to queries input into a generative artificial intelligence model (such as a large language model) configured to generate multiple output lexical units in each round of inference. For this purpose, the generative artificial intelligence model can be configured to generate a defined number of lexical units during each round of inference. Input prompts can be lexicalized and padded to generate a first set of lexical units comprising the defined number of lexical units. The first set of lexical units and the input prompts can be input into the generative artificial intelligence model to generate a second set of lexical units, and the first and second sets of lexical units can be compared. If one or more lexical units in the first and second sets match, the matching lexical unit can be output as a response to the input prompts, and in some cases, the second set of lexical units can be shifted and / or padded. By using per-round inference multi-term generation techniques to generate responses to input queries using generative artificial intelligence models, aspects of this disclosure can allow for variable (e.g., increased) term generation rates, which in turn can allow for faster completion of inference tasks performed by generative artificial intelligence models, reduced power consumption, and minimize or at least reduce the likelihood that the device will enter a lower-performance operating scenario during inference operations using generative artificial intelligence models (or other machine learning models that support variable-sized outputs).

[0017] Lexical generation in generative artificial intelligence models

[0018] Typically, autoregressive lexical generation (e.g., in large language models) takes historical lexical units as input to generate output. That is, autoregressive lexical generation can be represented by the following expression:

[0019] in Indicates time The generated lexical sequence has pairs of lexical units. to The conditional probability of choosing as a condition. ,and Indicates time The generated lexical sequence has pairs of lexical units. to The conditional probability of choosing as a condition. Typically, each execution of an autoregressive model generates a single lexical unit, meaning that execution is possible. N An inference to generate N A sequence of 100 lexical units.

[0020] Figure 1 Example 100 illustrates how a generative artificial intelligence model sequentially generates individual lexical units in response to an input query.

[0021] In Example 100, input prompt 110 can be fed into a generative artificial intelligence model for processing. To allow processing of input prompt 110, lexer 112 can transform input prompt 110 (which can be a natural language string or other word sequence) into lexicalized input prompt 114. Generally, lexicalized input prompt 114 can be an array, vector, or other data structure representing input prompt 110, where each element of the data structure corresponds to a portion of input prompt 110. Generally, each portion of input prompt 110 can be called an input lexical and can represent a word or a portion of a word in input prompt 110. Furthermore, as illustrated, each lexical can be stored as an index in a lexical dictionary, thereby allowing lexicalized input prompt 114 to be represented as a sequence of lexical indexes (e.g., ...). Figure 1 Examples include indices 273, 1026, 6062, etc.

[0022] Lexicalized input hint 114 can be fed into a prefix model 116, which generates a first output lexical 120 and feeds the lexicalized input hint 114 into a key-value (KV) cache 118. Generally, the prefix model 116 feeds the lexicalized input hint 114 into the KV cache 118 to allow at least the lexicalized input hint 114 to be prepended to one or more autoregressive generated lexicals (discussed in further detail below) to conditionally generate subsequent lexicals, or at least to provide context for their generation, which are to be included in the response to the input hint 110.

[0023] Subsequently, at block 122, the lexicalized input cue 114 stored in the KV cache 118 and the first output lexical 120 can be input into the autoregressive layer (labeled "KV AR1") of the generative artificial intelligence model to perform a first inference. The autoregressive layer can be configured to output a single lexical in response to the input of the lexicalized input cue 114 and one or more previously generated lexicals. As illustrated, the first inference performed at block 122 can result in the generation of a second output lexical 124 with lexical index 457.

[0024] Because the second output lexical unit 124 does not correspond to a termination lexical unit, and because the number of lexical units generated using the generative AI model has not yet reached the threshold number of lexical units, the generative AI model (and more specifically, its autoregressive layer) can continue to generate a second inference based on the second output lexical unit 124 at block 126. This second inference at block 126 can lead to the generation of a third output lexical unit 128 with index 863. Again, because the third output lexical unit 128 does not correspond to a termination lexical unit, and because the number of lexical units generated using the generative AI model has not yet reached the threshold number of lexical units, the generative AI model can continue to generate a third inference at block 130. The sequential generation of output lexical units by the generative AI model can continue until a termination condition is met. For example, as Figure 1 As illustrated, the autoregressive model performs its first iteration at block 132. N This inference leads to the generation of the final output lexical unit 134. The final output lexical unit 134 may have an index corresponding to a terminating lexical unit in the lexical dictionary used by the generative AI model (e.g., available for use as output by the generative AI model). Therefore, the final output lexical unit 134 can be output, and the inference operation for the input prompt 110 can be terminated.

[0025] Example multi-word output generation in generative artificial intelligence models

[0026] like Figure 1 As illustrated, lexical units can be generated sequentially and individually by a generative AI model. Therefore, the speed at which a generative AI model generates responses to input cues can be defined by the speed at which it performs any given round of inference. That is, if the generative AI model can execute within a defined time period when running on a specific computing device. m This inference suggests that the number of lexical units generated by the generative AI model within a defined time period can also be... m Each word element.

[0027] However, the computing device on which a generative AI model executes may be able to speculatively generate multiple lexical units in any round of inference while processing input prompts. When multiple output lexical units are generated during a particular round of inference, it can be assumed that the first output lexical unit included in the set of output lexical units is a valid output. However, lexical units following the first output lexical unit in the set of output lexical units are speculatively generated based on the assumptions of previously speculatively generated lexical units, and therefore may not be appropriate outputs for the input prompts. Therefore, in order to improve the inference speed of generative AI models (e.g., measured by the number of inferences generated within a defined time period), aspects of this disclosure allow the generation of multiple output lexical units in a single round of inference performed by the generative AI model, and allow the validation of output lexical units based on the previously generated set of output lexical units.

[0028] Figure 2 Example 200 illustrates the use of a generative artificial intelligence model to generate multiple output lexical units in a single round of inference in response to an input query, according to various aspects of this disclosure.

[0029] In Example 200, the generative artificial intelligence model can be configured to output in each round of inference. 1 word element, of which By doing so, generative AI models can be compared to word meta-generation models (e.g., such as...). Figure 1 (As illustrated) This improves the lexical generation rate of generative AI models, thereby accelerating the generation of responses to input prompts. While the preceding example illustrates generating lexicals in each round of inference... Regarding the output lexical units, it should be recognized that the multi-lexical generation technique described in this paper can be applied to... Any value of allows the use of generative artificial intelligence models to generate multiple output lemmas in any round of inference (e.g., where ). ).

[0030] To allow the generative AI model to generate multiple output lexicals in each round of inference, the lexer 202 can lexically transform the input prompt 110 and add one or more padding lexicals to generate a lexically transformed, padded input prompt 204. In some aspects, the number of padding lexicals added to the lexically transformed version of the input prompt 110 can be one less than the number of output lexicals configured for the generative AI model to output, such that the window processing lexicals thereon includes both the final actual lexical from the input prompt 110 and the padding lexicals added to the lexically transformed version of the input prompt 110. For example, where... The number of filler words can be 3, such as Figure 2 exemplified.

[0031] The lexicalized, padded input hints 204 can be fed into the prefix model 206, which populates the KV cache 208 with the lexicalized, padded input hints 204 and generates a KV cache with a length of The prefix output lexicon set 210. The first lexicon generated by the prefix model 206 (e.g., lexicon 1 in the prefix output lexicon set 210, which has a lexicon index 666 in the lexicon dictionary used by the generative artificial intelligence model) can be output as part of the response to the input prompt 110.

[0032] Subsequently, at block 212, the lexicalized, padded input cue 204 (in KV cache 208) and lexical 1 from the prefix output lexical set 210 can be fed into the autoregressive block (labeled "KVAR4") of the generative AI model to generate the first output lexical set 214. The first output lexical set may also have a length To determine which tokens from the first output token set should be output as part of the response to input prompt 110, tokens from the prefix output token set 210 and the first output token set can be compared by token offset to consider the acceptance and output of token 1 from the prefix output token set 210 as part of a valid response to input prompt 110.

[0033] In other words, the lexical element 2 from the prefix output lexical set 210 can be compared with the lexical element 1 from the first output lexical set 214 to determine if the lexical elements match, and thus determine whether the lexical element 2 from the prefix output lexical set 210 can be output as part of the response to the input prompt 110. In this example, the lexical element 2 from the prefix output lexical set 210 has a lexical index 742 in the lexical dictionary, while the lexical element 1 from the first output lexical set 214 has a lexical index 457. Because the lexical element 2 from the prefix output lexical set 210 does not match the lexical element 1 from the first output lexical set 214, the lexical element 1 from the first output lexical set 214 can be considered a valid lexical element for output, and therefore can be output as part of the response to the input prompt 110. Subsequently, as illustrated, a second round of inference can be performed at block 216 to generate a second output lexical set 218 based on the first output lexical set 214.

[0034] In the second round of inference, as illustrated, the resulting second output lexical set 218 may include a first lexical with lexical index 863 and a second lexical with lexical index 142. Since lexical 2 from the first output lexical set 214 is the same as lexical 1 from the second output lexical set 218, it is possible to base this on allowing inference at any round... The output pattern outputs multiple output tokens as the result of the second round of inference, where This corresponds to the number of matching words between the previously generated output word set and the current input word set (e.g., in this round of inference, Therefore, a comparison can be performed between lexical 3 from the first output lexical set 214 and lexical 2 from the second output lexical set 218. In this example, because lexical 3 from the first output lexical set 214 has lexical index 90 and lexical 2 from the second output lexical set 218 has lexical index 142, the generative AI model can select lexical 2 from the second output lexical set 218 as its output. Therefore, the output of the generative AI model in the second round of inference performed at block 216 can include lexicals with lexical indices 863 and 142 in the lexical dictionary.

[0035] Because multiple lexical units are accepted as outputs of the generative AI model at block 216, and because none of the accepted lexical units correspond to a terminating lexical unit in the dictionary, a third round of inference can be performed at block 222. For this purpose, the second output lexical unit set 218 can be transformed into a shifted and padded lexical unit set 220 to take into account the acceptance of both the first and second lexical units from the second output lexical unit set 218. This transformation can be performed by left-shifting the lexical units in the second output lexical unit set 218 to obtain an index. k The lexical units have an index in the shifted and padded lexical set 220. Lexicons with indices less than or equal to 0 can be discarded from the padded lexicon set 220. Therefore, as illustrated, when generating the shifted and padded lexicon set 220, discarded lexicon 1 can be discarded from the second output lexicon set 218, and the remaining lexicons from the second output lexicon set can be shifted left by one index (e.g., so that lexicon 2 from the second output lexicon set 218 is lexicon 1 in the shifted and padded lexicon set 220, lexicon 3 from the second output lexicon set 218 is lexicon 2 in the shifted and padded lexicon set 220, and so on). Furthermore, because the generative AI model can be configured to use a fixed-size input, therefore... A number of padded lexical units can be added to a subset of lexical units from the second output lexical unit set 218, such that the shifted and padded lexical unit set 220 includes... Each word element.

[0036] Subsequently, the shifted and padded lexical set 220 can be input into the autoregressive block of the generative AI model for processing at block 222, resulting in the generation of a third output lexical set 224. The third output lexical set 224 can be processed based on the techniques discussed above, and subsequent rounds of inference can be performed until a terminating lexical is accepted and output as a response to the input prompt, or the generative AI model has generated the maximum number of lexicals.

[0037] Because the technique illustrated in Example 200 allows for the generation and acceptance of multiple output lexical units during any round of inference using a generative artificial intelligence model, Example 200 can provide more than Figure 1 Example 100 illustrates higher lexical throughput. In Example 100, the generative AI model generates a single output lexical in each round of inference until a termination condition is met. Therefore, the processing of input cues using the multi-lexical generation techniques discussed herein can be accelerated using generative AI models (e.g., by generating responses using fewer inference rounds). Using fewer inference rounds saves power and allows computational resources (e.g., processing time, memory, etc.) to be used by other processes on the computing device on which the generative AI model executes.

[0038] Example operations generated using multi-word responses from generative artificial intelligence models.

[0039] Figure 3 Example operation 300, which is executable by a computing device according to various aspects of this disclosure, is illustrated to generate a response to an input prompt using a generative artificial intelligence model that generates multiple output lexical units in each round of inference. Operation 300 can be executed by a device on which at least a generative artificial intelligence model can be deployed, such as a smartphone, tablet computer, laptop computer, desktop computer, server, cloud computing instance hosted in a distributed computing environment, etc.

[0040] As illustrated, operation 300 may begin at box 310, generating a lexicalized input comprising multiple input lexical units. Generally, these multiple input lexical units may have a length corresponding to the maximum length of the output generated by the generative AI model. In some respects, the maximum length of the output generated by the generative AI model may be at least two.

[0041] In some respects, the lexicalized input includes lexical units corresponding to the last part of the received query and multiple filler lexical units. These filler lexical units can be one less than the maximum length of the output generated by the generative AI model, such that the number of lexical units included in the lexicalized input is equal to the maximum length of the output generated by the generative AI model.

[0042] At box 320, operation 300 continues based on the lexicalized input and the generative AI model to generate an output including multiple output lexical units. The output can have a length corresponding to the maximum length of the output generated by the generative AI model.

[0043] In some aspects, generating output may include generating a prefix lexical set based on lexicalized input and a prefix-based generative AI model. The prefix lexical set may include the number of lexical units corresponding to the length of the output generated by the generative AI model. Output may be generated based on the prefix lexical set and input prompts fed into the generative AI model for processing (as described above, it may be fed into a KV cache to conditionally condition the generative AI model's generation of at least a first output lexical set).

[0044] At box 330, operation 300 continues to identify a set of valid words from the output based on a comparison between the plurality of input words and the plurality of output words. In some aspects, identifying a set of valid words from the output may include comparing the word at a first index among the plurality of input words with the word at a second index among the plurality of output words, the second index being offset by one word from the first index. Typically, the set of valid words may include a continuous sequence of matching words from the input word set and the output word set, as well as a first word from the plurality of output words that does not match at the corresponding offset index among the plurality of input words.

[0045] At box 340, operation 300 continues to output a set of valid lexical units as a response to input prompts that are fed into the generative artificial intelligence model for processing.

[0046] In some aspects, operation 300 also includes determining the number of valid tokens in the output. The output can be shifted based on the number of valid tokens. Another lexicalized input can be generated by adding one or more padding tokens to the end of the shifted output, such that the other lexicalized input includes a number of tokens corresponding to the maximum length of the output generated by the generative AI model. The shifted and padded lexicalized input can then be used to generate a subsequent portion of the response to the generative AI model. To this end, another output including multiple additional output tokens can be generated based on the other lexicalized input and the generative AI model. The set of valid tokens can be identified from the other output based on a comparison of the tokens in the other lexicalized input with multiple additional output tokens.

[0047] In some respects, the valid set of terms includes terminating terms. In such cases, operation 300 can continue to terminate the subsequent generation of output for the received query.

[0048] An example processing system for generating multi-lexical responses to input queries using generative artificial intelligence models.

[0049] Figure 4 An example processing system 400 is described, which uses multi-lexical output generation techniques to generate responses to queries input into a generative artificial intelligence model during an inference round performed by a generative artificial intelligence model, such as those described in this paper. Figure 2 and3 As described.

[0050] Processing system 400 includes a central processing unit (CPU) 402, which in some examples may be a multi-core CPU. Instructions executed at CPU 402 may be loaded, for example, from program memory associated with CPU 402 or from a memory partition (e.g., memory 424).

[0051] The processing system 400 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 404, a digital signal processor (DSP) 406, a neural processing unit (NPU) 408, and a connectivity component 412.

[0052] NPUs, such as the NPU 408, are typically configured to implement dedicated circuitry for the control and arithmetic logic used to execute machine learning algorithms, such as those for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. NPUs are sometimes alternatively referred to as neural signal processors (NSPs), tensor processing units (TPUs), neural network processors (NNPs), intelligent processing units (IPUs), vision processing units (VPUs), or graphics processing units.

[0053] NPUs (such as the NPU 408) are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system-on-a-chip (SoC), while in other examples, such NPUs may be part of a dedicated neural network accelerator.

[0054] An NPU can be optimized for either training or inference, or in some cases configured to balance performance between the two. For an NPU capable of performing both training and inference, these two tasks can generally still be performed independently.

[0055] NPUs designed to accelerate training are typically configured to speed up the optimization of new models. This involves taking an existing dataset (usually labeled or sublabeled), iterating over the dataset, and subsequently tuning model parameters (such as weights and biases) to improve model performance—a highly computationally intensive operation. Generally, optimization based on incorrect predictions involves backtracking through the layers of the model and determining gradients to reduce prediction errors.

[0056] NPUs designed to accelerate inference are typically configured to operate on the full model. Therefore, such NPUs can be configured to take a new piece of data as input and process it rapidly through an already trained model to generate model output (e.g., inference).

[0057] In some implementations, the NPU 408 is part of one or more of the CPU 402, GPU 404, and / or DSP 406. These can be located in the user equipment (UE) of a wireless communication system or on another computing device.

[0058] In some examples, connectivity component 412 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., Long Term Evolution (LTE)), fifth-generation (5G) connectivity (e.g., New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Connectivity component 412 may be further coupled to one or more antennas 414.

[0059] The processing system 400 may also include one or more sensor processing units 416 associated with the sensor in any way, one or more image signal processors (ISPs) 418 associated with the image sensor in any way, and / or a navigation processor 420, which may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0060] The processing system 400 may also include one or more input and / or output devices 422, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, etc.

[0061] In some examples, one or more processors in the processing system 400 may be based on the ARM or RISC-V instruction set.

[0062] The processing system 400 also includes a memory 424, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 424 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 400.

[0063] Specifically, in this example, memory 424 includes an input prompt generation component 424A, an output generation component 424B, a lexical identification component 424C, a lexical output component 424D, and a machine learning model component 424E. The depicted components and other undepicted components can be configured to perform various aspects of the methods described herein.

[0064] Typically, the processing system 400 and / or its components can be configured to perform the methods described herein.

[0065] Example Terms

[0066] Specific implementation details of various aspects of this disclosure are described in the following numbered clauses.

[0067] Clause 1: A processor-implemented method for processing input prompts using a generative artificial intelligence model, the processor-implemented method comprising: generating a lexicalized input comprising a plurality of input lexicals having a length corresponding to a maximum length of an output generated by the generative artificial intelligence model, wherein the maximum length of the output generated by the generative artificial intelligence model comprises at least two lexicals; generating an output comprising a plurality of output lexicals based on the lexicalized input and the generative artificial intelligence model, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; identifying a set of valid lexicals from the output based on a comparison of the plurality of input lexicals and the plurality of output lexicals; and outputting the set of valid lexicals as a response to the input prompt.

[0068] Clause 2: The method according to Clause 1, wherein the lexicalized input includes a lexicalized version of the input prompt and one or more filler lexical elements appended to the lexicalized version of the input prompt.

[0069] Clause 3: The method described in Clause 2, wherein the number of one or more filler words is less than the maximum length of the output.

[0070] Clause 4: The method according to Clause 2 or 3, wherein generating the output comprises: generating a prefix lexical set based on the lexicalized input and the prefix generative AI model; and generating the output based on the prefix lexical set.

[0071] Clause 5: The method according to any one of Clauses 1 to 4, wherein the lexicalized input comprises a set of lexical units previously generated by the generative artificial intelligence model.

[0072] Clause 6: The method according to any one of Clauses 1 to 5, wherein identifying the set of valid lexical units from the output comprises: comparing a lexical unit at a first index among the plurality of output lexical units with a lexical unit at a second index among the plurality of input lexical units, the second index being offset by one lexical unit from the first index.

[0073] Clause 7: The method according to any one of Clauses 1 to 6, wherein the effective lexical set comprises a continuous sequence of matching lexical units in the input lexical set and the output lexical set, and a first lexical unit in the plurality of output lexical units that does not match at the corresponding offset index in the plurality of input lexical units.

[0074] Clause 8: The method according to any one of Clauses 1 to 7, the method further comprising: determining the number of valid lexical units in the output; shifting the output based on the number of valid lexical units; and generating another lexicalized input based on adding one or more filler lexical units to the end of the shifted output, such that the other lexicalized input includes a number of lexical units corresponding to the maximum length of the output generated by the generative artificial intelligence model.

[0075] Clause 9: The method according to Clause 8 further comprises: generating another output including a plurality of other output lexical units based on the other lexical input and the generative artificial intelligence model; and identifying a set of valid lexical units from the other output based on a comparison of the lexical units in the other lexical input with the plurality of other output lexical units.

[0076] Clause 10: The method according to any one of Clauses 1 to 9, wherein: the set of valid lexical terms includes a terminating lexical term; and the method further includes: terminating subsequent generation of output for the received query based on the fact that the set of valid lexical terms includes the terminating lexical term.

[0077] Clause 11: A processing system comprising: at least one memory storing executable instructions; and one or more processors coupled to the at least one memory and configured to execute the executable instructions to cause the processing system to perform any one of Clauses 1 to 10.

[0078] Clause 12: An apparatus comprising a component for performing an operation according to any one of Clauses 1 to 10.

[0079] Clause 13: A non-transitory computer-readable medium having executable instructions stored thereon, which, when executed by one or more processors, perform the operations pursuant to any one of Clauses 1 to 10.

[0080] Additional Notes

[0081] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, the function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as appropriate in various examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Moreover, the scope of this disclosure is intended to cover such apparatuses or methods practiced using other structures, functionalities, or structures and functionalities that complement or replace the various aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be embodied by one or more elements of these claims.

[0082] As used herein, the term “exemplary” means “serving as an example, instance, or illustration.” Any aspect described as “exemplary” in this document is not necessarily to be construed as preferred or superior to other aspects.

[0083] As used in this article, the phrase “at least one of” in a list of items refers to any combination of these items, including a single member. For example, “at least one of a, b, or c” is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0084] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., looking in a table, database, or other data structure), ascertainment, and similar actions. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0085] The methods disclosed herein include one or more steps or actions for implementing the methods. The steps and / or actions of the methods may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of a particular step and / or action may be modified without departing from the scope of the claims. Furthermore, the various operations of the methods described above may be performed by any suitable component capable of performing the corresponding function. This component may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, where the operations illustrated in the figures are present, those operations may have corresponding components with similar numbering plus functional components.

[0086] The following claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, reference to the singular form of an element is not intended to mean “one and only one,” but rather “one or more.” Unless otherwise specifically stated, the term “some” refers to one or more. No element of any claim should be interpreted in accordance with 35 USC § 112(f) unless the element is expressly referred to by the phrase “for a component of,” or, in the case of a method claim, by the phrase “for a step of.” All structural and functional equivalents of the elements throughout the various aspects described herein that are known to a person of ordinary skill in the art, or will be known later, are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly stated in the claims.

Claims

1. A processor-implemented method for processing input prompts using a generative artificial intelligence model, the processor-implemented method comprising: Generate a lexicalized input comprising multiple input lexical units, the multiple input lexical units having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the output generated by the generative artificial intelligence model comprises at least two lexical units; Based on the lexicalized input and the generative artificial intelligence model, an output including multiple output lexical units is generated, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; Based on the comparison between the multiple input words and the multiple output words, a set of valid words is identified from the output; as well as Output the set of valid lexical units as a response to the input prompt.

2. The method of claim 1, wherein the lexicalized input includes a lexicalized version of the input prompt and one or more filler lexical units appended to the lexicalized version of the input prompt.

3. The method of claim 2, wherein the number of the one or more filler words is one less than the maximum length of the output.

4. The method of claim 2, wherein generating the output comprises: Based on the lexicalized input and the prefix generative artificial intelligence model, a prefix lexical set is generated; as well as The output is generated based on the aforementioned prefix lexicon set.

5. The method of claim 1, wherein the lexicalized input comprises a set of lexical units previously generated by the generative artificial intelligence model.

6. The method of claim 1, wherein identifying the set of valid tokens from the output comprises: The word at the first index of the plurality of output words is compared with the word at the second index of the plurality of input words, wherein the second index is offset by one word from the first index.

7. The method according to claim 1, wherein the effective lexical set includes a continuous sequence of matching lexical units in the input lexical set and the output lexical set, and a first lexical unit in the plurality of output lexical units that does not match at the corresponding offset index in the plurality of input lexical units.

8. The method according to claim 1, further comprising: Determine the number of valid tokens in the output; The output is shifted based on the number of valid words; as well as Another lexicalized input is generated by adding one or more filler lexical units to the end of the shifted output, such that the other lexicalized input includes a number of lexical units corresponding to the maximum length of the output generated by the generative artificial intelligence model.

9. The method according to claim 8, further comprising: Based on the other lexicalized input and the generative artificial intelligence model, another output including multiple additional output lexical units is generated; as well as Based on the comparison between the lexical units in the other lexical input and the other plurality of output lexical units, a set of valid lexical units is identified from the other output.

10. The method according to claim 1, wherein: The set of valid lexical elements includes terminating lexical elements; and The method further includes: terminating the subsequent generation of output for the received query based on the fact that the set of valid lexical units includes the terminating lexical unit.

11. A processing system for processing input prompts using a generative artificial intelligence model, the processing system comprising: At least one memory, wherein executable instructions are stored in the at least one memory; and One or more processors, the one or more processors being configured to execute the executable instructions to cause the processing system to: Generate a lexicalized input comprising multiple input lexical units, the multiple input lexical units having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the output generated by the generative artificial intelligence model comprises at least two lexical units; Based on the lexicalized input and the generative artificial intelligence model, an output including multiple output lexical units is generated, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; Based on the comparison between the multiple input words and the multiple output words, a set of valid words is identified from the output; as well as Output the set of valid lexical units as a response to the input prompt.

12. The processing system of claim 11, wherein the lexicalized input includes a lexicalized version of the input prompt and one or more filler lexical elements appended to the lexicalized version of the input prompt.

13. The processing system of claim 12, wherein the number of the one or more filler words is one less than the maximum length of the output.

14. The processing system of claim 12, wherein, in order to generate the output, the one or more processors are configured to cause the processing system to: Based on the lexicalized input and the prefix-generative artificial intelligence model, a prefix lexical set is generated; and The output is generated based on the aforementioned prefix lexicon set.

15. The processing system of claim 11, wherein the lexicalized input comprises a set of lexical units previously generated by the generative artificial intelligence model.

16. The processing system of claim 11, wherein, in order to identify the set of valid lexical units from the output, the one or more processors are configured to cause the processing system to: compare a lexical unit at a first index among the plurality of output lexical units with a lexical unit at a second index among the plurality of input lexical units, the second index being offset by one lexical unit from the first index.

17. The processing system of claim 11, wherein the effective lexical set includes a continuous sequence of matching lexical units in the input lexical set and the output lexical set, and a first lexical unit in the plurality of output lexical units that does not match at the corresponding offset index in the plurality of input lexical units.

18. The processing system of claim 11, wherein the one or more processors are further configured to cause the processing system to: Determine the number of valid tokens in the output; Based on the number of valid tokens, shift the output; and Another lexicalized input is generated by adding one or more filler lexical units to the end of the shifted output, such that the other lexicalized input includes a number of lexical units corresponding to the maximum length of the output generated by the generative artificial intelligence model.

19. The processing system of claim 18, wherein the one or more processors are further configured to cause the processing system to: Based on the other lexicalized input and the generative artificial intelligence model, another output including multiple additional output lexical units is generated; and Based on the comparison between the lexical units in the other lexical input and the other plurality of output lexical units, a set of valid lexical units is identified from the other output.

20. The processing system according to claim 11, wherein: The set of valid lexical elements includes terminating lexical elements; and The one or more processors are further configured to cause the processing system to terminate subsequent generation of output for the received query based on the fact that the set of valid lexical units includes the terminating lexical unit.

21. An apparatus for processing input prompts using a generative artificial intelligence model, the apparatus comprising: A component for generating a lexicalized input comprising multiple input lexical units having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the output generated by the generative artificial intelligence model comprises at least two lexical units; A component for generating an output comprising multiple output lexical units based on the lexicalized input and the generative artificial intelligence model, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; A component for identifying a set of valid tokens from the output based on a comparison between the plurality of input tokens and the plurality of output tokens; and A component for outputting the set of valid lexical terms as a response to the input prompt.

22. The apparatus of claim 21, wherein the lexicalized input includes a lexicalized version of the input prompt and one or more filler lexical elements appended to the lexicalized version of the input prompt.

23. The apparatus of claim 22, wherein the component for generating the output comprises: A component for generating a prefix lexical set based on the lexicalized input and the prefix generative artificial intelligence model; and A component used to generate the output based on the set of prefix lexicons.

24. The apparatus of claim 21, wherein the lexicalized input comprises a set of lexical units previously generated by the generative artificial intelligence model.

25. The apparatus of claim 21, wherein means for identifying the set of valid word items from the output comprises: A component for comparing a word at a first index among the plurality of output words with a word at a second index among the plurality of input words, the second index being offset by one word from the first index.

26. The apparatus of claim 21, wherein the effective lexical set comprises a continuous sequence of matching lexical units in the input lexical set and the output lexical set, and a first lexical unit in the plurality of output lexical units that does not match at a corresponding offset index in the plurality of input lexical units.

27. The apparatus of claim 21, further comprising: A component used to determine the number of valid tokens in the output; A component used to shift the output based on the number of valid tokens; and The component for generating another lexicalized input by adding one or more padding lexical units to the end of the shifted output, such that the other lexicalized input includes a number of lexical units corresponding to the maximum length of the output generated by the generative artificial intelligence model.

28. The apparatus of claim 27, further comprising: A component used to generate another output including multiple additional output lexical units based on the other lexicalized input and the generative artificial intelligence model; and A component for identifying a valid set of lexical units from another output based on a comparison of lexical units in the other lexical input with the other plurality of output lexical units.

29. The apparatus according to claim 21, wherein: The set of valid lexical elements includes terminating lexical elements; and The apparatus further includes components for terminating subsequent generation of output for a received query based on the set of valid lexical units including the terminating lexical unit.

30. A non-transitory computer-readable medium storing executable instructions that, when executed by one or more processors, perform operations for processing input prompts using a generative artificial intelligence model, the operations including: Generate a lexicalized input comprising multiple input lexical units, the multiple input lexical units having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model, wherein the maximum length of the output generated by the generative artificial intelligence model comprises at least two lexical units; Based on the lexicalized input and the generative artificial intelligence model, an output including multiple output lexical units is generated, the output having a length corresponding to the maximum length of the output generated by the generative artificial intelligence model; Based on the comparison between the multiple input words and the multiple output words, a set of valid words is identified from the output; as well as Output the set of valid lexical units as a response to the input prompt.

Images