Method for improving processing efficiency of generative model, and electronic device for performing same

Abstract

This processing method of a generative model may comprise the steps of: acquiring a prompt; simplifying the prompt into a simplified prompt including intent and details; searching, on the basis of the intent of the simplified prompt, for records of tasks performed in the past; if the records of the tasks performed in the past, which correspond to the intent of the simplified prompt, are confirmed, using an intermediate computation result corresponding to the tasks performed in the past so as to execute the generative model according to the simplified prompt; and outputting the execution result of the generative model.

Description

Method for improving the processing efficiency of a generative model and an electronic device for performing the same

[0001] The present disclosure relates to a method for improving the processing efficiency of a generative model and an electronic device for performing the same, and more particularly, to a method for improving the processing efficiency by reducing the amount of computation that a generative model must perform and the amount of data transferred between memories during the processing.

[0002] Generative artificial intelligence (AI) technology is widely used in various fields such as summarizing text, answering questions, translation, or image generation.

[0003] Here's a brief example of how generative AI works: When a user inputs a prompt, a command containing a request or question, into a generative model, the generative model can generate a response corresponding to the prompt by performing operations between the matrix corresponding to the input prompt and the matrices contained in the generative model's layers.

[0004] Generative models contain a large number of matrices, which can require significant computational effort when processing prompts. As the length of the prompt increases, the corresponding matrix size increases, dramatically increasing the computational effort. Furthermore, this increased computational effort can also increase the amount of data transferred between memories during the generative model's computations, potentially leading to longer processing times.

[0005] Some aspects are described in detail below, while others will become apparent through the description or may be learned through practice of the examples.

[0006] According to one aspect of the present disclosure, a method for processing a generative model may include the steps of obtaining a prompt, simplifying the prompt into a prompt including an intent and details, searching for a record of operations performed in the past based on the intent of the simplified prompt, executing the generative model according to the simplified prompt using an intermediate computation result corresponding to the operation performed in the past when a record of an operation performed in the past corresponding to the intent of the simplified prompt is confirmed, and outputting an execution result of the generative model.

[0007] According to one aspect of the present disclosure, an electronic device includes a memory storing at least one instruction and at least one processor, and the at least one processor executes the at least one instruction stored in the memory, whereby the electronic device obtains a prompt, simplifies the prompt into a simplified prompt including an intent and details, searches for a record of operations performed in the past based on the intent of the simplified prompt, and, when a record of an operation performed in the past corresponding to the intent of the simplified prompt is confirmed, executes the generation model according to the simplified prompt using an intermediate computation result corresponding to the operation performed in the past, and then outputs an execution result of the generation model.

[0008] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0009] FIG. 1 is a drawing for explaining modules included in an electronic device according to one embodiment of the present disclosure.

[0010] FIG. 2 is a drawing for explaining a hardware configuration included in an electronic device according to one embodiment of the present disclosure.

[0011] FIG. 3 is a drawing for explaining detailed modules included in the token reasoner of FIG. 1 according to one embodiment of the present disclosure.

[0012] FIG. 4 is a diagram illustrating an overall flow of simplifying a prompt and executing the simplified prompt according to one embodiment of the present disclosure.

[0013] FIG. 5 is a diagram illustrating a method for simplifying a prompt by extracting key tokens from among tokens included in the prompt and reducing the number of tokens indicating intent according to one embodiment of the present disclosure.

[0014] FIG. 6 is a diagram illustrating a method for simplifying a prompt by performing vector quantization on tokens indicating intent according to one embodiment of the present disclosure.

[0015] FIG. 7 is a diagram illustrating a method for simplifying a prompt by performing vector quantization on a vector corresponding to the entire prompt according to one embodiment of the present disclosure.

[0016] FIG. 8 is a diagram illustrating a method for simplifying a prompt using a language model according to one embodiment of the present disclosure.

[0017] FIG. 9 is a diagram illustrating a method for simplifying an input sequence including input data and prompts when a generative model according to one embodiment of the present disclosure supports multimodal input.

[0018] FIGS. 10, 11, 12, 13, 14, 15, 16, and 17 are diagrams illustrating methods for an electronic device to retrieve a task history based on the intent of a prompt and execute a generative model using intermediate computational results of previously performed tasks, according to one or more embodiments of the present disclosure.

[0019] FIG. 18 is a diagram illustrating a process in which a generation model performs a full operation according to one embodiment of the present disclosure.

[0020] FIG. 19 and FIG. 20 are diagrams illustrating a process in which a generation model performs some operations using hidden state matrices stored in a previous operation execution process according to one or more embodiments of the present disclosure.

[0021] FIGS. 21, 22, 23, 24, 25, 26, 27 and 28 are flowcharts illustrating a method for improving the processing efficiency of a generation model according to one or more embodiments of the present disclosure.

[0022] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0023] In describing this disclosure, descriptions of technical details well-known in the technical field to which this disclosure pertains will be omitted. This is to avoid obscuring the essence of this disclosure by omitting unnecessary explanations and to convey it more clearly. Furthermore, the terms described below are defined based on their functions in this disclosure and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0024] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0025] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure is complete and to fully inform those skilled in the art of the present disclosure of the scope of the disclosure. An embodiment of the present disclosure may be defined according to the claims. Like reference numerals denote like elements throughout the specification. In addition, when describing an embodiment of the present disclosure, if a detailed description of a related function or configuration is determined to unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted.

[0026] In one embodiment, each block of the flowchart diagrams and combinations of the flowchart diagrams can be performed by computer program instructions. The computer program instructions can be installed on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, and the instructions, when executed by the processor of the computer or other programmable data processing apparatus, can create means for performing the functions described in the flowchart block(s). The computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing apparatus to implement the functions in a particular manner, and the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions can also be installed on a computer or other programmable data processing apparatus.

[0027] Additionally, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one embodiment, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0028] The term '~ unit' used in one embodiment of the present disclosure may represent software or a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the '~ unit' may perform a specific role. Meanwhile, the '~ unit' is not limited to software or hardware. The '~ unit' may be configured to be on an addressable storage medium and may be configured to play one or more processors. In one embodiment, the '~ unit' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided through a specific component or a specific '~ unit' may be combined to reduce the number of components or separated into additional components. In addition, in one embodiment, the '~ unit' may include one or more processors.

[0029] Below, the meanings of terms used in this disclosure are explained.

[0030] "Generative AI" can refer to AI technology capable of generating new text, images, etc. in response to prompts and input data (e.g., text, images, etc.). Representative examples of generative AI are described in the "Generative Models" section below.

[0031] A "generative model" may refer to a neural network model that implements generative AI technology. The generative model can generate text or images based on the intent contained in a prompt. Furthermore, the generative model can learn the patterns and structures of training data to generate new data with similar characteristics to the input data or new data corresponding to the input data. For example, if the prompt is text containing a question, the generative model can generate and output an answer to the question. Additionally, for example, if the prompt is text containing a request, the generative model can output text or images generated according to the request. A transformer executed in an electronic device according to an embodiment of the present disclosure corresponds to a generative model. Instead of "generative model," terms such as "generative artificial intelligence model," "language model," "neural network model," or "model" may also be used.

[0032] A 'prompt' is a sentence or keyword for interaction between a user and a model, and may be text for the user to ask a question or give a command to the model. In other words, a prompt may refer to text or other forms of input that guide the model on what type of output it should generate. In the present disclosure, 'execute a prompt' or 'execute a generative model according to a prompt' may refer to an action in which the generative model performs a task according to the request of the prompt, that is, an action in which the generative model performs a calculation and generates a result corresponding to the prompt when a prompt is input to the generative model. A prompt may include 'intent' and 'details', which will be described in detail below. Terms such as 'instruction' may also be used instead of 'prompt'.

[0033] The "intent" of a prompt can refer to the purpose or goal the user wants to achieve through the model, or it can refer to the intent inherent in the context of the prompt. The remainder of a prompt, excluding the intent, can be called its "details." In other words, the details of a prompt can refer to additional information or conditions that specify the intent of the prompt. For example, if the prompt is "Draw a picture of a bird flying in the sky," the intent could be "Draw a picture" or "Draw a picture of a bird," and "a bird flying in the sky" or "flying in the sky" could be the details. Terms such as "goal," "purpose," "request," or "inquiry" could also be used instead of "intent." Furthermore, terms such as "specifics" or "conditions" could also be used instead of "details."

[0034] "Input data" can refer to the actual data that a model must process or analyze. Input data can take various forms, such as text, images, or audio. For example, if a user requests translation by prompting the model with "Translate the following sentence into Korean," the text to be translated can be the input data. Similarly, if a user requests editing of an image by prompting the model with "Erase the clouds in the sky," the image to be edited can be the input data. Terms such as "source data" or "input values" can also be used instead of "input data."

[0035] An "input sequence" refers to the actual input supplied to a model, and can refer to the entire input that the model must process. In other words, an input sequence can refer to the entire data passed to the model's input layer, and can include not only text prompts but also other forms of input data, such as images and audio. In other words, an input sequence can be a combination of prompts and input data. For example, an input sequence for a text-to-image model can include image data to be edited and a prompt (text) instructing the editing.

[0036] For example, if a user inputs the sentence "He always inspires me" as input data to the model along with the prompt "Translate the following sentence into Korean," the input sequence may be "Translate the following sentence into Korean. He always inspires me." As a further embodiment, if a user inputs the image to be edited along with the prompt "Erase the clouds in the sky" as input data to the model, the input sequence may be a combination of "Erase the clouds in the sky in the photo" and the image. In one embodiment, the input data, the image, may be text-formatted, and the input sequence may be a combination of the text and the prompt. Instead of the 'input sequence,' terms such as 'complete input,' 'input stream,' or 'input series' may also be used.

[0037] A "token reasoner" may refer to a component that analyzes input text and performs transformations on the text. In the present disclosure, the token reasoner can simplify prompts, thereby enabling the generative model to operate more efficiently. The token reasoner can understand the intent and context of the prompt and simplify the prompt based on the results. For example, the token reasoner can restructure the prompt by removing or modifying some of the tokens included in the prompt.

[0038] In the present disclosure, an electronic device can use a token reasoner to make prompts clear and concise, thereby removing unnecessary information from the prompt and emphasizing important information, thereby enabling a model's responses to be more accurate. The rules by which the token reasoner transforms (simplifies) the prompt can be implemented in various ways, and various techniques can be used when transforming the prompt. For example, in one embodiment of the present disclosure, the token reasoner can simplify the prompt using at least one of token pruning and vector quantization.

[0039] The token reasoner can be implemented as part of the generative model or separately, external to the generative model. Furthermore, the token reasoner can be implemented as a rule-based system or can include a neural network. Instead of the term "token reasoner," terms such as "prompt simplification module" or "prompt compression module" may also be used.

[0040] A "token pruner" can refer to a component that performs token pruning, i.e., removing unnecessary tokens from a prompt or input sequence to improve model efficiency. A token pruner can evaluate the importance of each token and remove tokens with low importance. Specifically, a token pruner can retain key or main tokens and remove dummy or auxiliary tokens from the tokens included in a prompt or input sequence. The importance of each token can be determined by an attention mechanism or other various evaluation criteria.

[0041] A "task database" may refer to a space where data related to tasks performed using a model is stored. The task database may store information on a user's previous use of the model. According to one embodiment of the present disclosure, the task database may store information on prompts corresponding to previously performed tasks (e.g., an embedding matrix corresponding to a prompt, a codebook generated for a vector corresponding to a prompt, a codebook generated for a token sequence indicating an intent included in a prompt, etc.). In addition, the task database may store intermediate computation results generated in the process of performing a previous task. For example, a hidden state matrix output from each layer of the model (e.g., an attention value matrix, an activation matrix, etc.) may be stored. According to one embodiment of the present disclosure, the task database may be a personalized database implemented in the cloud or on-device. That is, the task database may be provided to correspond to each user account. Instead of 'work database', terms such as 'personal database', 'personal knowledge graph', or 'database' may be used.

[0042] The term 'Hidden State Matrix (HSM)' may refer to the intermediate operation results output from each layer (e.g., self-attention layer, etc.) included in the model when the model executes a prompt. In other words, the hidden state matrix may refer to the result value of performing an operation on tokens included in the input sequence using the weights included in the hidden layer of the model. There may be a corresponding HSM for each of the various layers included in the model (e.g., attention layer, activation layer, etc.). Therefore, the HSM may include an attention score matrix, an attention value matrix, an activation matrix, etc. Instead of 'hidden state matrix', terms such as 'latent matrix', 'latent variable matrix', or 'intermediate matrix' may also be used.

[0043] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0044] The present disclosure relates to a method for improving the processing efficiency of a generative model, and embodiments of the present disclosure have the characteristic of reducing the amount of computation that the generative model must perform and also reducing the amount of data transferred between memories during the processing by simplifying the prompts input to the generative model and utilizing intermediate computational results (e.g., HSM) obtained in the process of executing the same or similar prompts previously. Therefore, by using the method according to the embodiments of the present disclosure, the generative model can be executed even on electronic devices with relatively low specifications, thereby enabling on-device implementation of the generative model.

[0045] The features of the embodiments of the present disclosure are briefly summarized as follows.

[0046] (A) Simplifying the prompts input to the generative model (token reasoning)

[0047] - Remove dummy tokens from the tokens included in the prompt, leaving only the key tokens.

[0048] - Reduce the number of tokens by performing vector quantization on the tokens that indicate intent among the tokens included in the prompt.

[0049] - Reduce the number of tokens by performing vector quantization on the vector corresponding to the entire prompt.

[0050] (B) Using the intermediate operation results generated during the previous prompt execution process

[0051] - If a prompt that is essentially the same as the one entered has been run before, perform zero operations (omitting operations on the generated model).

[0052] - If a similar prompt to the one entered (a prompt with the same intent but different details) has been run before, perform some operations (perform operations on the generated model using HSM)

[0053] - If the same or similar prompt as the input prompt has not been executed before, perform a full operation (perform the operation of the generative model on the embedding matrix corresponding to the input prompt)

[0054] According to embodiments of the present disclosure, the amount of computation of a generation model can be primarily reduced by simplifying prompts, and the amount of computation can be secondarily reduced by performing zero or some operations in the generation model.

[0055] Although a generative model is used in the embodiments of the present disclosure, the method according to the embodiments of the present disclosure can also be applied to various other neural network models.

[0056] First, with reference to FIGS. 1 to 3, a configuration and operation of an electronic device according to an embodiment of the present disclosure will be described, with reference to FIGS. 4 to 9, a method for an electronic device according to an embodiment of the present disclosure to simplify a prompt will be described, and with reference to FIGS. 10 to 20, a method for an electronic device according to an embodiment of the present disclosure to execute a prompt using intermediate operation results stored in a previous task execution process will be described.

[0057] 1. Overall configuration and operation of the electronic device

[0058] FIG. 1 is a diagram illustrating modules included in an electronic device according to one embodiment of the present disclosure. Referring to FIG. 1, an electronic device (1000) according to one embodiment of the present disclosure may include a token listener (100), a task management module (200), a generation model (300), an output module (400), and a task database (500).

[0059] The modules (100, 200, 300, 400, 500) included in the electronic device (1000) of FIG. 1 are components classified based on their functions or roles. The modules (100, 200, 300, 400, 500) of the electronic device (1000) of FIG. 1 may be software components implemented by the processor (1300) of the electronic device (1000), which will be described later with reference to FIG. 2, executing a program stored in the memory (1400), or may be virtual components for which no matching hardware device actually exists. In other words, the operations performed by the processor (1300) of the electronic device (1000) by executing a program or instruction stored in the memory (1400) may be classified into a plurality of groups by function or purpose, and the entities performing the operations included in each classified group may be expressed as the modules (100, 200, 300, 400, 500) of FIG. 1. Accordingly, the operations described as being performed by the modules (100, 200, 300, 400, 500) of the electronic device (1000) illustrated in FIG. 1 may actually be viewed as being performed by the processor (1300) of the electronic device (1000) by executing a program or instruction stored in the memory (1400).

[0060] In FIG. 1, one electronic device (1000) is illustrated as including all modules (100, 200, 300, 400, 500), but this is not limited thereto, and at least some of the modules (100, 200, 300, 400, 500) may be implemented to be included in a separate device, or one module may be implemented to be included in another module. In this way, the modules (100, 200, 300, 400, 500) included in the electronic device (1000) according to one embodiment of the present disclosure may be a hardware configuration or a software configuration, and may be implemented in the form of various electronic devices (e.g., one electronic device or a combination of two or more electronic devices).

[0061] An electronic device (1000) according to one embodiment of the present disclosure may be a user's terminal (e.g., a smartphone, a laptop, a desktop, etc.), but is not limited thereto, and may also be a server that performs communication with the user's terminal.

[0062] The token reasoner (100) is a configuration for simplifying a prompt received from a user (1). The token reasoner (100) can simplify the prompt by reducing the number of tokens included in the prompt. When converting a prompt into an embedding matrix, the embedding dimension (embedding size) is often set to a large value because the generation model (300) can understand the complex meaning and contextual information of the prompt as the embedding dimension (embedding size) increases. However, if the embedding dimension has a large value, the computational load of the generation model (300) can increase significantly even if the number of tokens included in the prompt increases only slightly. Therefore, the token reasoner (100) can primarily reduce the computational load of the generation model (300) by reducing the number of tokens included in the prompt.

[0063] According to one embodiment of the present disclosure, the token reasoner (100) can simplify a prompt based on a history of a user (1) using the electronic device (1000). In other words, the token reasoner (100) can simplify a prompt based on previously executed prompts on the electronic device (1000). If the user (1) frequently and repeatedly inputs the same or similar prompts, the token reasoner (100) can simplify a newly input prompt based on the prompts previously input by the user (1). Furthermore, according to one embodiment of the present disclosure, the token reasoner (100) can simplify a prompt by removing some tokens based on the importance of the tokens included in the prompt, or by reducing the number of tokens while maintaining the same intent and context.

[0064] The token reasoner (100) may utilize vector quantization in the process of simplifying the prompt. For example, the token reasoner (100) may convert the entire prompt into a vector, or convert a set of tokens (token sequence) representing the intent among the tokens included in the prompt into a vector, and perform vector quantization using a code book stored in the work database (500).

[0065] If the generative model (300) supports multimodal input, the token reasoner (100) can also simplify an input sequence that combines a prompt (e.g., text requesting image editing) and input data (e.g., image data to be edited). For example, input data in the form of an image or audio, etc., is converted to text and then combined with a prompt to generate a text-based input sequence, and the token reasoner (100) can simplify the input sequence generated in this way. The method by which the token reasoner (100) simplifies the input sequence may be the same as the method by which the prompt is simplified.

[0066] According to one embodiment of the present disclosure, the token reasoner (100) can process a natural language processing process with the goal of "simplifying prompts." The token reasoner (100) may include a neural network, but may also operate without a neural network using other algorithms or rule-based methods. According to embodiments of the present disclosure, the token reasoner (100) can improve the processing efficiency of the generation model (300) by reducing the number of tokens in advance based on usage history, etc. before executing the generation model (300).

[0067] In this disclosure, the token reasoner (100) is expressed as 'simplifying' the prompt, but other expressions such as optimization of the prompt or compression of the prompt may also be used.

[0068] The specific method by which the token reasoner (100) simplifies the prompt is described in detail below with reference to FIGS. 4 to 9.

[0069] The task management module (200) is a configuration for managing the execution of a task using the generation model (300). The task management module (200) can obtain a simplified prompt from the token listener (100), check whether a prompt identical to or similar to the simplified prompt has been executed previously, and control the execution of the operation of the generation model (300) based on the confirmation result. If a similar prompt has been executed previously, the task management module (200) can shorten the operation time of the generation model (300) by caching the intermediate operation result stored in the task database (500). For example, the task management module (200) can read an HSM corresponding to a similar prompt that has been executed previously from a flash memory and store it in a cache memory (e.g., a cache memory of a CPU or a cache memory of an NPU, etc.), and cause the generation model (300) to use the HSM stored in the cache memory when performing an operation.

[0070] The work database (500) described below may store records of tasks performed in the past. In addition, the work database (500) may store information related to tasks performed in the past (e.g., prompts corresponding to previous tasks, intermediate operation results generated when performing previous tasks, etc.). The work management module (200) may search the work database (500) based on a simplified prompt to determine whether the same or similar prompt has been executed before.

[0071] If a prompt identical to the simplified prompt received from the token listener (100) has been previously executed, the task management module (200) may obtain an API request corresponding to the previously executed prompt from the task database (500) and request execution by transmitting the API request to the output module (400). At this time, the task management module (200) may control the generation model (300) not to perform an operation (zero operation).

[0072] If a prompt similar to, but not identical to, the simplified prompt received from the token reasoner (100) has been previously executed, the job management module (200) may obtain an intermediate operation result (e.g., HSM) corresponding to the previously executed prompt from the job database (500) and control the generation model (300) to perform an operation using the obtained intermediate operation result. (Some operations)

[0073] If a prompt identical to or similar to the simplified prompt received from the token reasoner (100) has not been executed before, the task management module (200) can control the generation model (300) to perform an operation using the embedding matrix corresponding to the simplified prompt as input. (Full operation)

[0074] In summary, the task management module (200) can control the generation model (300) to operate in one of zero operations, partial operations, or full operations based on whether a prompt identical to or similar to the prompt entered by the user (1) or a simplified prompt therefrom has been previously executed.

[0075] A specific method for controlling the task management module (200) to search the task database (500) based on a simplified prompt and to perform a calculation using the intermediate calculation result by the generation model (300) is described in detail below with reference to FIGS. 10 to 20.

[0076] The generative model (300) may be a generative AI model for generating text, images, audio, etc. according to a prompt input by a user (1). According to one embodiment of the present disclosure, the generative model (300) may be implemented in the form of a transformer and may include an encoder (310) and a decoder (320). The encoder (310) and the decoder (320) of the generative model (300) may each include a plurality of attention layers and feedforward layers.

[0077] In the process in which the generation model (300) performs operations according to the prompt, the layers included in the encoder (310) and decoder (320) output matrices and pass them to the next layer. The matrices generated in the middle of the operation process are called hidden state matrices (HSMs). For example, in the process in which the generation model (300) performs operations, the attention layers may output an attention value matrix, and the feedforward layers may output an activation matrix. Both the attention value matrix and the activation matrix are included in the HSM. The HSMs output from the layers of the generation model (300) may be stored in the work database (500) as intermediate operation results.

[0078] According to one embodiment of the present disclosure, the generation model (300) may be an on-device model installed in an electronic device (1000), which is a terminal of a user (1). For example, the generation model (300) may be executed by the processor (1300) of the electronic device (1000), which will be described later, executing a program stored in a memory (1400).

[0079] The output module (400) may be configured to generate output to be provided to the user (1) according to the output of the generation model (300) or a request of the task management module (200).

[0080] When the generation model (300) performs some or all operations, the output module (400) can appropriately transform and output the output of the generation model (300) according to the input / output interface (1200) of the electronic device (1000). For example, when the output of the generation model (300) is in the form of text, the output module (400) can output the text on the screen in a predetermined font and size, or convert it into voice and output it through a speaker.

[0081] When the generation model (300) performs a zero operation, the output module (400) can output the same result of a previously performed task by calling an API according to a request received from the task management module (200).

[0082] The task database (500) may be configured to store information related to tasks performed by the generation model (300). According to one embodiment of the present disclosure, the task database (500) may store records of tasks performed in the past. According to one embodiment of the present disclosure, the task database (500) may store information related to previously performed tasks. For example, the task database (500) may store a codebook generated by clustering vectors corresponding to previously executed prompts. For example, the task database (500) may store a codebook generated by clustering vectors corresponding to a set of tokens (token sequences) representing intent among tokens included in previously executed prompts. The codebooks stored in the task database (500) may be used by the token reasoner (100) in the process of simplifying prompts, or by the task management module (200) in the process of searching for previously performed tasks.

[0083] According to one embodiment of the present disclosure, the work database (500) may store HSMs output from layers included in the generation model (300) during the computation process. The work management module (200) may cache the HSMs stored in the work database (500) so that the generation model (300) may utilize them during computation when a prompt similar to a previously executed prompt is executed.

[0084] According to one embodiment of the present disclosure, the task database (500) may store token sequences that indicate the intent of frequently used prompts. Here, the term "token sequence" may refer to a set of one or more tokens or a unit listing one or more tokens. The stored token sequences may be used in the process of simplifying prompts. This will be described in detail below with reference to FIGS. 5 and 6.

[0085] According to one embodiment of the present disclosure, the work database (500) may be stored in a flash memory, and the electronic device (1000) may store some of the data stored in the work database (500) in a cache memory during the process of performing a work so as to quickly utilize it for calculation.

[0086] According to one embodiment of the present disclosure, the task database (500) may include one or more personalized knowledge graphs, wherein nodes in the knowledge graph may establish relationships between the intent and details of previously executed prompts, as well as the intermediate computational results. Accordingly, the knowledge graph may be used to determine which prompts have been previously executed, what intermediate computational results were obtained when executing the prompts, and so on.

[0087] According to one embodiment of the present disclosure, the work database (500) may include databases having various other structures and forms.

[0088] The data stored in the work database (500) will be further described below.

[0089] FIG. 2 is a diagram illustrating a hardware configuration included in an electronic device according to one embodiment of the present disclosure. Referring to FIG. 2, an electronic device (1000) according to one embodiment may include a communication interface (1100), an input / output interface (1200), a processor (1300), and a memory (1400). However, the components of the electronic device (1000) are not limited to the above-described examples, and the electronic device (1000) may include more or fewer components than the above-described components. Some or all of the communication interface (1100), the input / output interface (1200), the processor (1300), and the memory (1400) may be implemented in the form of a single chip.

[0090] The communication interface (1100) is a configuration for transmitting and receiving signals (such as control commands and data) with an external device via wire or wirelessly, and may be implemented to include a communication chipset that supports various communication protocols. The communication interface (1100) may receive signals from the outside and output them to the processor (1300), or transmit signals output from the processor (1300) to the outside. The electronic device (1000) may communicate with external devices via the communication interface (1100).

[0091] The input / output interface (1200) may include an input interface (e.g., a touch screen, a keyboard, a microphone, etc.) for receiving commands or information from a user (1), and an output interface (e.g., a display panel, a speaker, etc.) for displaying the results of execution of an operation according to a user's command or the status of the electronic device (1000). According to one embodiment of the present disclosure, the electronic device (1000) may receive a prompt and input data from the user (1) through the input / output interface (1200), and when a task is completed, may output the results of performing the task (e.g., an answer to a question included in a prompt, an image edited according to a request of a prompt, etc.) through the input / output interface (1200).

[0092] The processor (1300) controls a series of processes to operate the electronic device (1000) according to the embodiments described below, and may be composed of one or more processors. The one or more processors included in the processor (1300) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The one or more processors included in the processor (1300) may be a general-purpose processor such as a Central Processing Unit (CPU), a Micro Processor Unit (MPU), an Application Processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a Graphics Processing Unit (GPU), a Vision Processing Unit (VPU), an artificial intelligence-only processor such as a Neural Processing Unit (NPU), or a communication-only processor such as a Communication Processor (CP). When the one or more processors included in the processor (1300) are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0093] The processor (1300) can write data to the memory (1400) or read data stored in the memory (1400), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (1400). Accordingly, the processor (1300) can perform operations described in the following embodiments, and operations described as being performed by the electronic device (1000) or modules (100, 200, 300, 400, 500) included in the electronic device (1000) in the following embodiments can be regarded as being performed by the processor (1300) unless otherwise specified.

[0094] The memory (1400) is a configuration for storing various programs or data, and may be configured as a storage medium such as a ROM, a RAM, a hard disk, a CD-ROM, and a DVD, or a combination of storage media. The memory (1400) may not exist separately and may be configured to be included in the processor (1300). The memory (1400) may be configured as a volatile memory, a non-volatile memory, or a combination of volatile memory and non-volatile memory. A program or at least one instruction for performing operations according to embodiments described below may be stored in the memory (1400). The memory (1400) may also provide stored data to the processor (1300) upon request of the processor (1300).

[0095] FIG. 3 is a diagram illustrating detailed modules included in the token reasoner of FIG. 1. Referring to FIG. 3, the token reasoner (100) may include a token pruner (110) and a vector quantization module (120). According to one embodiment of the present disclosure, the token reasoner (100) may include only one of the token pruner (110) and the vector quantization module (120).

[0096] The token pruner (110) is configured to remove tokens with low importance from among the tokens included in the prompt. The token pruner (110) can remove tokens that are less relevant to the overall context or less relevant to the intent of the prompt from among the tokens included in the prompt.

[0097] According to one embodiment of the present disclosure, the token pruner (110) can use an attention mechanism to evaluate the importance of each token based on its weight or activation output, and remove tokens with low importance. According to one embodiment, the token pruner (110) can also determine the importance of tokens based on the frequency of use by the user (1). For example, the token pruner (110) can determine that a token with a high frequency of use has a high importance. According to one embodiment, the token pruner (110) can also determine the importance of tokens in a rule-based manner.

[0098] A token pruner (110) can reduce the number of tokens by classifying tokens with an importance level higher than a certain standard as key tokens and removing the remaining tokens (which can be referred to as 'dummy tokens').

[0099] The vector quantization module (120) is configured to reduce the number of tokens through vector quantization. According to one embodiment of the present disclosure, the vector quantization module (120) receives key tokens from the token pruner (110) and performs vector quantization on a set of tokens representing intent among the key tokens, thereby further reducing the number of tokens.

[0100] The vector quantization module (120) can generate a codebook through vector quantization, and the codebook can be used to map specific data (e.g., vectors corresponding to a token sequence) to multiple data groups. For example, if the input prompts have the same intention but are composed of different token sequences, such as "Turn on the TV," "Can you turn on the television?", or "I'm bored, can you turn on the television?", there is no problem in replacing all of them with the token sequence "Turn on the TV." Accordingly, the vector quantization module (120) can select the prompt "Turn on the TV" composed of the token sequence including the fewest tokens as a prompt to replace the above prompts.

[0101] The vector quantization module (120) converts a token sequence consisting of tokens representing the intent of a prompt into a single vector by normalizing it in the time dimension, and performs quantization using a codebook stored in the working database (500), thereby outputting a token sequence having the same intent but including a smaller number of tokens. In more detail, the vector quantization module (120) finds a code index that is closest to a single vector converted from the token sequence based on a Euclidean distance in the codebook stored in the working database (500), and outputs a token sequence having the same intent but including a smaller number of tokens as a representative token sequence through the code index.

[0102] That is, if the token pruner (110) primarily reduces the number of tokens based on the importance of the tokens, the vector quantization module (120) can secondarily reduce the number of tokens by performing vector quantization on tokens that indicate intent.

[0103] To this end, a codebook in which multiple token sequences are clustered by intent may be pre-stored in the work database (500). That is, a codebook in which token sequences representing intents are clustered may be pre-stored in the work database (500). In this case, the clustered token sequences may be extracted from prompts previously executed by the electronic device (1000). Additionally, when the electronic device (1000) executes a new prompt, the codebook may be updated according to the newly executed prompt.

[0104] As described above, the token reasoner (100) may be configured to include only one of the token pruner (110) and the vector quantization module (120). Therefore, according to one embodiment of the present disclosure, the vector quantization module (120) may perform vector quantization on a prompt that has not passed through the token pruner (110). For example, the vector quantization module (120) may convert a prompt input by a user (1) into a vector and perform quantization using a codebook stored in the work database (500), thereby outputting a prompt that includes a smaller number of tokens than the input prompt.

[0105] To this end, a codebook in which vectors corresponding to prompts previously executed on the electronic device (1000) are clustered may be pre-stored in the work database (500). Additionally, when the electronic device (1000) executes a new prompt, the codebook may be updated according to the newly executed prompt.

[0106] The method by which the token pruner (110) removes some tokens based on importance and the method by which the vector quantization module (120) reduces the number of tokens in a prompt through vector quantization are described in detail below with reference to FIGS. 5 to 7.

[0107] According to one embodiment of the present disclosure, the token reasoner (100) may simplify the prompt by not including a vector quantization module (120) and using various tree-based search algorithms. That is, the token reasoner (100) may express the text corresponding to the prompt in a tree form and perform a search based on the tree structure thus obtained, thereby reducing the number of tokens included in the prompt.

[0108] For example, a token reasoner (100) can make text concise by using a parse tree that represents the syntactic structure of text and hierarchically expresses sentence components (nouns, verbs, etc.) and omitting unimportant details or removing redundant expressions.

[0109] For example, a token reasoner (100) can store frequently appearing words or phrases in a text in a trie structure to find and remove or compress duplicate expressions.

[0110] For example, a token reasoner (100) can create a search tree by organizing multiple sentences with similar meanings into a tree structure, and can also select the most concise expression using the search tree.

[0111] Below, the process of the electronic device (1000) described above simplifying the prompt and performing the operation of the generation model (300) using the intermediate operation results stored in advance is described in detail.

[0112] 2. Simplifying the prompt

[0113] The token reasoner (100) of an electronic device (1000) according to one embodiment of the present disclosure can reduce the amount of data transferred between memories during the execution of a task by reducing the number of tokens included in a prompt, and can also increase computational efficiency. The token reasoner (100) can convert the prompt input by a user (1) into a more concise and clearer form, thereby enabling the generation model (300) to generate accurate and relevant output.

[0114] FIG. 4 is a diagram illustrating an overall flow of simplifying a prompt and executing the simplified prompt according to one embodiment of the present disclosure. When a user (1) inputs a prompt (41) such as "Could you tell me about the weather in Seoul tomorrow?" into an electronic device (1000), the token reasoner (100) of the electronic device (1000) can output a simplified prompt (42) by removing some tokens from the prompt and changing some tokens. The simplified prompt (42) of FIG. 4 includes a token sequence indicating an intent ("weather forecast") and a token sequence indicating details ("Seoul tomorrow").

[0115] The electronic device (1000) can provide an output (43) of "The weather in Seoul tomorrow will be partly cloudy with a high of 20℃" by executing the generative model (300) according to the simplified prompt (42).

[0116] The process of converting an input prompt (41) into a simplified prompt (42) using at least one of a token pruner (110) and a vector quantization module (120) is described in detail with reference to FIGS. 5 to 7.

[0117] (1) Remove some tokens based on importance (token pruning)

[0118] As previously described, the token pruner (110) may remove some tokens based on the importance of each token included in the prompt. The token pruner (110) may remove tokens that have little impact on the intent or context of the prompt.

[0119] FIG. 5 is a diagram illustrating a method for simplifying a prompt by extracting key tokens from among tokens included in the prompt and reducing the number of tokens indicating intent according to one embodiment of the present disclosure.

[0120] Referring to FIG. 5, the electronic device (1000) can extract multiple tokens by performing tokenization on the input prompt (41). As illustrated in FIG. 5, the prompt (41) can be divided into 10 tokens.

[0121] The token pruner (110) can extract only key tokens from the 10 extracted tokens and remove the remaining dummy tokens. As described above, the token pruner (110) can determine the importance of tokens based on their frequency of use or by using an attention mechanism, and classify tokens with an importance above a certain standard as key tokens. In Fig. 5, five tokens, namely "tell", "me", "weather", "Seoul", and "tomorrow", are classified as key tokens, and the remaining tokens are removed by the token pruner (110).

[0122] In this way, the token pruner (110) can leave only the core request portion by removing tokens with low importance from the prompt (41).

[0123] (2) Reduction of the number of tokens using vector quantization

[0124] The vector quantization module (120) may perform quantization on tokens indicating intent included in the prompt, or on the entire prompt. First, a method for performing quantization on tokens indicating intent will be described with reference to FIGS. 5 and 6 , and then a method for performing quantization on the entire prompt will be described with reference to FIG. 7 .

[0125] 1) Perform vector quantization on tokens that indicate intent.

[0126] Referring to FIG. 5, when the token pruner (110) extracts 5 key tokens from among 10 tokens separated from the prompt (41), the vector quantization module (120) can then perform vector quantization to further reduce the number of tokens.

[0127] According to one embodiment of the present disclosure, the vector quantization module (120) can identify the user's intention included in the prompt and simplify the prompt based on the intention. In FIG. 5, the vector quantization module (120) can perform quantization on a token sequence including "tell," "me," and "weather," which are tokens representing intentions among five key tokens. Since texts with the same intention may exist as long or short texts, the vector quantization module (120) can convert the text into a token sequence ("weather forecast") that includes a smaller number of tokens while maintaining the same intention through quantization. A specific method by which the vector quantization module (120) performs quantization on a token sequence representing an intention will be described in detail with reference to FIG. 6.

[0128] FIG. 6 is a diagram illustrating a method for simplifying a prompt by performing vector quantization on tokens indicating intent according to one embodiment of the present disclosure.

[0129] Fig. 6 illustrates a codebook (60) for vector quantization. Referring to Fig. 6, the codebook (60) includes 10 clusters, each cluster including a centroid (C1 to C10).

[0130] The vector quantization module (120) can convert a token sequence indicating an intention ("tell me weather") into a vector Vi, identify the cluster to which the vector Vi belongs in the codebook (60), and then assign a code index (code index 3) corresponding to the identified cluster to the vector Vi. The assigned code index (code index 3) can then be used by the task management module (200) in the process of finding an intermediate operation result (e.g., HSM). (See FIGS. 12 to 15)

[0131] According to one embodiment of the present disclosure, unlike a general vector quantization technique, the vector quantization module (120) may map the vector Vi to a vector (hereinafter, “minimum token vector”) corresponding to a token sequence including the smallest number of tokens among vectors included in the same cluster, rather than to a code vector representing the center point C3. To this end, the codebook (60) may include information on which the minimum token vector for each cluster is mapped to a code index.

[0132] Referring to Figure 6, vector Vi is included in the cluster corresponding to center point C3. The token sequence "show me weather" corresponding to center point C3 contains three tokens. The smallest token vector included in the cluster corresponding to center point C3 is vector Vs, and the token sequence "weather forecast" corresponding to vector Vs contains two tokens.

[0133] The vector quantization module (120) can reduce the number of tokens by mapping the vector Vs to the vector Vi and replacing the token sequence "tell me weather" with "weather forecast." Accordingly, the prompt (41) input by the user (1) can be simplified into a prompt (42) having four tokens, such as "weather forecast Seoul tomorrow," by passing through the token reasoner (110) and the vector quantization module (120).

[0134] The electronic device (1000) can perform clustering on token sequences included in previously executed prompts for vector quantization to generate a codebook (60) and store it in a work database (500).

[0135] 2) Perform vector quantization on the vector corresponding to the prompt.

[0136] According to one embodiment of the present disclosure, the token reasoner (100) may be configured to include only a vector quantization module (120) and not a token pruner (110). The vector quantization module (120) may simplify the prompt by performing quantization on the prompt entered by the user (1).

[0137] FIG. 7 is a diagram illustrating a method for simplifying a prompt by performing vector quantization on a vector corresponding to the entire prompt according to one embodiment of the present disclosure.

[0138] Fig. 7 illustrates a codebook (70) for vector quantization. The codebook (60) of Fig. 6, used when performing quantization on a token sequence indicating intent, and the codebook (70) of Fig. 7, used when performing quantization on the entire prompt, may be different from each other, but may also be identical. Referring to Fig. 7, the codebook (70) includes 10 clusters, each of which includes a center point (C1 to C10).

[0139] The vector quantization module (120) can convert a prompt (41) input by a user (1) into a vector Vp, check the cluster to which the vector Vp belongs in the codebook (70), and then assign a code index (code index 2) corresponding to the checked cluster to the vector Vp. The assigned code index (code index 2) can be used later in the process of the task management module (200) finding an intermediate operation result (e.g., HSM).

[0140] As previously described with reference to FIG. 6, according to one embodiment of the present disclosure, the vector quantization module (120) may map the vector Vp to a minimum token vector instead of mapping the vector Vp to a code vector representing the center point C2. To this end, the codebook (70) may include information on how the minimum token vector is mapped to a code index for each cluster.

[0141] Referring to Figure 7, vector Vp is included in the cluster corresponding to center point C2. The prompt "What will the weather be like in Seoul tomorrow?" corresponding to center point C2 contains 9 tokens. The smallest token vector included in the cluster corresponding to center point C2 is vector Vs, and the prompt "Weather forecast Seoul tomorrow" corresponding to vector Vs contains 4 tokens.

[0142] The vector quantization module (120) can reduce the number of tokens by mapping the vector Vs to the vector Vp and replacing the prompt "Could you tell me about the weather in Seoul tomorrow?" with "Weather forecast Seoul tomorrow." Accordingly, the prompt (41) input by the user (1) can be simplified into a prompt (42) having four tokens, such as "weather forecast Seoul tomorrow," by passing through the vector quantization module (120).

[0143] The electronic device (1000) can perform clustering on previously executed prompts for vector quantization to generate a codebook (70) and store it in a work database (500).

[0144] (3) Reduction of the number of tokens using a language model

[0145] According to one embodiment of the present disclosure, the token reasoner (100) may simplify the prompt using a language model.

[0146] FIG. 8 is a diagram illustrating a method for simplifying a prompt using a language model according to one embodiment of the present disclosure. The prompt conversion model (130) may be a language model trained to convert the core content, i.e., intent and context, of an input text (prompt) into text containing a smaller number of tokens. Referring to FIG. 8, when a prompt (41) input by a user (1) is input into the prompt conversion model (130), the prompt conversion model (130) may output a simplified prompt (42).

[0147] According to one embodiment of the present disclosure, the token reasoner (100) may include a prompt transformation model (130) instead of a token pruner (110) and a vector quantization module (120).

[0148] (4) Simplify the input sequence for multimodal input.

[0149] According to one embodiment of the present disclosure, the generative model (300) may support multimodal input. For example, the generative model (300) may receive input data (e.g., images, audio, etc.) along with a prompt requesting processing of the input data (e.g., editing, summarizing, etc.), and may process the input data according to the request of the prompt.

[0150] If the generation model (300) supports multimodal input, the token reasoner (100) can generate an input sequence by converting input data into text and then merging it with a prompt, and can simplify the input sequence. At this time, the method by which the token reasoner (100) simplifies the input sequence may be identical to the method by which the token reasoner (100) simplifies the prompt, as described above.

[0151] Referring to FIG. 9, when a user (1) inputs a prompt (92) saying "Change the color of the eyeglass frame to red" along with an input image (90) into an electronic device (1000), the token reasoner (100) of the electronic device (1000) can obtain the text (91) saying "Black horn-rimmed glasses" by converting the input image (90) into text.

[0152] The token reasoner (100) can obtain an input sequence (93) of "Change the color of the black horn-rimmed glass frame to red" by merging the text (91) converted from the input image (90) and the prompt (92). Subsequently, the token reasoner (100) can simplify the input sequence (93) by performing at least one of token pruning and vector quantization on the input sequence (93). As illustrated in FIG. 9, the input sequence (93) can be simplified to "Eyeglass frame red."

[0153] The token reasoner (100) can simplify the input sequence (93) according to the method described above with reference to FIGS. 5 to 8, and a detailed description thereof is omitted.

[0154] The token reasoner (100) transmits a simplified input sequence (94) to the task management module (200), and the task management module (200) can perform a task according to the simplified input sequence (94).

[0155] As described above, the token reasoner (100) can analyze the input text and perform necessary transformations, thereby enabling the generation model (300) to operate more efficiently. That is, the token reasoner (100) can analyze the prompt to understand the intent and context of the prompt and perform necessary transformations. In this process, the token reasoner (100) can perform tasks such as restructuring the structure of the text, removing unnecessary information, and emphasizing important information. As a result, the prompt can be expected to become clear and concise, which can have the effect of enabling the generation model (300) to operate efficiently. In addition, the generation model (300) can provide more accurate responses by removing unnecessary information from the prompt and emphasizing important information.

[0156] 3. Execution of the generative model using the results of previous intermediate operations

[0157] When the token reasoner (100) transmits a simplified prompt (42) to the task management module (200), the task management module (200) can execute the generation module (300) according to the simplified prompt (42). If a prompt with the same or similar intent as the simplified prompt (42) has been executed previously, the task management module (200) can control the generation model (300) to perform the operation using the intermediate operation result (e.g. HSM) stored in the task database (500).

[0158] (1) Determine the operation method based on the tasks performed in the past

[0159] The task management module (200) can determine the operation method of the generation model (300) as either zero operation, partial operation, or full operation based on the tasks performed in the past. Specifically, the task management module (200) can determine the operation method as zero operation if a prompt identical to the simplified prompt (42) has been executed, determine the operation method as partial operation if a prompt different from the simplified prompt (42) but with the same intent has been executed, and determine the operation method as full operation if a prompt identical to or similar to the simplified prompt (42) has never been executed.

[0160] FIGS. 10 and 11 are diagrams illustrating methods for an electronic device according to one embodiment of the present disclosure to search a work history based on the intent of a prompt and execute a generation model using intermediate operation results of previously performed work.

[0161] When the task management module (200) receives a simplified prompt (42), it can retrieve records of previously performed tasks from the task database (500) based on the intent included in the simplified prompt (42).

[0162] Referring to FIG. 10, a first prompt (1001) and a second prompt (1002) are stored in the task database (500). According to one embodiment of the present disclosure, the task database (500) may store embedding matrices corresponding to the prompts (1001, 1002). The task management module (200) may compare the embedding matrix corresponding to the simplified prompt (42) with the embedding matrices stored in the task database (500).

[0163] Both the first prompt (1001) and the second prompt (1002) have the intent of requesting weather information, which is identical to the intent contained in the simplified prompt (42) received from the token reasoner (100). However, the details of the first prompt (1001) (“Busan tomorrow”) and the details of the second prompt (1002) (“Jeju next week”) are different from the details of the simplified prompt (42) (“Seoul tomorrow”).

[0164] Based on the intent of the simplified prompt (42), if the previously executed prompts (1001, 1002) are identical in intent to the simplified prompt (42) but differ in details, the task management module (200) can control the generation model (300) to perform some operations using the intermediate operation results (HSM 1 or HSM 2) stored in the task database (500). That is, the task management module (200) can cache either HSM 1 or HSM 2, which are intermediate operation results matching the prompt (1001, 1002), so that the generation model (300) can use them when performing operations.

[0165] HSM 1 and HSM 2 may include an attention value matrix and an activation matrix obtained from each layer of the generation model (300) when executing the first prompt (1001) and the second prompt (1002), respectively.

[0166] Referring to FIG. 11, a third prompt (1003) is stored in the work database (500). An embedding matrix corresponding to the third prompt (1003) may be stored in the work database (500). The work management module (200) may compare the embedding matrix corresponding to the simplified prompt (42) with the embedding matrices stored in the work database (500).

[0167] The third prompt (1003) has the intent of requesting weather information, which is identical to the intent contained in the simplified prompt (42) received from the token reasoner (100). Furthermore, the details of the third prompt (1003) (“Seoul tomorrow”) are identical to the details of the simplified prompt (42) (“Seoul tomorrow”).

[0168] Based on the intent of the simplified prompt (42), if the task database (500) is searched and a prompt (1003) having the same intent and details as the simplified prompt (42) has been previously executed, the task management module (200) may prevent the generation model (300) from performing an operation (zero operation) and execute API request 1 matched to the prompt (1003). Since the previously executed third prompt (1003) is identical to the simplified prompt (42) not only in intent but also in details, the task may be completed by directly executing API request 1 without the need to perform an operation of the generation model (300) using HSM 3 matched to the third prompt (1003).

[0169] (2) Specific method for determining the operation method (search based on code index)

[0170] According to one embodiment of the present disclosure, information about code indices corresponding to the intent of previously executed prompts and details connected to the code indices may be stored in a task database (500). Hereinafter, with reference to FIGS. 12 to 15, a method will be described in which the task management module (200) finds a code indices corresponding to the intent of a prompt and determines an operation method of the generation model (300) based on information about details connected to the code indices.

[0171] Referring to FIG. 12, the task management module (200) can check the code index (code index 3) corresponding to the intent of the simplified prompt (42) and check the details (1201, 1202) connected to the checked code index (code index 3).

[0172] According to one embodiment of the present disclosure, the task management module (200) can utilize information about the code index (code index 3) confirmed by the token reasoner (100). As described above with reference to FIGS. 5 and 6, the token reasoner (100) can confirm the code index (code index 3) corresponding to the intent of the prompt (41) in the process of simplifying the prompt (41), and the confirmed code index (code index 3) also corresponds to the intent of the simplified prompt (42).

[0173] According to one embodiment of the present disclosure, the task management module (200) may directly find a code index (code index 3) corresponding to a token sequence ("weather forecast") indicating the intent of the simplified prompt (42) using the codebook (60) illustrated in FIG. 6.

[0174] The work database (500) may store information about details (1201, 1202) linked to the code index (code index 3), and intermediate operation results (HSM 1, HSM 2) matching each of the details (1201, 1202) may be stored.

[0175] Since the code index (code index 3) corresponds to the intent of requesting weather information, HSM 1 corresponds to the intermediate operation result output from the layers of the generation model (300) when a prompt requesting information on the weather in Busan tomorrow is executed. Similarly, HSM 2 corresponds to the intermediate operation result output from the layers of the generation model (300) when a prompt requesting information on the weather in Jeju next week is executed.

[0176] The task management module (200) can determine the operation method of the generation model (300) based on the result of comparing the details ("Seoul tomorrow") included in the simplified prompt (42) with the details (1201, 1202) linked to the code index (code index 3). Since none of the details (1201, 1202) linked to the code index (code index 3) in FIG. 12 matches the details of the simplified prompt (42), the task management module (200) can determine the operation method as a partial operation and cache either HSM 1 or HSM 2 so that the generation model (300) can use it when calculating.

[0177] If a code index corresponding to the intent of the simplified prompt (42) does not exist in the task database (500), the task management module (200) can control the generation model (300) to perform a full operation using the simplified prompt (42).

[0178] Referring to FIG. 13, the task management module (200) can check the code index (code index 3) corresponding to the intent of the simplified prompt (42) and check the details (1201, 1202, 1203) connected to the confirmed code index (code index 3). The method by which the task management module (200) finds the code index (code index 3) corresponding to the intent of the simplified prompt (42) is as described above with reference to FIG. 12.

[0179] The work database (500) may store information about details (1201, 1202, 1203) linked to a code index (code index 3), and intermediate operation results (HSM 1, HSM 2, HSM 3) matched to each of the details (1201, 1202, 1203) may be stored. In addition, the work database (500) may also store API requests matched to each of the details (1201, 1202, 1203).

[0180] The task management module (200) can determine the operation method of the generation model (300) based on the result of comparing the details ("Seoul tomorrow") included in the simplified prompt (42) with the details (1201, 1202, 1203) linked to the code index (code index 3). In FIG. 13, since one of the details (1203) linked to the code index (code index 3) matches the details of the simplified prompt (42), the task management module (200) can determine the operation method as zero operation, omit the operation of the generation model (300), and execute API request 1 linked to the details (1203).

[0181] According to one embodiment of the present disclosure, intermediate operation results may be stored by matching them to code indices corresponding to the intent. Related embodiments are illustrated in FIGS. 14 and 15 .

[0182] Referring to FIG. 14, the task management module (200) can check the code index (code index 3) corresponding to the intent of the simplified prompt (42) and check the details (1201, 1202) connected to the confirmed code index (code index 3). The method by which the task management module (200) finds the code index (code index 3) corresponding to the intent of the simplified prompt (42) is as described above with reference to FIG. 12.

[0183] The work database (500) may store information about details (1201, 1202) linked to the code index (code index 3), and may store intermediate operation results (HSM 4) matching the code index (code index 3).

[0184] Since the code index (code index 3) corresponds to the intent of requesting weather information, HSM 4 corresponds to the intermediate operation results output from the layers of the generation model (300) when a prompt requesting weather information is executed. For example, both the attention value matrix generated when a prompt requesting weather information for Busan for tomorrow is executed and the attention value matrix generated when a prompt requesting weather information for Jeju Island for the next week is executed can be included in HSM 4.

[0185] The task management module (200) can determine the operation method of the generation model (300) based on the result of comparing the details ("Seoul tomorrow") included in the simplified prompt (42) with the details (1201, 1202) linked to the code index (code index 3). Since none of the details (1201, 1202) linked to the code index (code index 3) in FIG. 14 matches the details of the simplified prompt (42), the task management module (200) can determine the operation method as a partial operation and cache HSM 4 so that the generation model (300) can use it during the operation.

[0186] Referring to FIG. 15, the task management module (200) can check the code index (code index 3) corresponding to the intent of the simplified prompt (42) and check the details (1201, 1202, 1203) connected to the confirmed code index (code index 3). The method by which the task management module (200) finds the code index (code index 3) corresponding to the intent of the simplified prompt (42) is as described above with reference to FIG. 12.

[0187] The work database (500) may store information about details (1201, 1202, 1203) linked to a code index (code index 3), and may store intermediate operation results (HSM 5) matched to the code index (code index 3). In addition, the work database (500) may also store API requests matched to each of the details (1201, 1202, 1203).

[0188] The task management module (200) can determine the operation method of the generation model (300) based on the result of comparing the details ("Seoul tomorrow") included in the simplified prompt (42) with the details (1201, 1202, 1203) linked to the code index (code index 3). In FIG. 15, since one of the details (1203) linked to the code index (code index 3) matches the details of the simplified prompt (42), the task management module (200) can determine the operation method as zero operation, omit the operation of the generation model (300), and execute API request 1 linked to the details (1203).

[0189] According to one embodiment of the present disclosure, the task management module (200) may search for a code index corresponding to a simplified prompt (42) and execute an API request matching the code index. To this end, information regarding code indices corresponding to previously executed prompts may be stored in the task database (500).

[0190] Referring to FIG. 16, the task management module (200) can check the code index (code index 2) corresponding to the simplified prompt (42) and execute API request 1 matching the checked code index (code index 2).

[0191] If the code index (code index 2) corresponding to the simplified prompt (42) does not exist in the task database (500), the task management module (200) can control the generation model (300) to perform a full operation using the simplified prompt (42).

[0192] According to one embodiment, if a code index (code index 2) corresponding to the simplified prompt (42) does not exist in the task database (500), the task management module (200) may search for a code index corresponding to the intent of the simplified prompt (42).

[0193] For example, the task management module (200) can convert a token sequence ("weather forecast") representing the intent of the simplified prompt (42) into a vector Vi, find the cluster to which the vector Vi belongs in the codebook (170) illustrated in FIG. 17, and assign a code index (code index 3) to the vector Vi.

[0194] After the task management module (200) confirms the code index (code index 3) corresponding to the intent of the simplified prompt (42), it can determine the operation method of the generation model (300) according to the method described above with reference to FIGS. 12 to 15.

[0195] (3) The generation model performs the operation according to the determined operation method.

[0196] Hereinafter, with reference to FIGS. 18 to 20, a process in which a generation model (300) performs an operation according to a determined operation method will be described. The embedding matrix (1800) illustrated in FIGS. 18 to 20 is a matrix obtained by performing an embedding transformation on the simplified prompt (42) of FIG. 4. In addition, HSM 1 or HSM 2 used in the operation process of the decoder (320) in FIGS. 19 and 20 corresponds to an intermediate operation result corresponding to the intention of the simplified prompt (42) as described in FIG. 10 or 12.

[0197] 1) Complete operation

[0198] Figure 18 illustrates a situation in which the generation model (300) performs a full operation. Referring to Figure 18, the task management module (200) can input an embedding matrix (1800) corresponding to a simplified prompt (42) into the encoder (310) of the generation model (300). Here, d corresponds to the embedding dimension (embedding size).

[0199] The layers included in the generation model (300) (e.g., multi-head attention layer, feedforward layer, etc.) can sequentially perform operations on the input embedding matrix (1800). The prompt (41) input by the user (1) into the electronic device (1000) includes a total of 10 tokens, but the prompt (42) simplified by the token reasoner (100) includes 4 tokens. Therefore, the size of the embedding matrix (1800) is reduced, and the amount of computation of the generation model (300) can be expected to be reduced. In addition, the amount of data transferred between memories during the computation can also be expected to be reduced. In particular, as the embedding dimension increases, the complex meaning and context of the text can be grasped, so the d value is often set large. As the d value increases, the effect of reducing the amount of computation due to the reduction in the number of tokens can be more clearly seen.

[0200] In this way, according to one embodiment of the present disclosure, even if the generation model (300) performs a full operation because a similar prompt has not been executed before, the amount of computation and the amount of data to be transmitted can be expected to be reduced.

[0201] 2) Some operations (performing operations using the results of previous intermediate operations)

[0202] Figures 19 and 20 illustrate a situation in which the generation model (300) performs some operations.

[0203] 2-1) Omit encoder operation and use pre-stored HSM for decoder operation.

[0204] Referring to FIG. 19, the task management module (200) can control the decoder (320) to perform the operation by omitting the operation of the encoder (310) and using the intermediate operation result (HSM 1 or HSM 2) stored in the task database (500). As described above with reference to FIG. 10 or FIG. 12, HSM 1 or HSM 2 can be an intermediate operation result stored in the process of executing a prompt having the same intention as the simplified prompt (42).

[0205] The reason for omitting the operation of the encoder (310) is as follows.

[0206] In the transformer-type generative model (300), the encoder (310) analyzes the context and meaning of the prompt to determine the user's intention contained in the prompt, and the decoder (320) performs an operation for generating output according to the determined user's intention. That is, the role of the encoder (310) in the generative model (300) is to determine the intention contained in the prompt and transmit it to the decoder (320).

[0207] In the embodiment of the present disclosure, the token reasoner (100) performs vector quantization in the process of simplifying the prompt and already identifies the intent included in the prompt, and the decoder (320) performs the operation using the intermediate operation result (HSM 1 or HSM 2) corresponding to the identified intent, so that the intent of the prompt can be sufficiently reflected in the output of the generation model (300).

[0208] Especially in the case of HSMs such as Attention Value Matrix, if the intent of the previously executed prompt and the newly entered prompt are the same, the desired result can be obtained by using the HSM generated during the process of executing the previously executed prompt during the process of executing the newly entered prompt, even if there are some differences in the details.

[0209] For example, if an HSM created in the process of executing a previous prompt "Draw a puppy" is saved and a new prompt "Draw a cat" is input, the previously saved HSM can be used in the process of executing the newly input prompt because the intent of both prompts is the same, "Draw."

[0210] Accordingly, the task management module (200) can omit the operation of the encoder (310) and allow the intent of the prompt to be reflected in the operation of the decoder (320) through the intermediate operation result (HSM 1 or HSM 2).

[0211] According to one embodiment of the present disclosure, the task management module (200) can control to omit operations of some layers among the plurality of layers included in the decoder (320). The decoder (320) can perform operations of the next layer using the matrix included in HSM 1 or HSM 2 as the output of the layer whose operations are omitted.

[0212] For example, the generation model (300) can omit the operation of a specific multi-head attention layer included in the decoder (320) and retrieve the attention value matrix corresponding to the layer from the cache memory and use it as the input of the next layer. To this end, the task management module (200) can retrieve HSM 1 or HSM 2 from the task database (500) stored in the flash memory and store it in the cache memory. In this way, by caching the attention value matrix and performing the operation of the decoder (320), the time-consuming process of generating the attention value matrix can be omitted, thereby effectively reducing the time.

[0213] According to one embodiment, the generation model (300) may perform operations while ignoring less influential tokens by referencing an activation matrix fetched from cache memory.

[0214] When the generation model (300) performs some operations, the simplified prompt (42) differs in details ("Seoul", "tomorrow") compared to previously executed prompts, and thus the differences need to be reflected in the operations. Accordingly, the generation model (300) can perform the operations of the decoder (320) by reflecting the portion corresponding to the details ("Seoul", "tomorrow") in the embedding matrix (1800).

[0215] According to one embodiment of the present disclosure, the generation model (300) can change some elements of the HSM used in the operation of the decoder (320) based on the embedding matrix (1800). For example, if HSM 1 is an attention value matrix generated in the process of executing the prompt "Tell me weather Busan tomorrow" as illustrated in FIG. 10, the generation model (300) can change the values ​​of elements included in portions corresponding to tokens indicating details in the attention value matrix (rows corresponding to "Busan" and rows corresponding to "tomorrow"). At this time, the generation model (300) can change the values ​​of elements of the HSM based on the values ​​of elements included in portions corresponding to tokens indicating details in the embedding matrix (rows corresponding to "Seoul" and rows corresponding to "tomorrow").

[0216] According to one embodiment of the present disclosure, the generation model (300) can infer and change new values ​​of elements of an HSM based on history data of previously executing prompts. If there is history data containing HSMs generated when the generation model (300) previously executed prompts containing various intentions and details, the generation model (300) can utilize the history data to infer how the values ​​of some elements of the HSM will change as the values ​​of some elements of the embedding matrix (1800) change.

[0217] According to one embodiment of the present disclosure, the task management module (200) may control to omit operations of only some layers among the layers included in the encoder (310) rather than omit operations of the entire encoder (310). Even when operations of some layers of the encoder (310) are omitted, similarly to the case where operations are omitted in the decoder (320), the HSM corresponding to the intent of the prompt may be cached and used as the output of the layer whose operations are omitted.

[0218] According to one embodiment of the present disclosure, the work management module (200) may control the HSM to be cached and used by omitting operations only in a specific type of layer (e.g., multi-head attention layer) among the layers included in the encoder (310) or decoder (320).

[0219] 2-2) Perform encoder operations and use pre-stored HSM for decoder operations.

[0220] Referring to FIG. 20, the task management module (200) may control the encoder (310) to perform a calculation on the embedding matrix (1800) and the decoder (320) to perform a calculation using an intermediate calculation result (HSM 1 or HSM 2) stored in the task database (500). As described above with reference to FIG. 10 or FIG. 12, HSM 1 or HSM 2 may be an intermediate calculation result stored in the process of executing a prompt having the same intention as the simplified prompt (42).

[0221] The output of the encoder (310) with the embedding matrix (1800) as input may be a high-dimensional vector containing contextual information for each token included in the prompt, and the output of the encoder (310) may be reflected in operations in some layers of the decoder (320).

[0222] Therefore, according to one embodiment of the present disclosure, the generation model (300) can reduce the amount of computation by performing all operations of the encoder (310) on the embedding matrix (1800) and using the pre-stored HSM when performing the operations of the decoder (320). Of course, the generation model (300) can reduce the amount of computation by using the pre-stored HSM for some layers even in the process of performing the operations of the encoder (310) on the embedding matrix (1800).

[0223] For example, the generation model (300) may perform operations in most layers according to the complete operation method described above with reference to FIG. 18, but may omit operations for some layers of the encoder (310) or decoder (320) and use HSM 1 or HSM 2 stored in the cache memory as the output of the layer for which the operation was omitted.

[0224] According to one embodiment of the present disclosure, when performing an operation of an encoder (310) or a decoder (320) using an HSM corresponding to a previously executed prompt, the operation may be performed by reflecting a portion corresponding to details (“Seoul”, “tomorrow”) in the embedding matrix (1800) as described above with reference to FIG. 19.

[0225] 3) Zero operation

[0226] If a simplified prompt (42) and a prompt with the same intent and details have been executed previously (see FIGS. 11 and 13), the task management module (200) can control the execution of a pre-stored API request, omitting the operation of the generation model (300).

[0227] 4. Describe the process by referring to the flowcharts.

[0228] Hereinafter, with reference to the flowcharts of FIGS. 21 to 28, a method for improving the processing efficiency of a generation model according to embodiments of the present disclosure will be described. The steps included in the flowcharts of FIGS. 21 to 28 are performed by the electronic device (1000) of FIGS. 1 and 2 , and therefore, the contents previously described with reference to FIGS. 1 to 20 may be equally applied to FIGS. 21 to 28 even if omitted below.

[0229] Referring to FIG. 21, in step 2101, the electronic device (1000) may obtain a prompt from a user and simplify the obtained prompt. The simplified prompt may also include intent and details. The electronic device (1000) may first reduce the number of tokens by performing token pruning on the prompt, and then secondarily reduce the number of tokens by performing vector quantization on a token sequence representing the intent. According to one embodiment, the electronic device (1000) may also reduce the number of tokens by performing vector quantization on a vector corresponding to the entire prompt. Each method will be described with reference to FIGS. 22 and 23.

[0230] Figure 22 is a flowchart for explaining the detailed steps included in step 2101 of Figure 21.

[0231] In step 2201, the electronic device (1000) can extract key tokens from among multiple tokens included in a prompt. The electronic device (1000) can evaluate the importance of each token using an attention mechanism or based on the frequency with which the token was included in previously executed prompts, and can classify tokens with an importance higher than a certain standard as key tokens.

[0232] In step 2202, the electronic device (1000) can reduce the number of tokens by performing vector quantization on a set of tokens representing the intent among the extracted key tokens. For example, the electronic device (1000) can convert a token sequence consisting of tokens representing the intent of the prompt into a vector and perform quantization using a pre-stored codebook, thereby outputting a token sequence having the same intent but including a smaller number of tokens. To this end, a codebook in which multiple token sequences are clustered by intent can be pre-stored.

[0233] The specific method by which the electronic device (1000) performs vector quantization is as described above with reference to FIG. 6.

[0234] Figure 23 is a flowchart for explaining the detailed steps included in step 2101 of Figure 21.

[0235] At step 2301, the electronic device (1000) can convert the prompt into a vector. At this time, the electronic device (1000) can convert the prompt into a vector corresponding to the entire prompt, rather than converting it into a vector for each token.

[0236] At step 2302, the electronic device (1000) can select a vector corresponding to the shortest prompt (minimum token vector) that is included in the same cluster as the transformed vector in the codebook for vector quantization.

[0237] In detail, the electronic device (1000) can identify the cluster to which the transformed vector belongs in the codebook, assign a code index corresponding to the cluster to the transformed vector, and map the minimum token vector included in the same cluster to the transformed vector. A specific method for the electronic device (1000) to map the minimum token vector to the vector corresponding to the prompt by performing vector quantization is as described above with reference to FIG. 7.

[0238] To this end, a codebook containing clustered vectors corresponding to previously executed prompts may be pre-stored in the electronic device (1000). Furthermore, when the electronic device (1000) executes a new prompt, the codebook may be updated according to the newly executed prompt.

[0239] At step 2303, the electronic device (1000) can replace the acquired prompt with a prompt corresponding to the selected vector. In other words, the electronic device (1000) can change the acquired prompt to the shortest prompt.

[0240] Returning to Figure 21, at step 2102, the electronic device (1000) may retrieve a record of previously performed tasks based on the intent contained in the simplified prompt. Information related to previously performed tasks may be stored in a task database.

[0241] Figure 24 is a flowchart for explaining the detailed steps included in step 2102 of Figure 21.

[0242] At step 2401, the electronic device (1000) can search for a prompt containing the same intent as the simplified prompt among the prompts stored in the task database. The electronic device (1000) can search for a prompt by comparing embedding matrices, or can search for a prompt based on a code index. A method for searching for a prompt based on a code index is described below with reference to FIG. 27. A specific method for the electronic device (1000) to search for a prompt containing the same intent as the simplified prompt is as described above with reference to FIGS. 10 to 15.

[0243] At step 2402, if a prompt having the same intent as the simplified prompt is found, the electronic device (1000) may search for a prompt containing the same details as the simplified prompt among the searched at least one prompt. In other words, if a prompt having the same intent as the simplified prompt is found, the electronic device (1000) may compare the details of the searched prompt with the details of the simplified prompt. The specific method by which the electronic device (1000) searches for a prompt containing the same details as the simplified prompt is as described above with reference to FIGS. 10 to 15 .

[0244] Figure 27 is a flowchart for explaining the detailed steps included in step 2102 of Figure 21.

[0245] In step 2701, the electronic device (1000) can obtain a codebook in which multiple token sequences are clustered by intent from the task database. In step 2702, the electronic device (1000) can identify a code index corresponding to an intent included in the simplified prompt from the codebook. In step 2703, the electronic device (1000) can compare at least one detail associated with the identified code index with the details included in the simplified prompt.

[0246] Returning to Figure 21, at step 2103, the electronic device (1000) can check whether there is a history of performing an action corresponding to the intent included in the simplified prompt. If there is no history of performing an action corresponding to the intent included in the simplified prompt, the electronic device (1000) can proceed to step 2105, where the electronic device (1000) can execute the generation model according to the simplified prompt.

[0247] However, if there is a history of a task corresponding to the intention included in the simplified prompt being performed, the electronic device (1000) can proceed to step 2104 and execute the generation model according to the simplified prompt using the intermediate operation results stored during the execution of the task. In other words, if a record of a task performed in the past corresponding to the intention of the simplified prompt is confirmed, the electronic device (1000) can execute the generation model according to the simplified prompt using the intermediate operation results corresponding to the task performed in the past. That is, the electronic device (1000) can control the generation model to perform some operations, which will be described in detail with reference to FIGS. 25, 26, and 28.

[0248] FIG. 25, FIG. 26 and FIG. 28 are flowcharts for explaining detailed steps included in step 2104 of FIG. 21.

[0249] Referring to FIG. 25, in step 2501, when a first prompt with the same intent as the simplified prompt but different details is searched for, the electronic device (1000) may acquire a hidden state matrix obtained in the process of executing the generation model according to the first prompt from the working database. For example, the electronic device (1000) may read the attention value matrix generated in the process of executing the first prompt from flash memory and store it in cache memory so that the generation model can use it when performing calculations.

[0250] At step 2502, the electronic device (1000) can reflect the details included in the simplified prompt and perform operations on the generative model using the acquired hidden state matrix. For example, the electronic device (1000) can convert the simplified prompt into an embedding matrix and change the values ​​of some elements of the hidden state matrix based on the values ​​of elements corresponding to the details in the embedding matrix.

[0251] Referring to FIG. 26, at step 2601, when a second prompt having the same intent and details as the simplified prompt is retrieved, the electronic device (1000) can obtain an API request obtained in the process of executing the generation model according to the second prompt from the task database.

[0252] At step 2602, the electronic device (1000) can skip the operation of the generation model and execute the API request.

[0253] Referring to FIG. 28, at step 2801, the electronic device (1000) can determine whether at least one detail associated with the identified code index is identical to the detail included in the simplified prompt.

[0254] If the judgment result in step 2801 is not the same, in step 2802, the electronic device (1000) can obtain a hidden state matrix corresponding to the verified code index from the working database. Subsequently, in step 2803, the electronic device (1000) can reflect the details included in the simplified prompt and perform calculations of the generation model using the obtained hidden state matrix.

[0255] If the judgment result in step 2801 is the same, in step 2804, the electronic device (1000) can obtain an API request corresponding to at least one detail linked to the verified code index from the work database. Subsequently, in step 2805, the electronic device (1000) can skip the operation of the generation model and execute the API request.

[0256] Returning to FIG. 21 again, at step 2106, the electronic device (1000) can output the execution result of the generation model.

[0257] According to the embodiments described above, the size of the embedding matrix corresponding to the prompt is reduced, and by utilizing pre-stored intermediate computational results, computations in some layers are omitted, thereby reducing the computational complexity of the generative model. Furthermore, the process of simplifying the prompts is expected to result in clearer and more concise prompts, which in turn facilitates efficient operation of the generative model.

[0258] A method for processing a generative model according to one embodiment of the present disclosure may include the steps of obtaining a prompt, simplifying the prompt into a simplified prompt including an intent and details, searching for a record of operations performed in the past based on the intent of the simplified prompt, executing the generative model according to the simplified prompt using an intermediate computation result corresponding to the operation performed in the past when a record of an operation performed in the past corresponding to the intent of the simplified prompt is confirmed, and outputting an execution result of the generative model.

[0259] According to one embodiment, the step of simplifying the prompt may include the step of extracting key tokens from among a plurality of tokens included in the prompt, and the step of reducing the number of tokens by performing vector quantization on a set of tokens representing intent from among the extracted key tokens.

[0260] In one embodiment, the step of simplifying the prompt may include the steps of converting the prompt into a vector, selecting a vector corresponding to the shortest prompt and included in the same cluster as the converted vector in a code book for vector quantization, and changing the prompt to the shortest prompt.

[0261] According to one embodiment, the task database stores information related to previously performed tasks and prompts corresponding to previously performed tasks, and the step of searching for the tasks performed in the past may include the step of searching for prompts having the same intent as the simplified prompt among the prompts stored in the task database, and the step of comparing at least one or more details of the at least one or more prompts with the details of the simplified prompt when at least one prompt having the same intent as the simplified prompt is searched.

[0262] According to one embodiment, the step of executing the generation model may include the step of, when a first prompt having different details from the simplified prompt is identified among the at least one prompt, obtaining a hidden state matrix corresponding to the first prompt from the work database, and changing the obtained hidden state matrix based on details included in the simplified prompt, and performing an operation of the generation model using the changed hidden state matrix.

[0263] According to one embodiment, the step of executing the generation model may include the step of obtaining an API request corresponding to the second prompt from the work database when a second prompt having the same details as the simplified prompt is identified among the at least one prompt, and the step of omitting the operation of the generation model and executing the API request.

[0264] According to one embodiment, the task database stores information related to previously performed tasks and prompts corresponding to previously performed tasks, and the step of retrieving the previously performed tasks may include the steps of obtaining a codebook in which a plurality of token sequences are clustered by intent from the task database, the step of identifying a code index corresponding to the intent of the simplified prompt from the codebook, and the step of comparing at least one detail associated with the identified code index with the details of the simplified prompt.

[0265] According to one embodiment, the step of executing the generation model may include the step of obtaining a hidden state matrix corresponding to the confirmed code index from the working database when at least one detail associated with the confirmed code index is different from the detail included in the simplified prompt, and the step of modifying the obtained hidden state matrix based on the detail included in the simplified prompt, and performing an operation of the generation model using the modified hidden state matrix.

[0266] In one embodiment, the step of executing the generation model may include the step of obtaining an API request corresponding to at least one detail associated with the identified code index from the job database, if at least one detail associated with the identified code index is identical to a detail included in the simplified prompt, and the step of omitting the operation of the generation model and executing the API request.

[0267] An electronic device according to one embodiment of the present disclosure includes a memory storing at least one instruction and at least one processor, and the at least one processor executes at least one instruction stored in the memory, whereby the electronic device obtains a prompt, simplifies the prompt into a simplified prompt including an intent and details, searches for a record of operations performed in the past based on the intent of the simplified prompt, and, when a record of an operation performed in the past corresponding to the intent of the simplified prompt is confirmed, executes the generation model according to the simplified prompt using an intermediate computation result corresponding to the operation performed in the past, and then outputs an execution result of the generation model.

[0268] According to one embodiment, the electronic device can reduce the number of tokens by extracting key tokens from among a plurality of tokens included in the prompt, and then performing vector quantization on a set of tokens representing intent from among the extracted key tokens, thereby simplifying the prompt.

[0269] According to one embodiment, the electronic device may simplify the prompt by converting the prompt into a vector, selecting a vector corresponding to the shortest prompt and included in the same cluster as the converted vector in a code book for vector quantization, and then replacing the prompt with the shortest prompt.

[0270] According to one embodiment, a task database stores information related to previously performed tasks and prompts corresponding to previously performed tasks, and when searching for tasks performed in the past, the electronic device searches for prompts among the prompts stored in the task database that have the same intention as the simplified prompt, and when at least one prompt having the same intention as the simplified prompt is searched, the electronic device can compare one or more details of the at least one prompt with the details of the simplified prompt.

[0271] According to one embodiment, when executing the generation model, if a first prompt having different details from the simplified prompt is identified among the at least one prompt, the electronic device may obtain a hidden state matrix corresponding to the first prompt from the task database, and then change the obtained hidden state matrix based on the details included in the simplified prompt, and perform an operation of the generation model using the changed hidden state matrix.

[0272] According to one embodiment, when executing the generation model, if a second prompt having the same details as the simplified prompt is identified among the at least one prompt, the electronic device may obtain an API request corresponding to the second prompt from the task database, skip the operation of the generation model, and execute the API request.

[0273] According to one embodiment, the task database stores information related to previously performed tasks, and the electronic device, when searching for tasks performed in the past, obtains a codebook in which a plurality of token sequences are clustered by intent from the task database, verifies a code index corresponding to the intent of the simplified prompt from the codebook, and then compares at least one detail connected to the verified code index with the details of the simplified prompt.

[0274] According to one embodiment, when executing the generation model, if at least one detail associated with the identified code index is different from the details included in the simplified prompt, the electronic device may obtain a hidden state matrix corresponding to the identified code index from the work database, and then change the obtained hidden state matrix based on the details included in the simplified prompt, and perform an operation of the generation model using the changed hidden state matrix.

[0275] According to one embodiment, when executing the generation model, the electronic device may obtain an API request corresponding to the at least one detail associated with the identified code index from the task database if the at least one detail associated with the identified code index is identical to the detail included in the simplified prompt, and then skip the operation of the generation model and execute the API request.

[0276] Various embodiments of the present disclosure may be implemented or supported by one or more computer programs, and the computer programs may be formed from computer-readable program code and embodied in a computer-readable medium. In the present disclosure, "application" and "program" may refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in computer-readable program code. "Computer-readable program code" may include various types of computer code, including source code, object code, and executable code. "Computer-readable medium" may include various types of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), a hard disk drive (HDD), a compact disc (CD), a digital video disc (DVD), or various types of memory.

[0277] Additionally, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a 'non-transitory storage medium' is a tangible device and may exclude wired, wireless, optical, or other communication links that transmit temporary electrical or other signals. Meanwhile, this 'non-transitory storage medium' does not distinguish between cases where data is permanently stored in the storage medium and cases where it is temporarily stored. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored. A computer-readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and removable and non-removable media. A computer-readable medium includes a medium on which data can be permanently stored and a medium on which data can be stored and later overwritten, such as a rewritable optical disk or an erasable memory device.

[0278] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0279] The above description of the present disclosure is for illustrative purposes only, and those skilled in the art will appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present disclosure. For example, suitable results can be achieved even if the described techniques are performed in a different order than the described method, and / or components of the systems, structures, devices, circuits, etc. described are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form.

[0280] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. In the processing method of the generation model, Steps to obtain a prompt; A step of simplifying the above prompt into a simplified prompt that includes intent and details; A step of retrieving a record of operations performed in the past based on the intent of the simplified prompt; When a record of a task performed in the past corresponding to the intention of the simplified prompt is confirmed, a step of executing the generation model according to the simplified prompt using an intermediate computation result corresponding to the task performed in the past; and A method comprising a step of outputting the execution result of the above generation model.

2. In paragraph 1, The steps to simplify the above prompt are: A step of extracting key tokens from among multiple tokens included in the above prompt; and A method characterized by including a step of reducing the number of tokens by performing vector quantization on a set of tokens indicating intent among the extracted key tokens.

3. In either of paragraphs 1 and 2, The steps to simplify the above prompt are: Step of converting the above prompt into a vector; A step of selecting a vector corresponding to the shortest prompt and included in the same cluster as the transformed vector in a code book for vector quantization; and A method characterized by comprising the step of changing the above prompt to the shortest prompt.

4. In any one of paragraphs 1 to 3, The task database stores information related to previously performed tasks and prompts corresponding to previously performed tasks. The step of searching for the tasks performed in the past is: A step of searching for prompts that contain the same intent as the simplified prompt among the prompts stored in the above task database; and A method characterized by comprising the step of comparing at least one detail of at least one prompt with the details of the simplified prompt when at least one prompt having the same intent as the simplified prompt is retrieved.

5. In any one of paragraphs 1 to 4, The steps for executing the above generation model are: If a first prompt having different details from the simplified prompt is identified among the at least one prompt, a step of obtaining a hidden state matrix corresponding to the first prompt from the task database; and A method characterized by comprising the step of modifying the obtained hidden state matrix based on the details included in the simplified prompt and performing the operation of the generation model using the modified hidden state matrix.

6. In any one of paragraphs 1 to 5, The steps for executing the above generation model are: If a second prompt having the same details as the simplified prompt is identified among the at least one prompt, a step of obtaining an API request corresponding to the second prompt from the job database; and A method characterized by including a step of omitting the operation of the above generation model and executing the API request.

7. In any one of paragraphs 1 to 6, The task database stores information related to previously performed tasks and prompts corresponding to previously performed tasks. The step of searching for the tasks performed in the past is: A step of obtaining a codebook in which a plurality of token sequences are clustered by intent from the above-mentioned work database; A step of checking a code index corresponding to the intent of the simplified prompt from the above codebook; and A method comprising the step of comparing at least one detail associated with the identified code index with the details of the simplified prompt.

8. In any one of paragraphs 1 to 7, The steps for executing the above generation model are: If at least one detail associated with the identified code index is different from the details included in the simplified prompt, obtaining a hidden state matrix corresponding to the identified code index from the working database; and A method characterized by comprising the step of modifying the obtained hidden state matrix based on the details included in the simplified prompt and performing the operation of the generation model using the modified hidden state matrix.

9. In any one of paragraphs 1 to 8, The steps for executing the above generation model are: If at least one detail associated with the above-mentioned verified code index is identical to the detail included in the above-mentioned simplified prompt, obtaining an API request corresponding to at least one detail associated with the above-mentioned verified code index from the above-mentioned work database; and A method characterized by including a step of omitting the operation of the above generation model and executing the API request.

10. In an electronic device (1000), A memory (1400) in which at least one instruction is stored; and At least one processor (1300) comprising a processing circuit, The electronic device (1000) executes the at least one instruction stored in the memory (1400) by the at least one processor (1300) alone or in cooperation. Get the prompt, Simplify the above prompt into a simplified prompt that includes intent and details, Based on the intent of the simplified prompt above, retrieve the history of operations performed in the past, If a record of a task performed in the past corresponding to the intention of the simplified prompt is confirmed, the intermediate computation result corresponding to the task performed in the past is used to execute the generation model (300) according to the simplified prompt, An electronic device that outputs the execution result of the above generation model (300).

11. In paragraph 10, The electronic device (1000) simplifies the prompt by having the at least one processor (1300) execute the at least one instruction in the memory (1400). After extracting key tokens from among the multiple tokens included in the above prompt, An electronic device characterized in that the number of tokens is reduced by performing vector quantization on a set of tokens indicating intent among the extracted key tokens.

12. In any one of paragraphs 10 and 11, The electronic device (1000) simplifies the prompt by having the at least one processor (1300) execute the at least one instruction in the memory (1400). Convert the above prompt to a vector, In the code book for vector quantization, the vector corresponding to the shortest prompt and included in the same cluster as the transformed vector is selected, An electronic device characterized in that it changes the above prompt to the shortest prompt.

13. In any one of paragraphs 10 to 12, The task database (500) stores information related to previously performed tasks and prompts corresponding to previously performed tasks. The electronic device (1000) searches for tasks performed in the past by executing the at least one instruction in the memory (1400) by the at least one processor (1300). After searching for prompts that contain the same intent as the simplified prompt among the prompts stored in the above work database (500), An electronic device characterized in that, when at least one prompt having the same intent as the simplified prompt is retrieved, one or more details of the at least one prompt are compared with the details of the simplified prompt.

14. In any one of paragraphs 10 to 13, The electronic device (1000) executes the generation model (300) by having the at least one processor (1300) execute the at least one instruction in the memory (1400). If a first prompt is identified among the above at least one prompt and the simplified prompt in detail, a hidden state matrix corresponding to the first prompt is obtained from the work database (500). An electronic device characterized in that the obtained hidden state matrix is ​​changed based on the details included in the simplified prompt, and the operation of the generation model (300) is performed using the changed hidden state matrix.

15. In any one of paragraphs 10 to 14, The electronic device (1000) executes the generation model (300) by having the at least one processor (1300) execute the at least one instruction in the memory (1400). If a second prompt having the same details as the simplified prompt is identified among at least one of the above prompts, an API request corresponding to the second prompt is obtained from the work database (500). An electronic device characterized in that it omits the operation of the above generation model (300) and executes the above API request.

Images