Method and electronic device for multisession service of language model using kv cache dump

Abstract

A multi-session service method for a language model using a KV cache dump, performed by at least one processor, is disclosed, comprising the steps of: generating an embedding vector based on natural language input during the execution of a first session; generating a first key matrix and a first value matrix corresponding to the input natural language and used in an attention mechanism based on the generated embedding vector; storing the first key matrix and the first value matrix in a cache memory; and storing the first key matrix and the first value matrix stored in the cache memory in a database in response to a change request from the first session to a second session.

Description

Technology Field

[0001] The present disclosure relates to a multi-session service method and electronic device for a language model using a KV cache dump. Background Technology

[0002] Natural language processing is a technology that enables interaction between computers and human language and is one of the core functions of artificial intelligence. This natural language processing is used in fields such as speech recognition, content summarization, translation, user sentiment analysis, text classification, question answering systems, and chatbots, and is making remarkable progress due to the rapid development of artificial intelligence and machine learning.

[0003] Furthermore, language models infer the next word with the highest probability to generate the most natural sentences. These language models achieve high performance by being trained using deep learning algorithms. The training methods of language models can be classified into those that predict the next word based on previously given words, or those that predict an intermediate word based on given words. However, both training methods consider context, allowing for the continuous reflection of the preceding and succeeding context whenever analyzing sentence tokens (words). In this context, an attention mechanism can be utilized. The attention mechanism calculates how related each word or token in the input sequence is to other words, ultimately generating an output that reflects the context. In other words, in natural language processing using language models, an attention mechanism can be employed to output natural sentences while considering the context.

[0004] Meanwhile, to enhance the efficiency of the attention mechanism in natural language processing using a language model, the key vector (or matrix) and value vector (or matrix) corresponding to the input natural language used in the attention mechanism can be cached. For example, a query vector, a key vector, and a value vector are generated for each token in an input sequence, and the key vector and value vector can be stored in a cache memory (hereinafter referred to as the KV Cache). Accordingly, when a new input is added during the inference process of the language model, the model can quickly perform predictions without redundant calculations by using the key vector and value vector stored in the KV Cache.

[0005] However, when providing natural language processing services using language models in a multi-session environment, prompts generated based on multiple natural languages ​​input during the execution of each session are saved when switching from one session to another. Since the KV cache is reconstructed by generating key vectors and value vectors using the saved prompts when returning to the original session, a delay occurs until the first token is output, which can lead to service latency. Accordingly, when providing natural language processing services using language models in a multi-session environment, there is a need for technological development regarding the configuration of the KV cache in response to session switching. The problem to be solved

[0006] The present disclosure provides a multi-session service method and an electronic device for a language model using a KV cache dump to solve the above-mentioned problems. means of solving the problem

[0007] The present disclosure may be implemented in various ways, including methods, devices (systems), and / or computer programs stored on computer-readable storage media.

[0008] According to one embodiment of the present disclosure, a multi-session service method for a language model using a KV cache dump, performed by at least one processor, may include the steps of: generating an embedding vector based on natural language input during the execution of a first session; generating a first key matrix and a first value matrix corresponding to the input natural language and used in an attention mechanism based on the generated embedding vector; storing the first key matrix and the first value matrix in a cache memory; and storing the first key matrix and the first value matrix stored in the cache memory in a database in response to a change request from the first session to a second session.

[0009] According to one embodiment, the step of storing in a database may include the step of mapping the first key matrix and the first value matrix to information about the first session and storing them in the database.

[0010] According to one embodiment, a multi-session service method for a language model using a KV cache dump may further include the step of identifying whether a second key matrix and a second value matrix associated with a second session are stored in a database, and if the second key matrix and the second value matrix are stored in the database, the step of storing the second key matrix and the second value matrix stored in the database in a cache memory.

[0011] According to one embodiment, a request for a change from a first session to a second session may be received based on at least one of a specified time period, user input, user change, or the result of analyzing the input natural language.

[0012] According to one embodiment, a request for a change from a first session to a second session may be received in response to the detection of a user change or a change in a category associated with the input natural language, based on the analysis results of the input natural language.

[0013] According to one embodiment, a multi-session service method for a language model using a KV cache dump may further include the steps of summarizing a prompt generated based on a plurality of natural languages ​​input during the execution of a first session, generating a second key matrix and a second value matrix based on the summarized prompt, and storing the second key matrix and the second value matrix in a database in response to a change request from the first session to the second session.

[0014] According to one embodiment, a multi-session service method for a language model using a KV cache dump may further include the step of deleting a first key matrix and a first value matrix stored in a database based on at least one of the deletion of a first session or user input.

[0015] According to one embodiment, the database may be included in at least one of a server providing a service using a language model or a user terminal receiving natural language input.

[0016] A computer program stored on a computer-readable recording medium may be provided to execute a method according to one embodiment of the present disclosure on a computer.

[0017] According to one embodiment of the present disclosure, an electronic device comprises a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, and the at least one program may include instructions for generating an embedding vector based on natural language input during the execution of a first session, generating a first key matrix and a first value matrix corresponding to the input natural language used in an attention mechanism based on the generated embedding vector, storing the first key matrix and the first value matrix in a cache memory, and storing the first key matrix and the first value matrix stored in the cache memory in response to a change request from the first session to a second session in a database. Effects of the invention

[0018] According to some embodiments of the present disclosure, by dumping the key matrix and value matrix upon session switching and restoring the dumped key matrix and value matrix to the cache memory upon session return, the time required to calculate the key matrix and value matrix can be reduced, thereby preventing service delays.

[0019] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing

[0020] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a diagram illustrating an exemplary electronic device for providing a multi-session service of a language model using a KV cache dump according to one embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to communicate with a plurality of user terminals in relation to data processing according to one embodiment of the present disclosure. FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating the configuration of an electronic device that provides a multi-session service of a language model using a KV cache dump according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating a method for processing a key matrix and a value matrix in response to a session change request according to one embodiment of the present disclosure. FIG. 6 is a diagram illustrating another method for processing a key matrix and a value matrix in response to a session change request according to one embodiment of the present disclosure. FIG. 7 is a diagram illustrating a multi-session service method for a language model using a KV cache dump according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating a method for restoring a key matrix and a value matrix in response to a session change request according to one embodiment of the present disclosure. FIG. 9 is a diagram illustrating a method for processing a key matrix and a value matrix generated based on a summarized prompt according to one embodiment of the present disclosure. Specific details for implementing the invention

[0021] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0022] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0023] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0024] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0025] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0026] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0027] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Marked Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0028] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.

[0029] Additionally, in the following embodiments, where it is stated that one component is 'connected', 'coupled', or 'joined' to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be 'connected', 'coupled', or 'joined' between each component.

[0030] Additionally, the terms 'comprises' and / or 'comprising' as used in the following embodiments do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0031] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0032] FIG. 1 is a diagram illustrating an exemplary electronic device (100) for providing a multi-session service of a language model (102) using a KV cache dump according to one embodiment of the present disclosure. Referring to FIG. 1, the electronic device (100) can process input natural language (110) using the language model (102). For example, the electronic device (100) can provide services such as speech recognition, translation, and question answer based on the input natural language (110) using the language model (102). Here, the language model (102) may include a Large Language Model (LLM), a Small Language Model (SLM), or a Large Multimodal Model (LMM).

[0033] In relation to natural language (110) processing using a language model (102), an electronic device (100) can generate an embedding vector based on the input natural language (110). Here, the embedding vector may be a method of representing data as a fixed-dimensional real vector or a result obtained through this. When there is high semantic similarity between the data, the embedding vectors corresponding to the data may be located adjacent to each other in the space in which they are represented.

[0034] Then, the electronic device (100) can generate a key matrix and a value matrix corresponding to the natural language (110) used in the attention mechanism and input based on the generated embedding vector. For example, the electronic device (100) can generate a query vector, a key vector, and a value vector based on the embedding vector generated based on the natural language (110). Here, the query is a representation of information that the language model (102) is interested in regarding specific input data, and can serve as a criterion used to evaluate the importance of each token in the input sequence. Additionally, the key is a representation in which the language model (102) stores information about a specific token in the input sequence, and can be used to determine which token is important by calculating the similarity between the query and the key. Additionally, the value is a representation in which the language model (102) contains actual information about the input data, and information can be selected and conveyed according to the importance calculated using the query and the key, and the value corresponding to the key with high importance can be emphasized more to contribute to the final output.

[0035] Then, the electronic device (100) can store the key matrix and value matrix in the cache memory (120). For example, to increase the efficiency of the attention mechanism in natural language (110) processing using the language model (102), the key matrix and value matrix corresponding to the input natural language (110) used in the attention mechanism can be cached. Here, the cache memory (120) in which the key matrix and value matrix are cached may be referred to as the KV cache. Accordingly, when a new input is added during the inference process of the language model (102), the language model (102) can quickly perform a prediction without redundant calculation by using the key vector and value vector stored in the KV cache.

[0036] For example, the process of processing the natural language (110) described above, from the process of acquiring the natural language (110) to the process of storing the key matrix and value matrix corresponding to the natural language (110) in the cache memory (120), can be performed on a session-by-session basis. For example, when providing a natural language (110) processing service using a language model (102) in multiple sessions, the electronic device (100) can generate an embedding vector based on the natural language (110) input during the execution of each session, generate a key matrix and value matrix corresponding to the natural language (110) used in the attention mechanism based on the generated embedding vector, and store the generated key matrix and value matrix in the cache memory (120).

[0037] Then, when the session is switched, the electronic device (100) can store the key matrix and value matrix stored in the cache memory (120) in the database (130). For example, the electronic device (100) can dump the first key matrix and the first value matrix cached in association with the first session into the database (130) in response to a change request from the first session to the second session. At this time, the electronic device (100) identifies whether the second key matrix and the second value matrix associated with the second session are stored in the database (130), and if the second key matrix and the second value matrix are stored in the database (130), the second key matrix and the second value matrix stored in the database (130) can be stored in the cache memory (120).

[0038] Then, when the session is restored, the electronic device (100) can store the key matrix and value matrix stored in the database (130) in the cache memory (120). For example, the electronic device (100) can identify whether the first key matrix and first value matrix associated with the first session are stored in the database (130) in response to a change request from another session to the first session, and if the first key matrix and first value matrix are stored in the database (130), it can restore the first key matrix and first value matrix stored in the database (130) to the cache memory (120).

[0039] As described above, when providing a natural language (110) processing service using a language model (102) in a multi-session, when switching sessions, the key matrix and value matrix can be dumped into a database (130), and when returning to a session, the dumped key matrix and value matrix can be restored to a cache memory (120). Accordingly, the electronic device (100) can reduce the time required to calculate the key matrix and value matrix, thereby preventing delays in the natural language (110) processing service using the language model (102) in a multi-session.

[0040] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to communicate with a plurality of user terminals (210_1, 210_2, 210_3) in relation to data processing according to one embodiment of the present disclosure. The information processing system (230) may include system(s) capable of providing a data processing service (e.g., a multi-session service of a language model using a KV cache dump). In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the data processing service, or one or more distributed computing devices and / or distributed databases based on cloud computing services. For example, the information processing system (230) may include separate systems (e.g., servers) for the data processing service.

[0041] Data processing services, etc. provided by the information processing system (230) can be provided to the user through a data processing application, a web browser application, etc. installed on each of the multiple user terminals (210_1, 210_2, 210_3).

[0042] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication devices and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0043] For example, multiple user terminals (210_1, 210_2, 210_3) can transmit a data processing request and a command associated with a user request for data processing to an information processing system (230) through a network (220), and the information processing system (230) can receive this.

[0044] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto. The user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running data processing applications. For example, user terminals may include smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0045] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of executing data processing applications, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through the network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0046] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a non-perishable permanent mass storage device such as ROM (read-only memory), a disk drive, an SSD (solid-state drive), or a flash memory. As another example, a non-perishable permanent mass storage device such as ROM, an SSD, a flash memory, or a disk drive may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, the memory (312, 332) may store an operating system and at least one program code (e.g., code for an application associated with a data processing service).

[0047] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module (316, 336) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program (e.g., an application associated with a data processing service, etc.) that is installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0048] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0049] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data (e.g., a data processing request or data, etc.) generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) through the communication module (336) and the network (220) via the communication module (316) of the user terminal (210).

[0050] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. In FIG. 3, the input / output device (320) is not shown as being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is shown as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0051] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. However, it is not necessary to clearly illustrate most of the conventional technical components. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to include various components such as an accelerometer, a gyroscope, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0052] According to one embodiment, the processor (314) of the user terminal (210) may be configured to operate a data processing application or a web browser application that provides data processing services. At this time, program code associated with the application may be loaded into the memory (312) of the user terminal (210). While the application is operating, the processor (314) of the user terminal (210) may receive information and / or data provided from an input / output device (320) through an input / output interface (318) or receive information and / or data from an information processing system (230) through a communication module (316), and may process the received information and / or data and store it in the memory (312). Additionally, such information and / or data may be provided to the information processing system (230) through the communication module (316).

[0053] While the data processing application is in operation, the processor (314) may receive voice data, text, images, videos, etc., that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc., connected to the input / output interface (318), and may store the received voice data, text, images and / or videos, etc. in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220). In one embodiment, the processor (314) may receive user input input through an input device and provide data / requests corresponding to the received user input to an information processing system (230) through a network (220) and a communication module (316).

[0054] The processor (314) of the user terminal (210) can transmit information and / or data to an input / output device (320) through an input / output interface (318) and output it. For example, the processor (314) of the user terminal (210) can output the processed information and / or data through an output device (320), such as a display output device (e.g., touch screen, display, etc.) or a voice output device (e.g., speaker).

[0055] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0056] FIG. 4 is a diagram illustrating the configuration of an electronic device for providing a multi-session service of a language model using a KV cache dump according to one embodiment of the present disclosure. Referring to FIG. 4, an electronic device (400) (e.g., the electronic device (100) of FIG. 1) for providing a multi-session service of a language model using a KV cache dump (e.g., the language model (102) of FIG. 1) may include a processor (410), a memory (420), and a communication circuit (430). However, the configuration of the electronic device (400) is not limited thereto. According to various embodiments, the electronic device (400) may omit at least one of the components described above and may include at least one additional component. For example, the electronic device (400) may further include a display for displaying the processing results of natural language.

[0057] The processor (410) can execute software (or a program) to control at least one other component (e.g., a hardware or software component) of the electronic device (400) connected to the processor (410) and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (410) can load commands or data received from another component (e.g., a communication circuit (430)) into volatile memory, process the commands or data stored in volatile memory, and store the resulting data in non-volatile memory.

[0058] The memory (420) may store various data used by at least one component of the electronic device (400) (e.g., processor (410)). The data may include, for example, input data or output data for software (or programs) and related instructions. The memory (420) may include volatile memory or non-volatile memory. According to one embodiment, the memory (420) may include at least one of a cache memory (e.g., cache memory (120) of FIG. 1) or a database (e.g., database (130) of FIG. 1) for storing a key matrix and a value matrix. Additionally, the memory (420) may include a language model.

[0059] The memory (420) may include at least one instruction related to a multi-session service of a language model using a KV cache dump. The at least one instruction may include, for example, instructions related to natural language acquisition, natural language processing, session management, and service provision. For example, the memory (420) may include a natural language acquisition module (422), a natural language processing module (424), a session management module (426), and a service provision module (428). However, the types of modules included in the memory (420) are classified according to the function of the instructions, and the types and number thereof are not limited thereto. Additionally, the modules included in the memory (420) (or instructions included in the modules) are executed by the processor (410) and may be implemented by the processor (410) itself.

[0060] The natural language acquisition module (422) can acquire natural language. For example, the natural language acquisition module (422) can acquire natural language or a prompt generated based on natural language from an external electronic device through a communication circuit (430). For another example, the natural language acquisition module (422) can acquire natural language from a user through a user interface. The user interface may include, for example, a microphone for receiving the user's voice, a keyboard for receiving text input from the user, a touchscreen, user input via a network, etc. If the user interface includes a microphone, the natural language acquisition module (422) can convert the user's voice received through the microphone into text through a Speech-To-Text (STT) function.

[0061] The natural language processing module (424) can process the acquired natural language. For example, the natural language processing module (424) can analyze the input natural language text and understand the meaning of the text. In addition, the natural language processing module (424) can generate natural text from the input natural language text.

[0062] According to one embodiment, the natural language processing module (424) can generate an embedding vector based on the input natural language. Additionally, the natural language processing module (424) can generate a key matrix and a value matrix corresponding to the input natural language and used in the attention mechanism based on the generated embedding vector. Additionally, the natural language processing module (424) can store the key matrix and the value matrix in a cache memory.

[0063] According to one embodiment, when the natural language processing module (424) operates in multiple sessions, it can process natural language input per session. For example, the natural language processing module (424) can generate an embedding vector based on natural language input during the execution of each session, generate a key matrix and a value matrix corresponding to the input natural language used in the attention mechanism based on the generated embedding vector, and store the generated key matrix and value matrix in a cache memory.

[0064] The session management module (426) can manage sessions in multiple sessions. Sessions are intended to maintain the interaction state between a user and an electronic device (400) for a certain period and can be created per user or per category. For example, if a first user inputs natural language related to a topic belonging to a first category, the session management module (426) can create a first session. Additionally, if the first user inputs natural language related to a topic belonging to a second category different from the first category, the session management module (426) can create a second session different from the first session. Additionally, if a second user other than the first user inputs natural language, the session management module (426) can create a third session different from the first and second sessions.

[0065] The session management module (426) determines the session transition conditions and can transition the session according to the determined transition conditions. The session transition conditions may include automatic transition conditions and manual transition conditions. Automatic transition conditions are, for example, conditions under which a session can be automatically transitioned without receiving an explicit request for session transition from a user, and may include at least one of a specified time period, a change in user, or a change in category. Additionally, manual transition conditions are, for example, conditions under which a session can be manually transitioned by receiving an explicit request for session transition from a user, and may include user input.

[0066] Regarding the conditions for automatic session switching, the session management module (426) can switch sessions at specified time intervals. Additionally, the session management module (426) can switch sessions when the user changes. According to one embodiment, the session management module (426) can determine whether the user has changed through the user identification process (e.g., login process) of the electronic device (400). According to another embodiment, the session management module (426) can determine whether the user has changed based on the analysis results of input natural language. For example, the type or pattern of natural language input by user may differ, and the session management module (426) can detect user changes by analyzing this. Additionally, the session management module (426) can switch sessions when the category changes. According to one embodiment, the session management module (426) can determine whether the category has changed based on the analysis results of input natural language. For example, the session management module (426) can identify the category to which the topic associated with the input natural language belongs based on the analysis of the input natural language, and can detect a change in the category based on the identified category.

[0067] Regarding the conditions for manual session switching, the session management module (426) can switch sessions based on user input. For example, if the user selects a button, page, etc. related to session switching or category change, the session management module (426) can switch sessions.

[0068] According to one embodiment, the session management module (426) can terminate the session based on user input. According to another embodiment, the session management module (426) can terminate the session if subsequent natural language is not entered for a specified period of time after natural language has been entered.

[0069] According to one embodiment, the session management module (426) may store the key matrix and value matrix stored in the cache memory in the database when the session is switched. For example, the session management module (426) may store the first key matrix and the first value matrix stored in the cache memory associated with the first session in the database in response to a request to change from the first session to the second session. At this time, the session management module (426) identifies whether the second key matrix and the second value matrix associated with the second session are stored in the database, and if the second key matrix and the second value matrix are stored in the database, the second key matrix and the second value matrix stored in the database may be stored in the cache memory.

[0070] According to one embodiment, the session management module (426) may store the key matrix and value matrix stored in the database in the cache memory when the session is restored. For example, the session management module (426) may identify whether the first key matrix and the first value matrix associated with the first session are stored in the database in response to a change request from the second session to the first session, and if the first key matrix and the first value matrix are stored in the database, the first key matrix and the first value matrix stored in the database may be stored in the cache memory.

[0071] According to one embodiment, the session management module (426) can store the key matrix and value matrix in a database by mapping them to information about the associated session. For example, the first key matrix and the first value matrix associated with the first session can be stored in the database by mapping them to information about the first session, and the second key matrix and the second value matrix associated with the second session, which is different from the first session, can be stored in the database by mapping them to information about the second session.

[0072] According to one embodiment, the session management module (426) can delete the key matrix and value matrix associated with the deleted (or terminated) session stored in the database based on at least one of the deletion (or termination) of the session or user input.

[0073] According to one embodiment, the database may be included in at least one of a server providing a service using a language model or a user terminal receiving natural language input. If the database is included in a server providing a service using a language model (e.g., within the service provider's system), the service content may always remain in the system because KV cache information (e.g., cached key matrix and value matrix) regarding user prompts and responses using the language model is managed directly in the service provider's system. However, if the database is included in a user terminal receiving natural language input, the service content may remain in the system only during the service. For example, when a user uses a service using a language model, the KV cache information stored in the user terminal is transmitted to the server, and the user terminal can update the KV cache stored in the user terminal by receiving the updated KV cache information from the server. However, upon termination of the service, the server may delete the KV cache information, so the service content may remain in the system only while the user is using the service. In this case, the user terminal may maintain the storage state of the KV cache information until the user directly deletes it.

[0074] According to one embodiment, the natural language processing module (424) may summarize the prompts and generate key matrices and value matrices based on the summarized prompts. Then, the session management module (426) may store the key matrices and value matrices generated based on the summarized prompts in a database in response to a request to switch sessions. For example, the natural language processing module (424) may summarize the prompts generated based on a plurality of natural languages ​​input during the execution of the first session. Additionally, the natural language processing module (424) may regenerate the key matrices and value matrices associated with the first session based on the summarized prompts. Then, the session management module (426) may store the regenerated key matrices and value matrices associated with the first session in a database in response to a request to change from the first session to the second session.

[0075] The service provision module (428) can provide services using a language model. For example, the service provision module (428) can provide services such as speech recognition, translation, and question answer based on input natural language using a language model. Additionally, when the service provision module (428) provides services using a language model in a multi-session, it can provide services for the currently active session.

[0076] The communication circuit (430) (or communication module) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (400) and an external electronic device, and the performance of communication through the established communication channel. According to one embodiment, the electronic device (400) can receive a natural language or a prompt generated based on natural language from an external electronic device through the communication circuit (430).

[0077] FIG. 5 is a diagram illustrating a method for processing a key matrix and a value matrix in response to a request for a change of session according to an embodiment of the present disclosure. Referring to FIG. 5, a processor (e.g., processor (410) of FIG. 4) of an electronic device (e.g., electronic device (100) of FIG. 1 or electronic device (400) of FIG. 4) for providing a multi-session service of a language model (520) (e.g., language model (102) of FIG. 1) using a KV cache dump can switch sessions when a change of category is detected. For example, based on the analysis result of natural language input by a user of a user terminal (510), if the processor determines that the user inputs natural language associated with a topic belonging to a first category (512) and then inputs natural language associated with a topic belonging to a second category (514), the processor can change from a first session associated with the first category (512) to a second session associated with the second category (514). As another example, the processor may change from the first session associated with the first category (512) to the second session associated with the second category (514) if it determines, based on the input of the user of the user terminal (510), that the user has selected a button or page related to a session transition from the first session to the second session or a change from the first category (512) to the second category (514). Of course, the session change may be applied in the same or similar way not only to the change from the first session to the second session, but also to the change from the first session to the third session associated with the third category (516), the change from the second session to the first session or the third session, and the change from the third session to the first session or the second session. Additionally, the user terminal (510) of the user who wishes to use the service may be provided separately from the electronic device that provides the multi-session service of the language model using the KV cache dump, or it may be provided as a single integrated electronic device.

[0078] When a session is switched, the processor can store the key matrix and value matrix stored in the cache memory (530) in the database (540). For example, in response to a change request from the first session to the second session, the processor can store the first key matrix and the first value matrix stored in the cache memory (530) associated with the first session in the database (540). At this time, the processor identifies whether the second key matrix and the second value matrix associated with the second session are stored in the database (540), and if the second key matrix and the second value matrix are stored in the database (540), the processor can store the second key matrix and the second value matrix stored in the database (540) in the cache memory (530).

[0079] When the session is restored, the processor can store the key matrix and value matrix stored in the database (540) in the cache memory (530). For example, in response to a change request from the second session to the first session, the processor identifies whether the first key matrix and first value matrix associated with the first session are stored in the database (540), and if the first key matrix and first value matrix are stored in the database (540), the processor can store the first key matrix and first value matrix stored in the database (540) in the cache memory (530).

[0080] FIG. 6 is a diagram illustrating another method of processing a key matrix and a value matrix in response to a request for a change of session according to one embodiment of the present disclosure. Referring to FIG. 6, a processor (e.g., processor (410) of FIG. 4) of an electronic device (e.g., electronic device (100) of FIG. 1 or electronic device (400) of FIG. 4) for providing a multi-session service of a language model (620) (e.g., language model (102) of FIG. 1) using a KV cache dump can switch sessions when a change of user is detected. For example, if the processor determines through a user identification process (e.g., a login process) that a user connected to an electronic device providing a service using a language model has changed from a first user (user of the first user terminal (612)) to a second user (user of the second user terminal (614)), it can change from a first session associated with the first user to a second session associated with the second user. As another example, if the processor determines, based on the analysis results of the input natural language, that the user has changed from the first user to the second user, it may change from the first session associated with the first user to the second session associated with the second user. Of course, the session change may be applied in the same or similar way not only to the change from the first session to the second session, but also to the change from the first session to the third session associated with the third user (user of the third user terminal (616)), the change from the second session to the first session or the third session, and the change from the third session to the first session or the second session. Additionally, although FIG. 6 describes a situation in which multiple users each access an electronic device using different user terminals (612, 614, 616), it is not limited thereto, and even in a situation where multiple users share a single user terminal, the determination of whether a user has changed may be made in the same or similar way as described above.In addition, the user terminal (612, 614, 616) of a user who wishes to use the service may be provided separately from the electronic device that provides the multi-session service of the language model using the KV cache dump, or it may be provided as a single integrated electronic device.

[0081] When a session is switched, the processor can store the key matrix and value matrix stored in the cache memory (630) in the database (640). For example, in response to a change request from the first session to the second session, the processor can store the first key matrix and the first value matrix stored in the cache memory (630) associated with the first session in the database (640). At this time, the processor identifies whether the second key matrix and the second value matrix associated with the second session are stored in the database (640), and if the second key matrix and the second value matrix are stored in the database (640), the processor can store the second key matrix and the second value matrix stored in the database (640) in the cache memory (630).

[0082] When the session is restored, the processor may store the key matrix and value matrix stored in the database (640) in the cache memory (630). For example, in response to a change request from the second session to the first session, the processor may identify whether the first key matrix and first value matrix associated with the first session are stored in the database (640), and if the first key matrix and first value matrix are stored in the database (640), the processor may store the first key matrix and first value matrix stored in the database (640) in the cache memory (630).

[0083] According to one embodiment, an electronic device (or a system including the same) that provides a multi-session service of a language model using a KV cache dump can support multiple user connections. For example, multiple users may each access the electronic device using different user terminals (612, 614, 616). Additionally, the electronic device may receive user input (e.g., natural language or a prompt generated based on natural language) from each of the multiple user terminals (612, 614, 616). At this time, the electronic device may identify the user based on the received user input and determine whether the user has changed. Additionally, if it is determined that the user has changed, the electronic device may change (or switch) from the session associated with the existing user (e.g., a first session) to the session associated with the changed user (e.g., a second session). For example, the electronic device may respond to a session change request to store the key matrix and value matrix stored in the cache memory (630) associated with the session associated with the existing user in the database (640), identify whether the key matrix and value matrix associated with the session associated with the changed user are stored in the database, and if they are stored, retrieve the key matrix and value matrix associated with the session associated with the changed user from the database and store them in the cache memory (630).

[0084] FIG. 7 is a diagram illustrating a method for multi-session service of a language model using a KV cache dump according to one embodiment of the present disclosure. Referring to FIG. 7, a processor (e.g., processor (410) of FIG. 4) of an electronic device (e.g., electronic device (100) of FIG. 1 or electronic device (400) of FIG. 4) for providing multi-session service of a language model using a KV cache dump (e.g., language model (102) of FIG. 1) may, in step 710 (S710), generate an embedding vector based on input natural language.

[0085] In step 720 (S720), the processor can generate a key matrix and a value matrix based on the generated embedding vector. For example, the processor can generate a key matrix and a value matrix corresponding to the input natural language used in the attention mechanism based on the generated embedding vector.

[0086] In step 730 (S730), the processor may store the key matrix and the value matrix in cache memory. For example, to increase the efficiency of the attention mechanism in natural language processing using a language model, the key matrix and the value matrix corresponding to the input natural language used in the attention mechanism may be cached.

[0087] In step 740 (S740), the processor may store the key matrix and value matrix stored in the cache memory in the database in response to a session change request. For example, in a situation where a service using a language model operates in multiple sessions, steps 710 through 730 may be performed per session. That is, the processor may generate an embedding vector based on the natural language input during the execution of each session, generate a key matrix and value matrix corresponding to the input natural language used in the attention mechanism based on the generated embedding vector, and store the generated key matrix and value matrix in the cache memory. At this time, when a session change request (or session switching request) is received, the processor may store the key matrix and value matrix stored in the cache memory in the database. For example, the processor may dump the first key matrix and first value matrix cached in association with the first session into the database in response to a change request from the first session to the second session.

[0088] FIG. 8 is a diagram illustrating a method for restoring a key matrix and a value matrix in response to a session change request according to an embodiment of the present disclosure. Referring to FIG. 8, a processor (e.g., processor (410) of FIG. 4) of an electronic device (e.g., electronic device (100) of FIG. 1 or electronic device (400) of FIG. 4) for providing a multi-session service of a language model (e.g., language model (102) of FIG. 1) using a KV cache dump can, in response to a session change request (or session switch request), identify in step 810 (S810) whether a key matrix and a value matrix associated with the changed session are stored in a database. For example, the processor can identify whether a second key matrix and a second value matrix associated with the second session are stored in a database in response to a change request from a first session to a second session. As another example, when a previously active session returns (changed from the first session to the second session and then back to the first session), the processor can identify whether the key matrix and value matrix associated with the returned session (the first session) are stored in the database.

[0089] If it is identified that the key matrix and value matrix associated with the changed session are not stored in the database (S810 - No), the processor may terminate the process of restoring the key matrix and value matrix in response to the session's change request.

[0090] If it is identified that the key matrix and value matrix associated with the changed session are stored in the database (S810 - Yes), the processor may store the key matrix and value matrix stored in the database in the cache memory at step 820 (S820). For example, in response to a change request from the first session to the second session, if the processor identifies that the second key matrix and second value matrix associated with the second session are stored in the database, the processor may restore the second key matrix and second value matrix associated with the second session stored in the database to the cache memory. For another example, in response to a return request of a previously active session (changing from the first session to the second session and then back to the first session), if the processor identifies that the key matrix and value matrix associated with the returned session (first session) are stored in the database, the processor may restore the key matrix and value matrix associated with the returned session (first session) stored in the database to the cache memory.

[0091] FIG. 9 is a diagram illustrating a method for processing a key matrix and a value matrix generated based on a summarized prompt according to an embodiment of the present disclosure. Referring to FIG. 9, a processor (e.g., processor (410) of FIG. 4) of an electronic device (e.g., electronic device (100) of FIG. 1 or electronic device (400) of FIG. 4) for providing a multi-session service of a language model (e.g., language model (102) of FIG. 1) using a KV cache dump can summarize a prompt generated based on a plurality of input natural languages ​​in step 910 (S910). For example, the processor can generate a prompt based on input natural languages ​​and summarize the generated prompt, either separately from or simultaneously with the operation of generating an embedding vector based on input natural languages ​​in steps 710 to 730 of FIG. 7, generating a key matrix and a value matrix based on the generated embedding vector, and storing the key matrix and the value matrix in a cache memory. As another example, instead of performing step 740 of Fig. 7, the processor can generate a prompt based on the input natural language and summarize the generated prompt.

[0092] In step 920 (S920), the processor can generate a key matrix and a value matrix based on the summarized prompt. For example, the processor can generate an embedding vector based on the natural language contained in the summarized prompt, and generate a key matrix and a value matrix based on the generated embedding vector.

[0093] In step 930 (S930), the processor may store the key matrix and value matrix in the database in response to a session change request. For example, the processor may dump the first key matrix and the first value matrix in the database in response to a change request from the first session to the second session, generated based on a prompt that is generated based on multiple natural languages ​​entered during the first session and summarized.

[0094] The above-described flowchart and description are merely examples and may be implemented differently in some embodiments. For instance, in some embodiments, the order of each step may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0095] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0096] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0097] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0098] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0099] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or marked data storage device, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.

[0100] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0101] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and discs include CDs, laser discs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0102] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0103] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0104] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0105] 100, 400: Electronic devices 120: Cache memory 130: Database 410: Processor 420: Memory 430: Communication circuit

Claims

Claim 1 A multi-session service method for a language model using a KV cache dump, performed by at least one processor, comprising: generating an embedding vector based on natural language input during the execution of a first session; generating a first key matrix and a first value matrix corresponding to the input natural language and used in an attention mechanism based on the generated embedding vector; storing the first key matrix and the first value matrix in a cache memory; storing the first key matrix and the first value matrix stored in the cache memory in a database in response to a change request from the first session to a second session; identifying whether a second key matrix and a second value matrix associated with the second session and used in the attention mechanism are stored in the database in response to a change request from the first session to the second session. A multi-session service method for a language model using a KV cache dump, comprising: a step of storing the second key matrix and the second value matrix stored in the database in the cache memory when the second key matrix and the second value matrix are stored in the database. Claim 2 A multi-session service method for a language model using a KV cache dump, wherein, in claim 1, the step of storing in the database includes the step of mapping the first key matrix and the first value matrix to information about the first session and storing them in the database. Claim 3 delete Claim 4 A multi-session service method of a language model using a KV cache dump, wherein, in claim 1, a request for change from the first session to the second session is received based on at least one of a specified time period, user input, a change by the user, or an analysis result of the input natural language. Claim 5 A multi-session service method of a language model using a KV cache dump, wherein, in paragraph 4, a request for change from the first session to the second session is received in response to the detection of a change by the user or a change in a category associated with the input natural language based on the analysis result of the input natural language. Claim 6 A multi-session service method for a language model using a KV cache dump according to claim 1, further comprising: a step of summarizing a prompt generated based on a plurality of natural languages ​​input during the execution of the first session; a step of generating a second key matrix and a second value matrix based on the summarized prompt; and a step of storing the second key matrix and the second value matrix in the database in response to a change request from the first session to the second session. Claim 7 A multi-session service method for a language model using a KV cache dump, further comprising the step of deleting the first key matrix and the first value matrix stored in the database based on at least one of the deletion of the first session or user input in claim 1. Claim 8 A multi-session service method of a language model using a KV cache dump according to claim 1, wherein the database is included in at least one of a server providing a service using a language model or a user terminal receiving natural language input. Claim 9 A computer program stored on a computer-readable non-transient recording medium for executing a method according to any one of paragraphs 1, 2, and 4 through 8 on a computer. Claim 10 In electronic devices, memory; An electronic device comprising at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program comprises instructions for generating an embedding vector based on natural language input during the execution of a first session, generating a first key matrix and a first value matrix corresponding to the input natural language and used in an attention mechanism based on the generated embedding vector, storing the first key matrix and the first value matrix in a cache memory, storing the first key matrix and the first value matrix stored in the cache memory in response to a change request from the first session to a second session in a database, identifying whether a second key matrix and a second value matrix associated with the second session and used in the attention mechanism are stored in the database in response to a change request from the first session to the second session, and if the second key matrix and the second value matrix are stored in the database, storing the second key matrix and the second value matrix stored in the database in the cache memory.

Images