Electronic device and method for processing user query by using artificial intelligence model, and non-transitory storage medium

Abstract

The present document relates to an electronic device and a method for processing a user query by using an artificial intelligence model, and a non-transitory storage medium. According to one embodiment, the electronic device may comprise at least one processor and a memory for storing instructions. When executed individually or collectively by the at least one processor, the instructions can instruct the electronic device to: identify a predesignated first user query and a second user query related to the first user query; obtain, in order to request an artificial intelligence model to process the second user query, first information obtained by tokenizing the first user query and second information obtained by tokenizing the second user query, the first information being obtained from a prompt cache (320) of the memory, and the second information being obtained through an operation for tokenizing the second user query; provide the first information and the second information to the artificial intelligence model; obtain a first response to the first user query and the second user query through the artificial intelligence model, and store, in the prompt cache, the second information and third information obtained by tokenizing the first response; identify a third user query received after acquiring the first response to the first user query and the second user query; obtain, in order to request that the artificial intelligence model processes the third user query, fourth information obtained by tokenizing the third user query through an operation for tokenizing the third user query, and obtain the first information, the second information, and the third information from the prompt cache; and obtain a second response to the third user query through the artificial intelligence model. Other various embodiments are also possible.

Description

Electronic device, method, and non-transient storage medium for processing user queries using an artificial intelligence model

[0001] The present disclosure relates to an electronic device, a method, and a non-transient storage medium for processing user queries using an artificial intelligence model.

[0002] With the advancement of digital technology, electronic devices are being provided in various forms, such as smartphones, tablet PCs, and PDAs. Electronic devices are also being developed in wearable forms to enhance portability and user accessibility. Electronic devices can be configured in various forms to be worn on parts of the user's body, and as technology advances, technologies are being developed to provide real-world spaces that correspond to the actual external environment (e.g., virtual reality, augmented reality, or mixed reality).

[0003] Meanwhile, the electronic device may utilize artificial intelligence (AI) models to provide various services. At least some of the various AI models for various services may be implemented as generative AI models. Depending on the implementation, the AI ​​models may operate in a form where multiple AI models are connected.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] According to one embodiment of the present disclosure, an electronic device may include at least one processor comprising a processing circuit and a memory for storing instructions.

[0006] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to check a predetermined first user query and a second user query associated with the first user query.

[0007] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains first information tokenized from the first user query and second information tokenized from the second user query in order to request an artificial intelligence model to process the second user query. According to one embodiment, the electronic device obtains the first information from the prompt cache of the memory and obtains the second information through an operation to tokenize the second user query.

[0008] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device provides the first information and the second information to the artificial intelligence model, and obtains the first user query and the first response to the second user query through the artificial intelligence model, and stores the third information, which is tokenized from the second information and the first response, in the prompt cache.

[0009] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to check a received third user query after obtaining a first response to the first user query and the second user query.

[0010] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains the first information, the second information, and the third information from the prompt cache while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query in order to request the artificial intelligence model to process the third user query.

[0011] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains a second response to the third user query through the artificial intelligence model.

[0012] According to one embodiment, a method of operation in an electronic device includes an operation of verifying a first user query that is pre-fixed or pre-specified and a second user query related to the first user query.

[0013] According to one embodiment, the method includes the operation of obtaining first information tokenized from the first user query and second information tokenized from the second user query in order to request an artificial intelligence model to process the second user query. According to one embodiment, the first information is obtained from a prompt cache of the memory, and the second information is obtained through an operation to tokenize the second user query.

[0014] According to one embodiment, the method includes providing the first information and the second information to the artificial intelligence model, obtaining the first user query and the first response to the second user query through the artificial intelligence model, and storing the third information, which is tokenized from the second information and the first response, in the prompt cache.

[0015] According to one embodiment, the method includes the operation of verifying a received third user query after obtaining a first response to the first user query and the second user query.

[0016] According to one embodiment, the method includes the operation of obtaining the first information, the second information, and the third information from the prompt cache, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request the artificial intelligence model to process the third user query.

[0017] According to one embodiment, the method includes the operation of obtaining a second response to the third user query through the artificial intelligence model.

[0018] According to one embodiment, in a non-transient storage medium storing one or more programs, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to perform an operation of verifying a pre-fixed or pre-specified first user query and a second user query associated with the first user query.

[0019] According to one embodiment, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining first information tokenized from the first user query and second information tokenized from the second user query in order to request an artificial intelligence model to process the second user query. According to one embodiment, the first information is obtained from a prompt cache of the memory, and the second information is obtained through an operation to tokenize the second user query.

[0020] According to one embodiment, the one or more programs include an executable instruction to cause the electronic device, when executed by at least one processor of the electronic device, to provide the first information and the second information to the artificial intelligence model, and to obtain the first user query and the first response to the second user query through the artificial intelligence model, and to store the third information tokenized from the second information and the first response in the prompt cache.

[0021] According to one embodiment, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of verifying a received third user query after obtaining a first response to the first user query and the second user query.

[0022] According to one embodiment, the one or more programs include an executable instruction to cause the electronic device, when executed by at least one processor of the electronic device, to execute the operation of obtaining the first information, the second information, and the third information from the prompt cache, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request the artificial intelligence model to process the third user query.

[0023] According to one embodiment, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining a second response to the third user query through the artificial intelligence model.

[0024] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0025] FIG. 2 is a block diagram showing an example of the configuration of an electronic device according to one embodiment.

[0026] FIG. 3 is a diagram showing an example of the configuration of an artificial intelligence model in an electronic device according to one embodiment.

[0027] FIGS. 4a and 4b are block diagrams illustrating an example of processing a user query using an artificial intelligence model in an electronic device according to one embodiment.

[0028] FIG. 5 is a diagram showing an example of a method of operation in an electronic device according to one embodiment.

[0029] FIGS. 7a and 7b are drawings illustrating an example of processing a user query using an artificial intelligence model in an electronic device according to one embodiment.

[0030] FIG. 8 is a diagram illustrating an example of a method of operation in an electronic device according to one embodiment.

[0031] FIG. 9 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0032] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0033] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. The term "user" as used in the embodiments of the present disclosure may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0034] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0035] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0036] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0037] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0038] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0039] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0040] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0041] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0042] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0043] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0044] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0045] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0046] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0047] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0048] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0049] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0050] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0051] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0052] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0053] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0054] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0055] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be of the same or different type as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0056] FIG. 2 is a block diagram showing an example of the configuration of an electronic device according to one embodiment.

[0057] Referring to FIG. 2, an electronic device (401) according to one embodiment (e.g., the electronic device (101) of FIG. 1, or the electronic device (201) of FIG. 2) may include at least one processor (210), memory (220), display (230), camera circuit (240), communication circuit (250), microphone circuit (260), and speaker (270). Not limited thereto, the electronic device (201) may be implemented identically or similarly to the electronic device (101) of FIG. 1 and may further include other components of the electronic device (101) of FIG. 1. In addition, the electronic device (200) may be configured to include other components necessary for the method of operation of the present disclosure. According to one embodiment, the electronic device (201) may be a device in the form of an on-device that stores an artificial intelligence model (310) in memory (120). An electronic device according to one embodiment may be a wearable device that a user can wear on their body (e.g., head), such as a smartphone-shaped device, a head-mounted display (HMD) device, augmented reality (AR) glasses, and / or a VST device. In one embodiment, the main components of the electronic device (101, 201) of FIGS. 1 and 2 have been described. However, in various embodiments, the components illustrated in FIGS. 1 and 2 are not all essential components, and the electronic device (101, 201) may be implemented by more components than those illustrated, or by fewer components. Additionally, the location of the main components of the electronic device (101, 201) described above in FIGS. 1 and 2 may be changed according to various embodiments.Referring to FIG. 2, according to one embodiment, a processor (210) (e.g., processor (120) of FIG. 1) can process a user query (e.g., text or voice information) based on the user query (e.g., text or voice-converted text) using an artificial intelligence model (e.g., 310 of FIG. 3) to improve processing speed by reducing the number of operations in the artificial intelligence model. According to one embodiment, the electronic device (201) may use an artificial intelligence model contained in memory (220) in an on-device form (e.g., used in operations where privacy protection is important) or may use an artificial intelligence model contained in a server (e.g., server (108) of FIG. 1). It is not limited thereto, and depending on the operation, both the artificial intelligence model contained in memory (220) and the artificial intelligence model contained in the server (e.g., server (108) of FIG. 1) may be used.

[0058] According to one embodiment, the processor (210) may pre-fix or pre-specify a first user query to generate a response to a user query. The operation of the processor (210) to pre-fix or pre-specify and process the user query is described in detail later in FIG. 4a.

[0059] FIG. 3 is a diagram showing an example of the configuration of an artificial intelligence model in an electronic device according to one embodiment.

[0060] Referring to FIG. 3, an electronic device (201) according to one embodiment may be configured to include an artificial intelligence model (310) in memory (220). For example, the artificial intelligence model (310) may include a large language model (LLM) (311a) included in storage (221) and a large language model (LLM) (311a) included in RAM (223). According to one embodiment, memory (220) may store a prompt cache (320). The prompt cache (320) may be divided into a prompt cache (320a) included in storage (221) and a prompt cache (320b) included in RAM (223). According to one embodiment, memory (220) may include a KV cache (313) (e.g., token information cache) in RAM (223). For example, the KV cache (313) may be included in the artificial intelligence model (310). According to one embodiment, the processor (210) may perform an operation to process a user's query using the artificial intelligence model (310) to obtain information through a tokenization operation of the user's query when generating a response, and may store the obtained tokenized information in the prompt cache (320) of the memory (220). The artificial intelligence model (310) may store the information obtained through the tokenization operation in the KV cache (313) to provide the tokenized information to the prompt cache (320). The prompt cache (320) may be a part that pre-caches commands (e.g., prefix input or pre-specified input) in the memory (220). The prompt cache (320) can cache commands (e.g., save) using a specified prompt caching pool for frequently used functions and / or frequently used sentences.The prompt cache (320a) stored in storage (221) may be stored according to priority for prompts that the user has recently used frequently. Here, priority may change depending on time, place, or various other factors. The prompt may be configured to include commands and content. The command (e.g., first user query) is an instruction based on the operation of the artificial intelligence model (310) and may be used identically for each use case. The KV cache (313) may be a part that generates a kv cache value when a token is input and loads the stored kv cache value during the next token operation. The KV cache (313) may pre-calculate kv cache values ​​(e.g., key and value) for a pre-specified command (e.g., first user query) and pre-store them in the prompt cache (320) (e.g., part of the prompt cache (320a)).

[0061] According to one embodiment, the processor (210) can pre-assign a command (e.g., the first user query (411) of FIG. 4a) input for the operation of the artificial intelligence model (310) by selecting one of various priority criteria based on words recently used by the user in a category set according to the product, and can pre-store the specified command in memory (220). For example, the processor (210) can pre-store commands such as "Please create an appropriate response in the conversation," "Please summarize the sentence," and "Please correct and rectify errors in the input sentence." For example, the prompt caching pool for each command can be set to three. For example, if you set the prompt caching pool to 3, the first pool of the first command (e.g., command(summary)) can be set to "Please summarize the sentence", the second pool of the first command (e.g., command(summary), user contents 1) can be set to "Please summarize the sentence, weather", and the third pool of the second command (e.g., command(summary), user contents 2) can be set to "Please summarize the sentence, traffic". For example, if the prompt caching pool is set to three, the first pool of the second command (e.g., command (correction)) can be set to "Please correct errors in the input sentence and correct it," the second pool of the second command (e.g., command (correction), user contents 1) can be set to "Please correct errors in the input sentence and correct it, paper," and the third pool of the third command (e.g., command (correction), user contents 2) can be set to "Please correct errors in the input sentence and correct it, patent."

[0062] According to one embodiment, the processor (210) may, for example, identify (e.g., load or obtain) a command such as "Please create an appropriate response in the next conversation" from the prompt cache (320) of the memory (220) as a first user query among the commands stored in advance in the prompt cache (320) of the memory (220), and use the KV cache (313) to obtain a first kv cache value (e.g., key and value) for the first user query from the prompt cache (320) (e.g., load the first kv cache value stored in the prompt cache (320b) by copying from the prompt cache (320a) portion).

[0063] According to one embodiment, the processor (210) can check a second user query (e.g., "What is the weather like today?"). The processor (210) can calculate only the second kv cache value (e.g., key and value) for the second user query using the KV cache (313), without calculating the first kv cache value (e.g., key and value) for the first user query using the KV cache (313). Here, the first kv cache value may also be described as first information, and the second kv cache value may also be described as second information. According to one embodiment, the processor (210) can pass a first user query (e.g., "Please create an appropriate response in the next conversation.") and a second user query (e.g., "What is the weather like today?") as inputs to an artificial intelligence model (310), and pass a first cache value obtained from a prompt cache (320) and a second cache value calculated using a KV cache (313) to the artificial intelligence model (310). According to one embodiment, the first cache value and the second cache value can be stored in the prompt cache (320) of the memory (220) (e.g., the prompt cache (320a) portion). According to one embodiment, the processor (210) can obtain a first response (e.g., "It's clear") generated based on a first user query and a second user query through an artificial intelligence model (310), and store a third cache value (e.g., third information) tokenized from the first response in a prompt cache (320) (e.g., the prompt cache (320a) portion via the prompt cache (320b)). Accordingly, the electronic device (201) can perform the operation result quickly by not recalculating the first cache value for the first user query when calculating the second cache value for the second user query, as in the prior art.

[0064] According to one embodiment, the processor (210) can check a third user query (e.g., "I'm going camping"). The processor (210) can calculate only the fourth kv cache value (e.g., key and value) for the third user query using the KV cache (313), without calculating the first kv cache value for the first user query and the second cache value for the second user query using the KV cache (313). Here, the fourth kv cache value may also be described as fourth information. According to one embodiment, the processor (210) can pass a first user query (e.g., "Please create an appropriate response in the next conversation"), a second user query (e.g., "What is the weather like today?"), a first response (e.g., "It's sunny"), and a third user query (e.g., "I'm going camping") as inputs to an artificial intelligence model (310), and pass a first cache value, a second cache value, and a third cache value obtained from a prompt cache (320) using a KV cache (313), and a calculated fourth cache value to the artificial intelligence model (310). According to one embodiment, the first cache value, the second cache value, the third cache value, and the fourth cache value can be stored in the prompt cache (320) (e.g., the prompt cache (320a) portion) of the memory (220). According to one embodiment, the processor (210) may obtain a second response (e.g., "Do you have a plan?") generated based on a first user query, a second user query, a first response, and a third user query through an artificial intelligence model (310), and may store a fifth cache value (e.g., fifth information) tokenized from the second response in a prompt cache (320) (e.g., the prompt cache (320a) portion via the prompt cache (320b)).Accordingly, when the electronic device (201) calculates the fourth cache value for the third user query as in the prior art, it does not recalculate the first cache value for the first user query, the second cache value for the second user query, and the third cache value for the first response, thereby allowing the calculation result to be performed quickly and thus the execution speed can be improved compared to the prior art.

[0065] FIGS. 4a and 4b are block diagrams illustrating an example of processing a user query using an artificial intelligence model in an electronic device according to one embodiment.

[0066] Referring to FIGS. 4a and 4b, a processor (210) of an electronic device (201) according to one embodiment loads (e.g., obtains or confirms) a pre-specified first user query (411) (e.g., command) from a prompt cache (320) of memory (220), and when confirming a second user query (412) (e.g., "hello") associated with the first user query (411), the first user query (411) (e.g., "<make reply> ...) and a second user query (412) can be provided. For example, the first user query (411) can be identified as a command such as "Please create an appropriate response in the next conversation" among the commands pre-stored in the prompt cache (320) of the memory (220).

[0067] According to one embodiment, the processor (210) may provide the first user query (411) and the second user query (412) to the artificial intelligence model (310) to request the artificial intelligence model (310) to process the second user query (412). According to one embodiment, when the processor (210) requests the artificial intelligence model (310) to process the second user query (412), it may use the KV cache (313) to obtain (e.g., copy or load) the first information (421) (e.g., key and value) of the first user query (411) from the prompt cache (320). The processor (210) may use the KV cache (321) to tokenize the second user query (412) and obtain the calculated second information (422) (e.g., key and value). Here, the first information (421) may be a pre-stored first key (e.g., key vector) and first value (e.g., value vector) copied (or loaded) from the prompt cache (320) of the memory (220). Here, the key vector represents information that a word can provide in context, and the value vector may represent information that the word actually possesses. Here, the second information (422) may include a second key and a second value obtained by separating the text information of the second user query (412) into at least one token for each word and performing a key and value operation for each of the separated tokens.

[0068] According to one embodiment, when the processor (210) requests the artificial intelligence model (310) to process a first user query (411) and a second user query (412), it may provide the artificial intelligence model (310) with the first information (421) copied from the prompt cache (320) and the second information (422) calculated using the KV cache (321), and store the second information (422) in the prompt cache (320) of the memory (220) for processing the next user query.

[0069] According to one embodiment, the processor (210) can obtain (e.g., generate) a first response (431) (e.g., "hi") to a first user query (411) and a second user query (412) through an artificial intelligence model (310). According to one embodiment, the processor (210) can display the first response (431) through a display (230), tokenize the first response (431), and store the third information (423) calculated using the KV cache (313) in the prompt cache (320). For example, the processor (210) can update the prompt cache (320) by transmitting the second information and the third information (423) separately to the prompt cache (320) or by transmitting the second information (422) and the third information (423) together (or simultaneously) to the prompt cache (320) for processing the next user query. The processor (210) can output the first response (431) through the speaker (270) if the first response (431) contains voice information. Here, the third information (423) may include a third key and a third value obtained through an attention operation that checks the contextual association of at least one word included in the first response (431).

[0070] According to one embodiment, the processor (210) can check a received third user query (413) (e.g., "how are you") after obtaining a first response (431) to a first user query (411) and a second user query (412).

[0071] According to one embodiment, the processor (210) may request the artificial intelligence model (310) to process the third user query (413) by providing the first user query (411), the second user query (412), and the third user query (413) to the artificial intelligence model (310), tokenize the third user query (413) through an operation to tokenize the third user query (413), and obtain the fourth information (424) for the tokens separated through tokenization by calculating the fourth information (424) using the KV cache (313). According to one embodiment, when the processor (210) obtains the fourth information (424), it may obtain (e.g., copy or load) the first information (421), the second information (422), and the third information (423) from the prompt cache (320). Here, the fourth information (424) may include a fourth key and a fourth value obtained (e.g., calculated) using the KV cache (313) by separating the text information of the third user query (413) into at least one token per word and performing a key and value operation for each of the separated tokens (e.g., an attention operation to check the contextual association of at least one word).

[0072] According to one embodiment, the processor (210) may obtain a second response (432) (e.g., "good") to a third user query (413) through an artificial intelligence model (310) and display the obtained second response (432) on a display (230). While obtaining the second response (432), the processor (210) may store fourth information (424) and fifth information (425) regarding the second response (432) in a prompt cache (320). If the second response (432) includes voice information, the processor (210) may output it through a speaker (270). Here, the fifth information (425) may include a fifth key and a fifth value obtained (e.g., calculated) using the KV cache (313) through an operation for tokenizing the second response (e.g., an attention operation to check the contextual association of at least one word included in the second response) and an operation for each of the separated tokens (e.g., an attention operation to check the contextual association of at least one word).

[0073] According to one embodiment, the processor (210) may use a pre-configured token table (e.g., token information) (510) by mapping a unique index (e.g., token ID) to each word (vocabulary) for tokenization of a user query using an artificial intelligence model (310), as illustrated in FIG. 5. For example, the token table (510) may be received from an external electronic device (e.g., server) and stored in memory (220). For example, a word may be separated into one token or multiple tokens.

[0074] According to one embodiment, an artificial intelligence model (310) can tokenize text information contained in a first user query (411) (e.g., “Please create an appropriate response in the next conversation.”) received from a processor (210) in word units to separate the tokens, and obtain (e.g., generate or produce) first information (421) including a first key and a first value using a KV cache (313) through an operation for tokenization (e.g., attention operation) based on an index assigned to the separated tokens using a pre-specified token table (510). The first information (421) for the first user query (411) may be calculated in advance and stored in advance in the prompt cache (320) of the memory (220) before performing an operation to process user queries according to the conversation.

[0075] According to one embodiment, the artificial intelligence model (310) can, as illustrated in FIG. 4a, tokenize the text information contained in the second user query (412) (e.g., "hello") received from the processor (210) in word units to separate the tokens, and obtain (e.g., generate or produce) second information (422) including a second key and a second value using the KV cache (313) through an operation for tokenization (e.g., attention operation) based on an index assigned to the separated tokens using a pre-assigned token table (510). The artificial intelligence model (310) can generate a first response (431) (e.g., "hi") to a second user query (412) using second information (422) obtained using the KV cache (313) and first information (421) obtained (e.g., copied) from the prompt cache (320) of the memory (220). According to one embodiment, the artificial intelligence model (310) can obtain (e.g., generate or produce) third information (423) including a third key and a third value using the KV cache (313) through an operation (e.g., attention operation) for tokenization of the first response (431).

[0076] According to one embodiment, the artificial intelligence model (310) may, in response to a processing request for a third user query (413) (e.g., "how are you") received from the processor (210) as illustrated in FIG. 4b, tokenize the text information contained in the third user query (413) in word units to separate the tokens, and obtain (e.g., generate or produce) a fourth information (424) containing a key and a value through an operation for tokenization (e.g., an attention operation) based on an index assigned to the separated tokens using a pre-assigned token table (510). The artificial intelligence model (310) may generate a second response (432) to the third user query (413) using the obtained fourth information (424) and the first information (421), second information (422), and third information (423) obtained from the prompt cache (320). According to one embodiment, the artificial intelligence model (310) can obtain (e.g., generate or produce) a fifth piece of information (425) including a key and a value through an operation (e.g., an attention operation) for tokenization of the second response (432).

[0077] According to one embodiment, the processor (210) may be a hardware component (function) or a software element (program) comprising at least one component provided in the electronic device (201), such as a hardware module or a software module (e.g., an application program). According to one embodiment, the processor (210) may include, for example, one or more combinations of hardware, software, or firmware. The processor (210) may be configured to omit at least some of the components or to include additional components for performing image processing operations in addition to the components.

[0078] According to one embodiment, the memory (220) (e.g., the memory (130) of FIG. 1) may store applications. For example, the memory (220) may store applications related to artificial intelligence (functions or programs) and applications related to receiving user queries (e.g., messenger applications). The memory (220) may store tokenized information provided by the artificial intelligence model (310). The memory (220) may include a database that stores contextual information learned by the artificial intelligence model (310).

[0079] According to one embodiment, the memory (220) may store various data generated during the execution of the program (140), including a program used for functional operation (e.g., the program (140) of FIG. 1). The memory (220) may store data (e.g., a program or function) related to an on-device artificial intelligence model. The memory (220) may include a prompt cache that stores tokenized information. For example, the memory (220) may include a program (140) area and a data area (not shown). The program (140) area may store program information related to the operation of the electronic device (201), such as an operating system (OS) (e.g., the operating system (142) of FIG. 1) that boots the electronic device (201). The data area (not shown) may store transmitted and / or received data and generated data according to various embodiments. Additionally, the memory (220) may be configured to include at least one storage medium among flash memory, hard disk, multimedia card micro type memory (e.g., secure digital (SD) or extreme digital (XD) memory), RAM, and ROM.

[0080] According to one embodiment, a display (230) (e.g., the display module (160) of FIG. 1, the display (251, 252) of FIG. 2) can display information related to processing a user query or a screen related to user query and response (e.g., the execution screen of a messenger application) under the control of a processor (210). According to one embodiment, the display (230) can be implemented in the form of a touch screen. When the display (230) is implemented in the form of a touch screen together with an input module, it can display various information generated according to the user's touch operation. According to one embodiment, the display (230) can be composed of at least one of an LCD (liquid crystal display), a TFT-LCD (thin film transistor LCD), an OLED (organic light emitting diodes), an LED, an AMOLED (active matrix organic LED), a flexible display, and a 3-dimensional display. Additionally, some of these displays may be configured to be transparent or light-transmitting so that the outside can be seen through them. This may be configured in the form of a transparent display including a TOLED (transparent OLED). According to one embodiment, other display modules (e.g., an expansion display or a flexible display) mounted in addition to the display (230) may be further included.

[0081] According to one embodiment, the camera circuit (240) (e.g., the camera module (180) of FIG. 1) may include at least one camera and may capture an image of the external environment (e.g., a 2D image or a 3D image) so that the actual external environment is displayed through a display in a real space (e.g., a virtual reality space, an augmented reality space, or a mixed reality space) or on a screen corresponding to the real space (e.g., to display a screen).

[0082] According to one embodiment, a communication circuit (250) (e.g., a communication module (190) of FIG. 1) can communicate with an external electronic device (e.g., an electronic device (102, 104) of FIG. 1, a server (108) of FIG. 1, or another user's electronic device). For example, the communication circuit (250) can receive a message regarding a user query from an external electronic device and transmit a message regarding a response to the user query to the external electronic device. According to one embodiment, the communication circuit (250) may include a cellular module, a Wi-Fi (wireless-fidelity) module, a Bluetooth module, or a near field communication (NFC) module.

[0083] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and / or the electronic device (201) of FIG. 2) may implement a software module (e.g., the program (140) of FIG. 1) related to an artificial intelligence model. The memory of the electronic device (e.g., the memory (130) of FIG. 1 and / or the memory (220) of FIG. 2) may store instructions (e.g., instructions) to implement the software module. At least one processor (e.g., processor (120) of FIG. 1 and / or processor (210) of FIG. 2) can execute instructions stored in memory to implement a software module and can control hardware associated with the function of the software module (e.g., sensor module (176) of FIG. 1, camera module (180), communication module (190) of FIG. 1 and / or communication circuit (250) of FIG. 2, display module (160) of FIG. 1 and / or display (230) of FIG. 2).

[0084] A software module of an electronic device (101, 201) according to one embodiment may be configured to include a kernel (or HAL), a framework (e.g., middleware (144) of FIG. 1), and an application (e.g., application (146) of FIG. 1). At least some of the software modules may be preloaded onto the electronic device (101, 201) or downloadable from a server (e.g., server (108)).

[0085] According to one embodiment, the kernel may include, for example, a system resource manager or a device driver, but may be configured to include other modules, not limited thereto. The system resource manager may perform control, allocation, or reclamation of system resources. The device driver may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an IPC (inter-process communication) driver.

[0086] According to one embodiment, the framework may provide functions commonly required by the application, or provide various functions to the application through an application programming interface (API) (not shown) so that the application can efficiently use limited system resources within the electronic device (101, 201). The framework may include modules that form combinations of various functions of the components. The framework may provide modules specialized for each type of operating system to provide differentiated functions. The framework may dynamically delete some existing components or add new components.

[0087] According to one embodiment, the application may be configured to include an application (e.g., a module, a manager, or a program) for displaying an image of the external environment in real space. The application may include an application received from an external electronic device (e.g., a server (108) or an electronic device (102, 104)). According to one embodiment, the application may include a preloaded application or a third-party application downloadable from a server. The components of the software module and the names of the components according to the illustrated embodiments may vary depending on the type of operating system. According to one embodiment, at least a portion of the software module may be implemented as software, firmware, hardware, or a combination of at least two of these. At least a portion of the software module may be implemented (e.g., executed) by a processor (e.g., AP). At least a portion of the software module may include, for example, a module, a program, a routine, a set of instructions, or a process for performing at least one function.

[0088] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may include at least one processor (processor (120) of FIG. 1 or processor (210) of FIG. 2) including a processing circuit and a memory (e.g., memory (130) of FIG. 1 or memory (220) of FIG. 2) for storing instructions.

[0089] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to check a predetermined first user query and a second user query associated with the first user query.

[0090] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device obtains first information tokenized from the first user query and second information tokenized from the second user query in order to request processing of the second user query from an artificial intelligence model (e.g., the artificial intelligence model (310) of FIG. 3). According to one embodiment, the first information may be obtained from a prompt cache of the memory (e.g., the prompt cache (320) of FIG. 3), and the second information may be obtained through an operation to tokenize the second user query.

[0091] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first information and the second information to be provided to the artificial intelligence model, and through the artificial intelligence model, to obtain the first user query and the first response to the second user query, and to store the third information tokenized from the second information and the first response in the prompt cache.

[0092] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to check a third user query received after obtaining a first response to the first user query and the second user query.

[0093] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the first information, the second information, and the third information from the prompt cache, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request the artificial intelligence model to process the third user query.

[0094] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a second response to the third user query through the artificial intelligence model.

[0095] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may cause the third user query tokenized fifth information and the second response tokenized fifth information to be stored in the prompt cache while obtaining the second response.

[0096] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first response to be displayed through the display of the electronic device or the first response to be output through the speaker of the electronic device.

[0097] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the second response to be displayed through the display or output through the speaker.

[0098] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the device to separate text information of the second user query into at least one token by word through the artificial intelligence model based on requesting processing of the second user query from the artificial intelligence model, and to obtain the second information including a second key and a second value obtained through key and value operations on the tokens separated from the second user query based on an index assigned to the at least one token through the artificial intelligence model.

[0099] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to separate text information of the third user query into at least one token word by word through the artificial intelligence model based on requesting processing of the third user query to the artificial intelligence model, and to obtain the fourth information including a fourth key and a fourth value obtained through key and value operations on the tokens separated from the third user query based on an index assigned to the at least one token through the artificial intelligence model.

[0100] According to one embodiment, the first information may include a first key and a first value obtained through an attention operation that checks the contextual association of at least one word included in the first user query.

[0101] According to one embodiment, the third information may include a third key and a third value obtained through an attention operation that checks the contextual association of at least one word included in the first response.

[0102] According to one embodiment, the fifth information may include a fifth key and a fifth value obtained through an attention operation that checks the contextual association of at least one word included in the first response.

[0103] FIG. 6 is a diagram illustrating an example of an operation method in an electronic device according to one embodiment. FIG. 7a and FIG. 7b are diagrams illustrating an example of processing a user query using an artificial intelligence model in an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0104] Referring to FIGS. 6, FIGS. 7a, and FIGS. 7b, an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and the electronic device (201) of FIG. 2) can, in operation 601, check a pre-fixed or pre-specified first user query (711) (e.g., the first user query (411) of FIG. 4a and FIG. 4b) and a second user query (712) related to the first user query (711) (e.g., "hello"). For example, the electronic device can perform a processing operation to generate a response to messages received using a messenger application using an artificial intelligence model (e.g., the artificial intelligence model (310) of FIG. 3), as shown in FIG. 7a. The electronic device can pre-fix the first user query before running the messenger application. The electronic device may pre-set a pre-specified command, such as "Please create an appropriate response in the next conversation," as the first user query (711).

[0105] In operation 603, the electronic device may obtain (e.g., load or copy) a pre-stored first information (e.g., kv cache value) for a first user query (711) from a prompt cache (e.g., prompt cache (320) of FIG. 3) of memory (e.g., memory (130) of FIG. 1 or memory (220) of FIG. 2) in order to request processing of the second user query (712) from an artificial intelligence model (e.g., artificial intelligence model (310) of FIG. 3) based on verifying the second user query (712).

[0106] In operation 605, the electronic device may tokenize the second user query (712) and obtain (e.g., calculate or generate) second information regarding the separated tokens using a KV cache (e.g., the KV cache of FIG. 3). Here, the second information may include a second kV cache value calculated through an operation to tokenize the second user query (712) using the KV cache of the artificial intelligence model. The electronic device may store the second information in the prompt cache of memory while providing the first information and the second information to the artificial intelligence model.

[0107] In operation 607, the electronic device can obtain a first response (713) (e.g., "hi") to a first user query (711) and a second user query (712) through an artificial intelligence model, and tokenize the first response (713) to obtain (e.g., kv cache value) third information for the separated tokens using the KV cache (e.g., calculate or generate). The electronic device can display the first response (713) on a display (e.g., the display module (160) of FIG. 1 and the display (230) of FIG. 2) and store the third information tokenized from the first response in a prompt cache.

[0108] In operation 609, the electronic device can check whether the next user query has been confirmed. For example, the electronic device can check that there is no next user query when the next user query is not confirmed for a specified period of time, or when an application executed in relation to the method of operation of the present disclosure (e.g., the messenger application of FIG. 7a and FIG. 7b) is terminated. If, as a result of the check, the next user query is not confirmed, the electronic device terminates the operation, and if the next user query is confirmed, the electronic device can perform operation 611.

[0109] In operation 611, the electronic device can obtain the first response (713) to the first user query (711) and the second user query (712), and then check the next received user query, the third user query (714) (e.g., "how are you"). To request processing of the third user query (714) from the artificial intelligence model, the electronic device can obtain the information stored in the prompt cache (e.g., first information, second information, and third information) by using the KV cache to obtain (e.g., calculate) the next information (e.g., fourth information about the tokens separated by tokenizing the third user query (714)) by tokenizing the third user query through an operation to tokenize the third user query (714).

[0110] In operation 613, the electronic device can obtain (e.g., generate) a second response (715) (e.g., "good") to a third user query (714) through an artificial intelligence model, and tokenize the second response (715) to obtain (e.g., calculate or generate) fifth information (e.g., kv cache value) for the separated tokens using the KV cache. The electronic device can display the second response (715) on a display and store the fifth information tokenized from the second response (715) in a prompt cache. Afterward, the electronic device can perform operation 609. When the next user query (e.g., the fourth user query (716) ("what's the weather like today?")) is confirmed, processing of the next user query is requested, such as in the 611 and 613 operations, and information obtained and stored that tokenizes the next user query through operations to obtain and tokenize the next response (e.g., the third response (717) ("It's clear")), and information obtained and stored that tokenizes the next response.

[0111] FIG. 8 is a diagram illustrating an example of a method of operation in an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. FIG. 8 is a diagram illustrating a specific description of an operation to process a predetermined first user query, and an operation to process the user query and to obtain (e.g., calculate) and store information about the user query (e.g., kv cache values ​​(key and value)) by an artificial intelligence model in operations 605, 607, 611, and 613 of FIG. 6.

[0112] Referring to FIG. 8, an artificial intelligence model (e.g., an artificial intelligence model (310) of FIG. 3) of an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and the electronic device (201) of FIG. 2) can, in operation 801, perform tokenization on the text (e.g., sentence) included in the transmitted user query in word units based on a processing request for a user query (e.g., a second user query) to separate the tokens.

[0113] In operation 803, the artificial intelligence model can check whether the kv cache value for the separated tokens exists (e.g., is stored in advance) in the prompt cache (e.g., the prompt cache (320) of FIG. 3) of memory (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2). If the check reveals that the kv cache value for the separated tokens exists in the prompt cache, the artificial intelligence model performs operation 805, and if the artificial intelligence model does not find that the kv cache value for the separated tokens exists in the prompt cache, the artificial intelligence model performs operation 807.

[0114] In operation 805, the artificial intelligence model can obtain (e.g., load or copy) the kv cache value from the prompt cache based on identifying that the kv cache value for the separated tokens exists in the prompt cache. Then, the artificial intelligence model can perform operation 809.

[0115] In operation 807, the artificial intelligence model can calculate (e.g., generate or obtain) the kv cache value for the separated tokens using the KV cache (e.g., the KV cache (313) of FIG. 3) based on identifying that the kv cache value for the separated tokens does not exist in the prompt cache.

[0116] In operation 809, the artificial intelligence model can perform processing on a user query and obtain a processing result (e.g., a response). For example, the artificial intelligence model can process a first user query and a second user query requested for processing for a pre-specified command, and generate a first response to the first user query and the second user query as a processing result.

[0117] In operation 811, the artificial intelligence model can use the KV cache (e.g., the KV cache (313) of FIG. 3) to calculate (e.g., generate or obtain) the kv cache value for the processing result (e.g., response).

[0118] In operation 813, the artificial intelligence model may store the kv cache value (e.g., second cache value) for the user query requested for processing (e.g., second user query) and the kv cache value for the processing result (e.g., response) in the prompt cache in memory. Afterward, the artificial intelligence model may terminate the operation to process the user query requested.

[0119] When the artificial intelligence model receives a request to process the next user query, it can perform the operations of FIG. 8 described above again to process the transmitted user query.

[0120] FIG. 9 is a diagram illustrating a generative artificial intelligence system according to one embodiment.

[0121] Referring to FIG. 9, in a generative artificial intelligence system (900) according to one embodiment, a user query / response interface (910) (e.g., an input module (150) of FIG. 1 or a microphone circuit (260) of FIG. 2, and / or a display module (160), a first display (230) of FIG. 2) may receive user input. The user input may be in the form of natural language, images, and / or videos, but is not limited thereto. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Additionally, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (910) can output results of the generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of actions requested by the user. The user query / response interface (910) can output results of the generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of actions requested by the user.

[0122] An artificial intelligence framework (940) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can receive input from a user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0123] User input received from the user query / response interface (910) can be transmitted to a prompt design component (941) (e.g., the processor (120) of FIG. 1). The prompt design component (941) can be used to generate prompts suitable for inputting user input into a large language model (LM), a large vision model (LVM), or a large multimodal model (LMM). The prompt design component (941) may be an artificial intelligence component that uses machine learning algorithms or neural networks to develop better prompts over time. The prompt design component (941) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples based on user input, and can transmit the generated prompts to the LLM, LVM, or LMM.

[0124] An API / Plug-in management component (942) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model (e.g., the generative artificial intelligence model (310) of FIG. 3 or a cloud artificial intelligence (AI) model). The API / Plug-in management component (942) establishes a channel to communicate with the outside of the artificial intelligence framework (940) via an API, and through the established channel, it can access various data sources (e.g., a knowledge store (920)) (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2). Additionally, the API / plugin management component (942) may request the application / service component (930) (e.g., the processor (120) of FIG. 1) via the API if the application or service needs to perform an action that ultimately executes user input rather than an intermediate result. Information obtained from the outside may be used to generate a prompt in the prompt design component (941) along with user input, or it may be passed as input to a generative artificial intelligence model (960) (e.g., the artificial intelligence model (310) of FIG. 3 or a cloud artificial intelligence model).

[0125] An output modification component (or refiner component) (943) (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2) can finely tune the output of a generative artificial intelligence model (960) (e.g., the artificial intelligence model (310) of FIG. 3 or a cloud artificial intelligence model). For example, the output modification component (943) can verify whether the content generated through LLM, LVM, and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (943) can determine the extent to which the output matches the desired result and, if additional processing is required, proceed with that process. Furthermore, the output modification component (943) can configure and provide hints to the user to avoid unwanted output.

[0126] A generative AI model (960) (e.g., the AI ​​model (310) of FIG. 3 or a cloud AI model) generally refers to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (960) may include a model that generates images and / or a model that generates language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational autoencoders), and examples include diffusion-based generative models that use VAEs and transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. Additionally, there are LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and voice, and generate new data corresponding to them.

[0127] In one embodiment, the artificial intelligence framework (940) and / or generative artificial intelligence model (960) may be included within an artificial intelligence module (e.g., including a processing circuit) within the electronic device. For example, the artificial intelligence module may be operatively coupled with at least one processor of the electronic device (e.g., at least one processor (120) of FIG. 1 or processor (210) of FIG. 2). For example, the artificial intelligence module may be operatively coupled with a sensor hub of the electronic device for one or more sensors within the electronic device.

[0128] According to one embodiment, a method of operation in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) includes an operation of checking a first user query that is pre-fixed or pre-specified and a second user query associated with the first user query.

[0129] According to one embodiment, the method includes the operation of obtaining first information obtained by tokenizing the first user query and second information obtained by tokenizing the second user query in order to request processing of the second user query from an artificial intelligence model (e.g., artificial intelligence model (310) of FIG. 3). According to one embodiment, the first information is obtained from a prompt cache (e.g., prompt cache (320) of FIG. 3) of the memory of the electronic device (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2), and the second information is obtained through an operation to tokenize the second user query.

[0130] According to one embodiment, the method includes providing the first information and the second information to the artificial intelligence model, obtaining the first user query and the first response to the second user query through the artificial intelligence model, and storing the third information, which is tokenized from the second information and the first response, in the prompt cache.

[0131] According to one embodiment, the method includes the operation of verifying a received third user query after obtaining a first response to the first user query and the second user query.

[0132] According to one embodiment, the method includes the operation of obtaining the first information, the second information, and the third information from the prompt cache, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request the artificial intelligence model to process the third user query.

[0133] According to one embodiment, the method may further include obtaining a second response to the third user query through the artificial intelligence model.

[0134] According to one embodiment, while obtaining the second response, the operation may further include storing the fifth information tokenized from the third user query and the fifth information tokenized from the second response in the prompt cache.

[0135] According to one embodiment, the method may further include an operation of displaying the first response through a display of the electronic device or an operation of outputting the first response through a speaker of the electronic device.

[0136] According to one embodiment, the method may further include an operation of displaying the second response through a display of the electronic device or an operation of outputting the second response through a speaker of the electronic device.

[0137] According to one embodiment, the operation of obtaining the second information obtained by tokenizing the second user query may include, based on requesting processing of the second user query to the artificial intelligence model, the operation of separating text information of the second user query into at least one token by word through the artificial intelligence model, and the operation of obtaining the second information including a second key and a second value obtained through key and value operations on the tokens separated from the second user query based on an index assigned to the at least one token through the artificial intelligence model.

[0138] According to one embodiment, the operation of obtaining the fourth information obtained by tokenizing the third user query may include, based on requesting processing of the third user query to the artificial intelligence model, the operation of separating text information of the third user query into at least one token by word through the artificial intelligence model, and the operation of obtaining the fourth information including a fourth key and a fourth value obtained through key and value operations on the tokens separated from the third user query based on an index assigned to the at least one token through the artificial intelligence model.

[0139] According to one embodiment, the first information may include a first key and a first value obtained through an attention operation that checks the contextual association of at least one word included in the first user query.

[0140] According to one embodiment, the third information may include a third key and a third value obtained through an attention operation that checks the contextual association of at least one word included in the first response.

[0141] According to one embodiment, the fifth information may include a fifth key and a fifth value obtained through an attention operation that checks the contextual association of at least one word included in the first response.

[0142] According to one embodiment, in a non-transient storage medium storing one or more programs, the one or more programs include an executable instruction to cause the electronic device to perform an operation of verifying a first user query and a second user query related to the first user query when executed by at least one processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1 and electronic device (201) of FIG. 2).

[0143] According to one embodiment, the one or more programs include an executable instruction to cause the electronic device to perform an operation to obtain first information tokenized from the first user query and second information tokenized from the second user query in order to request processing of the second user query from an artificial intelligence model (e.g., artificial intelligence model (310) of FIG. 3) when executed by at least one processor of the electronic device. According to one embodiment, the first information is obtained from a prompt cache (e.g., prompt cache (320) of FIG. 3) of the memory of the electronic device (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2), and the second information is obtained through an operation to tokenize the second user query.

[0144] According to one embodiment, the one or more programs include an executable instruction to cause the electronic device, when executed by at least one processor of the electronic device, to provide the first information and the second information to the artificial intelligence model, and to obtain the first user query and the first response to the second user query through the artificial intelligence model, and to store the third information tokenized from the second information and the first response in the prompt cache.

[0145] According to one embodiment, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of verifying a received third user query after obtaining a first response to the first user query and the second user query.

[0146] According to one embodiment, the one or more programs include an executable instruction to cause the electronic device, when executed by at least one processor of the electronic device, to execute the operation of obtaining the first information, the second information, and the third information from the prompt cache, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request the artificial intelligence model to process the third user query.

[0147] According to one embodiment, the one or more programs include an executable instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of obtaining a second response to the third user query through the artificial intelligence model.

[0148] According to one embodiment of this document, information for tokenization operations to process user queries in an artificial intelligence model is stored in advance and reused when processing subsequent user queries, thereby avoiding repetitive operations and reducing the number of operations, which in turn improves processing speed. While memory usage (e.g., storage usage) increases compared to existing methods, where approximately 20MB is used per user query processing, this document can improve memory usage to approximately 50MB. In addition, various effects that can be identified directly or indirectly through this document may be provided. The effects obtainable from this disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this disclosure pertains from the description below.

[0149] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of the technology described in this document. Accordingly, the scope of this document should be interpreted to include all modifications or various other embodiments based on the technical concept of this document.

[0150] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0151] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0152] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0153] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0154] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0155] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101, 201), At least one processor (120, 210) including a processing circuit; and It includes memory (130, 220) for storing instructions, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Check a pre-specified first user query and a second user query related to the first user query, and In order to request processing of the second user query in the artificial intelligence model (310), first information obtained by tokenizing the first user query and second information obtained by tokenizing the second user query are obtained, the first information is obtained from the prompt cache (320) of the memory, and the second information is obtained through an operation to tokenize the second user query. The artificial intelligence model provides the first information and the second information, and through the artificial intelligence model, obtains the first user query and the first response to the second user query, and stores the third information, which is tokenized from the second information and the first response, in the prompt cache. After obtaining the first response to the first user query and the second user query, check the received third user query, and In order to request processing of the third user query from the artificial intelligence model, while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, the first information, the second information, and the third information are obtained from the prompt cache, and An electronic device that causes to obtain a second response to the third user query through the artificial intelligence model above.

2. In paragraph 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes to store in the prompt cache the fifth information tokenized from the third user query and the fifth information tokenized from the second response while obtaining the second response.

3. In claim 1 or 2, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes the first response to be displayed through a display of the electronic device or output through a speaker of the electronic device.

4. In any one of claims 1 to 3, when the instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes the second response to be displayed through the display or output through the speaker.

5. In any one of claims 1 to 4, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on requesting the processing of the second user query to the artificial intelligence model, the text information of the second user query is separated into at least one token for each word through the artificial intelligence model, and An electronic device that causes to obtain second information including a second key and a second value obtained through a key and value operation for tokens separated in the second user query based on an index assigned to at least one token through the artificial intelligence model.

6. In any one of claims 1 to 5, when the instructions are executed individually or collectively by the at least one processor, the electronic device: Based on requesting the processing of the third user query to the artificial intelligence model, the text information of the third user query is separated into at least one token per word through the artificial intelligence model, and An electronic device that causes to obtain the fourth information including a fourth key and a fourth value obtained through key and value operations for tokens separated in the third user query based on an index assigned to at least one token through the artificial intelligence model.

7. In any one of paragraphs 1 through 6, The electronic device comprising the first information including a first key and a first value obtained through an attention operation that verifies the contextual association of at least one word included in the first user query.

8. In any one of paragraphs 1 through 7, The third information includes a third key and a third value obtained through an operation that verifies the contextual association of at least one word included in the first response, and The electronic device comprising a fifth key and a fifth value obtained through an attention operation that verifies the contextual association of at least one word included in the second response, wherein the fifth information is the above-mentioned fifth information.

9. In a method of operation in an electronic device, An operation to verify a pre-fixed or pre-specified first user query and a second user query related to the first user query; To request processing of the second user query in an artificial intelligence model, the operation of obtaining first information that tokenizes the first user query and second information that tokenizes the second user query, wherein the first information is obtained from the prompt cache (320) of the memory, and the second information is obtained through an operation to tokenize the second user query; The operation of providing the first information and the second information to the artificial intelligence model, and, through the artificial intelligence model, obtaining the first user query and the first response to the second user query, and storing the third information obtained by tokenizing the second information and the first response in the prompt cache; An operation to verify a received third user query after obtaining a first response to the first user query and the second user query; An operation of obtaining the first information, the second information, and the third information from the prompt cache while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request processing of the third user query to the artificial intelligence model; and A method comprising the operation of obtaining a second response to the third user query through the artificial intelligence model.

10. In paragraph 9, the above method is, A method further comprising the operation of obtaining the second response, and storing the fifth information tokenized from the third user query and the fifth information tokenized from the second response in the prompt cache.

11. In paragraph 9 or 10, the above method is, The first response is displayed through the display of the electronic device or An operation to output the first response through the speaker of the electronic device; and Display the above second response through the display of the electronic device or A method further comprising the operation of outputting the second response through the speaker of the electronic device.

12. In any one of paragraphs 9 through 11, the operation of obtaining the second information obtained by tokenizing the second user query is, Based on requesting the artificial intelligence model to process the second user query, the operation of separating the text information of the second user query into at least one token per word through the artificial intelligence model; and A method comprising the operation of obtaining second information including a second key and a second value obtained through a key and value operation for tokens separated in the second user query based on an index assigned to at least one token through the artificial intelligence model.

13. In any one of paragraphs 9 through 12, the operation of obtaining the fourth information obtained by tokenizing the third user query is, Based on requesting the artificial intelligence model to process the third user query, the operation of separating the text information of the third user query into at least one token per word through the artificial intelligence model; and A method comprising the operation of obtaining the fourth information including the fourth key and the fourth value obtained through the operation of the key and value for the tokens separated from the third user query based on the index assigned to the at least one token through the artificial intelligence model.

14. In any one of paragraphs 9 through 13, The first information includes a first key and a first value obtained through an attention operation that checks the contextual association of at least one word included in the first user query, and The third information includes a third key and a third value obtained through an attention operation that verifies the contextual association of at least one word included in the first response, and A method comprising the fifth information including a fifth key and a fifth value obtained through an attention operation that verifies the contextual association of at least one word included in the first response.

15. In a non-transient storage medium storing one or more programs, the one or more programs, when executed by at least one processor of an electronic device, cause the electronic device: An operation to verify a pre-fixed or pre-specified first user query and a second user query related to the first user query; To request processing of the second user query in an artificial intelligence model, the operation of obtaining first information that tokenizes the first user query and second information that tokenizes the second user query, wherein the first information is obtained from the prompt cache (320) of the memory, and the second information is obtained through an operation to tokenize the second user query; The operation of providing the first information and the second information to the artificial intelligence model, and, through the artificial intelligence model, obtaining the first user query and the first response to the second user query, and storing the third information obtained by tokenizing the second information and the first response in the prompt cache; An operation to verify a received third user query after obtaining a first response to the first user query and the second user query; An operation of obtaining the first information, the second information, and the third information from the prompt cache while obtaining the fourth information obtained by tokenizing the third user query through an operation to tokenize the third user query, in order to request processing of the third user query to the artificial intelligence model; and A non-transient storage medium comprising executable commands to execute an operation to obtain a second response to a third user query through the artificial intelligence model.

Images