Method for processing user request using on-device ai model and electronic device for performing the same

Abstract

A method of processing a user request using a parameter efficient fine tuning (PEFT) model by an electronic device can include obtaining, by the electronic device, a prompt from a user, determining, by the electronic device, whether a PEFT model needs to be executed to process the prompt, executing, by the electronic device, a base model when it is determined that the PEFT model does not need to be executed, executing, by the electronic device, a user authentication when it is determined that the PEFT model needs to be executed, obtaining, by the electronic device, at least one matrix corresponding to the PEFT model when the user authentication is successful, and executing, by the electronic device, the PEFT model by using the at least one matrix.

Description

Technical Field

[0001] This disclosure relates to a method for processing user requests using an on-device artificial intelligence (AI) model and an electronic device for performing the method, and more specifically, to a method for enhancing security by encrypting (or locking) components used to generate a query key-value / weight (QKV / W) matrix required to execute (or run) the AI ​​model on the device. Background Technology

[0002] Recently, the approach of pre-training large models using a general dataset and then fine-tuning the pre-trained large model according to the individual needs of the user has been widely adopted. However, training all parameters of a large model during fine-tuning consumes too much time and computational resources. Therefore, parameter efficient fine-tuning (PEFT) techniques have been developed to improve efficiency by training only some of the parameters.

[0003] When fine-tuning large models using these PEFT techniques, users' efforts, expertise, and personal information can be utilized. Therefore, users may not want others to use the fine-tuned model without permission, and it is also necessary to maintain the security of the fine-tuned model in terms of protecting personal information. Summary of the Invention

[0004] Solution to the problem

[0005] According to one aspect of this disclosure, a method for an electronic device to process a user request using a Parametrically Efficient Fine-Tuning (PEFT) model includes: the electronic device receiving a prompt from a user; the electronic device determining whether it is necessary to execute the PEFT model to process the prompt; based on the determination that it is not necessary to execute the PEFT model, the electronic device executing a foundation model; based on the determination that it is necessary to execute the PEFT model, the electronic device performing user authentication; based on successful user authentication, the electronic device obtaining at least one matrix corresponding to the PEFT model; and the electronic device using the at least one matrix to execute the PEFT model.

[0006] According to one aspect of this disclosure, an electronic device includes: a memory storing a program or at least one instruction; and at least one processor configured to execute the program or at least one instruction individually or jointly, wherein, when executed individually or jointly by the at least one processor, the program or at least one instruction causes the electronic device to: receive a prompt from a user; determine whether to execute the PEFT model to process the prompt based on determining that a Parameter Efficient Fine-Tuning (PEFT) model needs to be executed; execute a base model based on determining that the PEFT model does not need to be executed; perform user authentication based on successful user authentication; obtain at least one matrix corresponding to the PEFT model; and use the at least one matrix to execute the PEFT model.

[0007] According to one aspect of this disclosure, a method for an electronic device to process a user request using a Parametric Efficient Fine-Tuning (PEFT) model includes: the electronic device receiving a prompt from a user; the electronic device determining whether it is necessary to execute a PEFT model to process the prompt; based on the determination that it is not necessary to execute a PEFT model, the electronic device executing a base model; based on the determination that it is necessary to execute a PEFT model, the electronic device performing user authentication; based on successful user authentication, the electronic device obtaining at least one matrix corresponding to the PEFT model by generating a query key-value / weight (QKV / W) matrix, storing the QKV / W matrix in a cache memory of the electronic device, and providing the QKV / W matrix to the PEFT model; and the electronic device using the at least one matrix to execute the PEFT model. Attached Figure Description

[0008] The above and other aspects and features of certain embodiments of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, in which: Figure 1 This is a diagram illustrating the process of fine-tuning the base model using the Parametric Efficient Fine-Tuning (PEFT) technique; Figure 2 This is a diagram illustrating the process of decomposing the weight matrix used to generate the query key-value (QKV) / weight (W) matrix into a low-dimensional matrix, and then fine-tuning the low-dimensional matrix using the low-rank adaptation (LoRA) method. Figure 3 This is a diagram illustrating modules included in an electronic device according to an embodiment of the present disclosure; Figure 4 This is a diagram illustrating hardware components included in an electronic device according to embodiments of the present disclosure; Figure 5 and Figure 6 This is a diagram illustrating the process by which an electronic device according to an embodiment of the present disclosure activates a PEFT model by generating a QKV / W matrix based on user authentication results; Figure 7and Figure 8 This is a diagram illustrating the process by which an electronic device, according to an embodiment of the present disclosure, activates a PEFT model by caching a weight matrix based on user authentication results; Figure 9 and Figure 10 This is a diagram illustrating the process by which an electronic device according to an embodiment of the present disclosure activates a PEFT model by caching a QKV / W matrix based on user authentication results; and Figures 11 to 19 This is a flowchart illustrating a method for processing user requests using a PEFT model according to an embodiment of the present disclosure. Detailed Implementation

[0009] Throughout the disclosure, the expressions “at least one of a, b, or c” and “at least one or more of a, b, and c” mean only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0010] In describing this disclosure, descriptions of technical concepts well-known in the art to which this disclosure pertains and not directly related to this disclosure will be omitted. This is to more clearly convey the essence of this disclosure without obscuring it by omitting unnecessary descriptions. Furthermore, the terminology used below is defined by consideration of the functions described in this disclosure and may be changed according to the intent, practice, etc., of the user or operator. Therefore, the definition of terminology should be based on the overall description of this disclosure.

[0011] For the same reason, some components are shown enlarged, omitted, or schematically in the accompanying drawings. Furthermore, the dimensions of each component do not perfectly reflect the actual dimensions. Throughout the drawings, the same reference numerals always denote the same or corresponding elements.

[0012] The features of this disclosure and its implementation methods will be more readily understood by referring to the following description and accompanying drawings of embodiments thereof. However, this disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth below. Rather, embodiments of this disclosure are provided to make this disclosure thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art to which this disclosure pertains. Embodiments of this disclosure may be defined by the appended claims. Throughout the specification, the same reference numerals denote the same elements. Furthermore, in the following description of this disclosure, relevant functions or configurations will not be described in detail where it is determined that such details would obscure the essence of this disclosure. Additionally, the terminology used below is defined by consideration of the functions described in this disclosure and may change according to the intent, practice, etc., of the user or operator. Therefore, the definition of terminology should be based on the overall description of this disclosure.

[0013] Terms such as “unit,” “module,” “component,” and “block” can refer to hardware or software. As used herein, multiple “units,” “modules,” “components,” and “blocks” can be implemented as a single component, or a single “unit,” “module,” “component,” and “block” can include multiple components.

[0014] It will be understood that when an element is referred to as being “connected” or “attached” to another element, it can be directly or indirectly connected to the other element, wherein an indirect connection may include a “connection via a wireless communication network”.

[0015] Furthermore, when a component "comprises" or "includes" an element, the component may also include other elements without excluding them, unless there is a specific description to the contrary.

[0016] It will be understood that although the terms “first,” “second,” “third,” etc., may be used herein to describe various elements, this disclosure should not be limited by these terms. These terms are used only to distinguish one element from another.

[0017] As used herein, the singular forms “a,” “one,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0018] Identification codes may be used for the purposes of description of any method or process described herein, but are not intended to indicate the order of each step or operation. Each step or operation may be performed in a different order than that shown unless the context clearly indicates otherwise. One or more steps or operations may be omitted unless the context of this disclosure clearly indicates otherwise.

[0019] The various actions, behaviors, blocks, steps, etc. in the flowchart can be executed in the order they are presented, in different orders, or simultaneously. Furthermore, in one or more embodiments, some of the actions, behaviors, blocks, steps, etc., may be omitted, added, modified, or skipped without departing from the scope of this disclosure.

[0020] In embodiments of this disclosure, each block and combination of blocks in the flowchart illustrations can be executed by computer program instructions. These computer program instructions can be loaded into a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, and the instructions, which are executed by the processor of the computer or other programmable data processing apparatus, can generate units for performing the functions specified in the flowchart blocks. The computer program instructions can also be stored in a computer-executable or computer-readable storage medium capable of instructing the computer or other programmable data processing apparatus to perform functions in a particular manner, and the instructions stored in the computer-executable or computer-readable storage medium can produce an article of writing including instructions for performing the functions specified in the flowchart blocks. The computer program instructions can also be loaded into a computer or other programmable data processing apparatus.

[0021] Furthermore, each block of the flowchart may represent a module, code segment, or code section comprising one or more executable instructions for performing a specified logical function. In embodiments of this disclosure, the functions mentioned in a block may occur out of order. For example, two blocks shown consecutively may execute substantially simultaneously, or these blocks may sometimes execute in reverse order depending on their corresponding functions.

[0022] As used in embodiments of this disclosure, the term "...unit" refers to a software element or hardware element, such as a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC), and is capable of performing a predetermined function. However, the term "...unit" is not limited to software or hardware. "...unit" may be configured to reside in addressable storage media or to operate one or more processors. In embodiments of this disclosure, the term "...unit" may include elements such as software elements, object-oriented software elements, class elements, and task elements, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and parameters. The functionality provided by a particular element or a particular "...unit" may be combined to reduce the number of elements or may be further divided into other elements. Furthermore, in embodiments of this disclosure, "...unit" may include one or more processors.

[0023] The meanings of the terms used in this article are described below.

[0024] The term "base model" can refer to a model that can be universally used across a variety of tasks after being trained using a large-scale dataset. Furthermore, the term "pre-trained model" can refer to a model that has already learned general knowledge or patterns using a large-scale dataset. In this disclosure, the terms "base model" and "pre-trained model" are used interchangeably, and they can be trained through fine-tuning to suit a specific task.

[0025] The term "Parameter Efficient Fine-Tuning (PEFT) Model" can refer to a model that has been fine-tuned using the PEFT technique. Similarly, "Low-Rank Adaptation (LoRA) Model" can refer to a model that has been fine-tuned using the LoRA technique. PEFT is one of the techniques used to perform fine-tuning and is an efficient technique that can reduce computational resources and processing time by updating only some parameters of a pre-trained model instead of updating all of its parameters. Terms such as "personalized model" or "user-specific AI model" can be used instead of "PEFT model".

[0026] A "PEFT layer" can refer to a layer added as a result of fine-tuning a base model using the PEFT technique. In other words, a PEFT layer can refer to an additional layer included in a PEFT model, compared to the base model. PEFT layers may include additional parameters learned during fine-tuning.

[0027] To summarize the relationship between the PEFT model and the PEFT layer, the PEFT model is the result of adding the PEFT layer to the base model.

[0028] The term "Query Key-Value (QKV) matrix" can encompass all concepts used in attention mechanisms, including the Q, K, and V matrices. A QKV matrix can be obtained by applying a weight matrix to the input embedding.

[0029] The term "weight matrix" refers to the matrix used to generate the QKV matrix. The weight matrix can be generated (updated) during the fine-tuning of the base model. The QKV matrix can be generated by multiplying the input embedding X by the weight matrix. For example, the input embedding X can be multiplied by the query's weight matrix W. Q This is used to generate the Q matrix, and similarly, it can be achieved by multiplying the input embedding X by the key weight matrix W. K The weight matrix W of the sum V To obtain the remaining K and V matrices respectively.

[0030] The "QKV matrix" and the "weight matrix" can be combined and collectively referred to as the "QKV / weight (W) matrix".

[0031] A "hint" is a sentence or keyword used for interaction between a user and a model, and can be text used by a user to provide a question or give a command to the model. In other words, a hint can be text or other form of input that instructs the model to generate a particular output. As used in this disclosure, "execute hint" or "execute (or run) a generative model according to a hint" can refer to an operation by which a generative model performs a task in response to a hint, that is, in response to a hint being input to the generative model, the generative model performs an operation to generate a result corresponding to the hint. Hints can include "intent" and "details," as described in detail below. Terms such as "instruction" may also be used instead of "hint."

[0032] "Input data" can refer to the actual data that the model needs to process or analyze. Input data can take various forms, such as text, images, and audio. For example, when a user requests a translation by inputting the prompt "Translate the following sentence into Korean," the text to be translated can be considered input data. Alternatively, for example, when a user requests to edit an image by inputting the prompt "Erase the clouds in the sky," the image to be edited can be considered input data. Terms such as "source data" or "input values" can also be used instead of "input data."

[0033] An "input sequence" is the actual input fed into the model and can refer to the complete input that the model needs to process. In other words, an input sequence can refer to all the data sent to the input layer of the model, and can include not only text prompts but also other forms of input data, such as images, audio, etc. In other words, an input sequence can be a combination of prompts and input data. For example, the input sequence of a text-to-image model can include the image data to be edited and prompts (text) indicating how to edit the image data.

[0034] As a concrete example, when a user inputs the sentence "He always inspires me" as input data to the model, along with the prompt "Translate the following sentence into Korean," the input sequence could be "Translate the following sentence into Korean. He always inspires me." Alternatively, when a user inputs an image to be edited with the prompt "Erase the clouds in the sky" as input data, the input sequence could be a combination of "Erase the clouds in the sky in the photo" and the image. Alternatively, in this case, the image used as input data could be converted to text, and the input sequence could be a combination of that text and the prompt. Terms such as "complete input," "input stream," or "input string" can also be used instead of "input sequence."

[0035] "Input embedding" can refer to the embedding matrix corresponding to the input sequence. In other words, the embedding matrix obtained as a result of performing embedding transformations on the tokens included in the input sequence can be the input embedding. The input sequence is the entire input to the model and can include only textual prompts entered by the user, or it can include prompts and data such as images or audio input along with the prompts.

[0036] Embodiments of this disclosure are described in detail below with reference to the accompanying drawings.

[0037] This disclosure relates to a method for processing user requests using a PEFT model and an electronic device for performing the method, and embodiments of this disclosure provide enhanced security by encrypting (or locking) the components required to perform the PEFT model.

[0038] Electronic devices according to embodiments of this disclosure enable the execution (or operation) of a PEFT model only when user authentication is successful, thereby increasing the security of the user's personal data (e.g., privacy-related information) and the structure of the model, as described in detail below.

[0039] PEFT models can reflect a user's personal data. In addition to the layers of the base model (the pre-trained model), a PEFT model can also include PEFT layers, and the parameters included in the PEFT layers can be generated (updated) by reflecting the user's personal data during the fine-tuning of the base model.

[0040] User personal data stored in electronic devices can also be used during the execution of a PEFT model. For example, the PEFT layers included in a PEFT model can use the user's personal data to generate a QKV / W matrix.

[0041] In this way, since the user's personal data may already be reflected in the PEFT model or used in the process of executing the PEFT model, the electronic device according to embodiments of this disclosure can enhance data security by executing (or running) the PEFT model only when the user authentication is successful.

[0042] The features of the embodiments of this disclosure are briefly summarized as follows: (A) Determine whether the PEFT model needs to be executed to process the user's request: - When the PEFT model needs to be executed, the electronic device performs user authentication.

[0043] When the PEFT model is not required, electronic devices use the base model to handle user requests.

[0044] (B) When user authentication is successful, the user's request is processed by executing the PEFT model: - Electronic devices encrypt the components required to execute the PEFT model, and decrypt the components to execute the PEFT model when the user is successfully authenticated.

[0045] - For example, the component used to generate the QKV / W matrix required to perform the PEFT model waits in a locked state, and is unlocked to generate the QKV / W matrix when the user authentication is successful.

[0046] - Alternatively, for example, the weight matrix corresponding to the PEFT model, i.e., the weight matrix used to generate the QKV matrix required to perform the PEFT model, is encrypted and stored in flash memory, and when the user authentication is successful, the encrypted weight matrix is ​​decrypted, stored in cache memory, and used to generate the QKV matrix.

[0047] (C) When user authentication fails, the user's request is handled by executing a dummy model: When user authentication fails, the electronic device generates a random weight matrix based on a random seed and uses the generated weight matrix to generate a QKV matrix (in this case, the PEFT model operates as a virtual model).

[0048] Before describing specific embodiments of this disclosure, refer to Figure 1 and Figure 2 This describes a method for fine-tuning the base model using the PEFT technique and the additional layers generated as a result of the fine-tuning. Then, refer to... Figure 3 and Figure 4 The configuration and overall operation of an electronic device according to embodiments of the present disclosure are described, and references are made to... Figures 5 to 8 Specific embodiments of this disclosure are described below.

[0049] 1. A method for fine-tuning based on PEFT technology ( Figure 1 and Figure 2 )

[0050] Figure 1 This is a diagram illustrating the process of fine-tuning the base model using the PEFT technique.

[0051] refer to Figure 1 The base model 10 may include N layers, namely, layer 11 to layer N 13, and each of layer 11 to layer N 13 may include a weight matrix. For example, at least one of layer 11, layer 12, ..., or layer N 13 may include a weight matrix W for generating the QKV matrix. Q W K and W V .

[0052] As mentioned above, the base model 10 can be a neural network model pre-trained using a general dataset. Users can fine-tune the base model 10 to suit specific tasks and use the PEFT technique to perform fine-tuning efficiently.

[0053] PEFT is a method of fine-tuning a pre-trained large-scale model to suit a specific task by adjusting (or updating) only some parameters of the pre-trained model, rather than adjusting all of its parameters. In other words, PEFT is a fine-tuning method that fixes the parameters of a large pre-trained model, adds new trainable neural network structures, and updates only the added parts.

[0054] Since the base model 10 is a large-scale model, a large amount of computing resources and memory are required to perform fine-tuning to update all the parameters included in the base model 10, and the PEFT technique is proposed to overcome this limitation.

[0055] When fine-tuning the base model 10 using PEFT techniques, a first PEFT model 20 or a second PEFT model 30 can be generated. Various types of PEFT techniques exist, such as adapter tuning methods, LoRA methods, prefix tuning methods, and cue tuning methods. Figure 1 In the illustrated embodiment, it is assumed that the first PEFT model 20 is generated using an adapter tuning method, and the second PEFT model 30 is generated using the LoRA method.

[0056] The adapter tuning method involves adding a small network (adapter) to an existing model (base model) and training only the parameters of the added adapter while keeping the parameters of the existing model unchanged.

[0057] refer to Figure 1 An adapter layer 21 is added between the first layer 11 and the second layer 12 of the first PEFT model 20 generated using the adapter tuning method. According to the adapter tuning method, only the parameters of the adapter layer 21 can be updated, while the parameters of the first to Nth layers 11, 12 and 13 of the existing base model 10 remain frozen.

[0058] According to embodiments of this disclosure, adapter layer 21 may include a weight matrix for generating the QKV matrix required to perform the first PEFT model 20.

[0059] The LoRA method involves decomposing the high-dimensional weight matrices in the layers of an existing model (base model) into low-dimensional weight matrices and training the decomposed low-dimensional weight matrices. In other words, according to the LoRA method, the parameters of the existing model (base model) are frozen, and a new low-rank matrix is ​​defined such that only the parameters of the corresponding matrix are updated.

[0060] When fine-tuning is performed according to the LoRA method, layers can be added by decomposing at least one of the first to Nth layers 11, 12, and 13 included in the base model 10 into a lower-dimensional space. (See reference) Figure 1 The second PEFT model 30 may include a 2-1 layer 31 and a 2-2 layer 32 generated by decomposing the second layer 12 of the base model 10 into a low-dimensional space. Similar to the second layer 12, the other layers 11 and 13 included in the base model 10 can be decomposed into a low-dimensional space and included in the second PEFT model 30. According to the LoRA method, only the parameters of the 2-1 layer 31 and the 2-2 layer 32 can be updated, while the parameters of the existing first to Nth layers 11, 12, and 13 of the base model 10 remain frozen.

[0061] According to embodiments of this disclosure, layers 2-1 31 and 2-2 32 may include weight matrices for generating the QKV matrix required to perform the second PEFT model 30. For example, layer 12 may include a weight matrix W for generating the QKV matrix. Q W K W V Furthermore, layers 2-1 (31) and 2-2 (32) may include weighting the weight matrix W. Q W K and W V Each of the matrices is obtained by decomposing it into a low-dimensional matrix. (See reference.) Figure 2 This will be described in detail.

[0062] Figure 2 This is a diagram illustrating the process of decomposing the weight matrix used to generate the QKV matrix into a low-dimensional matrix, and then fine-tuning the low-dimensional matrix using the LoRA method.

[0063] Figure 2 The weight matrix W used to generate the QKV matrix is ​​shown. Q 210, W K 220 and W V 230. Weight matrix W Q 210, W K 220 and W V 230 can be included in at least one of the first layer 11, the second layer 12, ..., or the Nth layer 13 of the base model 10.

[0064] refer to Figure 2 The weight matrix W used to generate the QKV matrix Q 210, W K 220 and W V Each of 230 can be decomposed into a low-dimensional matrix. The weight matrix W is used to generate the Q matrix. Q 210 can be decomposed into two low-dimensional matrices WQA 211 and W QB 212. Similarly, the weight matrix W used to generate the K matrix... K 220 can be decomposed into two low-dimensional matrices W KA 221 and W KB 222, and the weight matrix W used to generate the V matrix. V 230 can be decomposed into two low-dimensional matrices W VA 231 and W VB 232.

[0065] When fine-tuning the model using the LoRA method, only the decomposed low-dimensional matrix W needs to be updated. QA 211, W QB 212, W KA 221, W KB 222, W VA 231 and W VB 232, while the existing model (basic model) includes the weight matrix W. Q 210, W K 220 and W V 230 remains frozen.

[0066] The low-dimensional matrices updated as a result of fine-tuning are respectively derived from W QA ' 213, W QB ' 214, W KA ' 223, W KB ' 224, W VA ' 233 and W VB ' 234 indicates. W QA ' 213 and W QB ' The product of 214 is expressed in W. Q 'Indicates, W KA ' 223 and W KB ' The product of 224 is expressed in W. K ' indicates, and W VA ' 233 and W VB ' The product of 234 is expressed in W. V 'express.

[0067] W Q '、W K 'and W V'It can be the weight matrix corresponding to the PEFT model, and it can be the same as the weight matrix W in the base model. Q 210, W K 220 and W V 230 are used together to generate the QKV matrix. For example, this can be achieved by separately adding W... Q '、W K 'and W V 'Add to W Q 210, W K 220 and W V The obtained matrix is ​​used for input embedding to generate a QKV matrix.

[0068] According to embodiments of this disclosure, a QKV / W generation module, serving as a generative model (e.g., a language model) trained during fine-tuning, can generate a QKV / W matrix. For example, by learning the QKV / W matrix generated during fine-tuning of a base model into a PEFT model, the generative model (QKV / W generation module) can generate the QKV / W matrix required for performing the PEFT model. The QKV / W generation module can be encrypted and then decrypted upon successful user authentication to generate the QKV / W matrix.

[0069] According to embodiments of this disclosure, the updated weight matrix W is obtained by fine-tuning the base model using the LoRA method. Q '、W K 'and W V It can be encrypted and then decrypted and used when the PEFT model is executed. See below for reference. Figure 3 , Figure 5 and Figure 6 Describe the process in detail.

[0070] 2. Description of the configuration and overall operation of the electronic device running the PEFT model ( Figure 3 and Figure 4 )

[0071] Figure 3 This is a diagram illustrating modules included in an electronic device according to embodiments of the present disclosure. Reference Figure 3 An electronic device 1000 according to an embodiment of the present disclosure may include an input processing module 310, an authentication module 320, a PEFT model activation module 330, a flash memory 340, and a cache memory 350. Furthermore, the electronic device 1000 may include a PEFT model 30, and the PEFT model 30 may include a base model 10 and a PEFT layer 360.

[0072] Figure 3The components included in the electronic device 1000, namely, PEFT model 30, input processing module 310, authentication module 320, PEFT model activation module 330, flash memory 340 and cache memory 350, are classified based on their functions or roles. Figure 3 The components of the electronic device 1000, namely the PEFT model 30 to the cache memory 350, can be software components implemented by the processor 1300 of the electronic device 1000 executing a program stored in the memory 1400, as shown in the following reference. Figure 4 The described, or possibly virtual, component does not have an actual matching hardware device. In other words, the operations performed by the processor 1300 of electronic device 1000 executing programs or instructions stored in memory 1400 can be categorized into multiple groups according to function or purpose, and entities performing operations respectively included in multiple groups can be represented as... Figure 3 The components shown are, namely, PEFT model 30 to cache memory 350. Therefore, it is described as being composed of... Figure 3 The operations performed by the components of the electronic device 1000 (i.e., PEFT model 30 to cache memory 350) can be considered to be actually executed by the processor 1300 of the electronic device 1000 executing the program or instructions stored in memory 1400.

[0073] although Figure 3 An electronic device 1000 is shown comprising all components, namely, PEFT model 30 to cache memory 350; however, the electronic device 1000 is not limited thereto, and at least some of the components may be implemented as being included in a separate device, or one component may be implemented as being included in another component. In this way, according to embodiments of this disclosure, the components included in the electronic device 1000 (i.e., PEFT model 30 to cache memory 350) may be hardware or software components, and the electronic device 1000 may be implemented in the form of various electronic devices (e.g., one electronic device or a combination of two or more electronic devices).

[0074] According to embodiments of this disclosure, the electronic device 1000 may be a user's terminal (e.g., a smartphone, laptop, desktop computer, etc.) or a server that communicates with the user's terminal, but this disclosure is not limited thereto.

[0075] The input processing module 310 is a component used to determine the model to be executed based on the input from user 1. The electronic device 1000 can receive prompts from user 1 via the input / output (I / O) interface 1200 requesting the execution of a specific function (e.g., an AI function), as referenced below. Figure 4Described. For example, electronic device 1000 may receive text prompts from user 1 via a touchscreen or voice prompts from user 1 via a microphone. Furthermore, electronic device 1000 may receive prompts from user 1 in various other ways.

[0076] The input processing module 310 can determine whether the PEFT model 30 needs to be used to perform the task based on the prompts input by user 1. In other words, the input processing module 310 can determine whether the PEFT model 30 needs to be used to process the prompts received from user 1. For example, the input processing module 310 can determine whether the PEFT model 30 needs to be used based on the prompts received from user 1. Alternatively, for example, user 1 can request the use of the PEFT model 30 when inputting the prompts, and the input processing module 310 can determine that the PEFT model 30 needs to be used based on such a request.

[0077] User 1 can pre-store multiple PEFT models in electronic device 1000 and use them depending on the purpose. Although Figure 3 The electronic device 1000 is shown to include a PEFT model 30, but the electronic device 1000 may also include one or more PEFT models, and the input processing module 310 may determine which of the multiple PEFT models is needed to process the prompt from the user 1.

[0078] When the input processing module 310 determines that the PEFT model 30 is not needed to process the prompts from user 1, the input processing module 310 can request the base model 10 to perform the function requested by user 1. That is, the input processing module 310 can embed the input generated based on the prompts from user 1 through only the first to the Nth layers 11, 12 and 13 of the base model 10, so that computation can be performed.

[0079] As referenced above Figure 1 and Figure 2 As described, when fine-tuning the base model 10 according to the PEFT technique, the PEFT model can be generated in such a way that new layers (PEFT layers) are added while the first to Nth layers 11, 12, and 13 of the base model 10 remain unchanged. Therefore, the input processing module 310 can enable the processing of input embeddings by using only the first to Nth layers 11, 12, and 13 of the base model 10 (excluding the PEFT layer 360).

[0080] When it is determined that PEFT model 30 is needed to process the prompt from user 1, input processing module 310 may request authentication module 320 to perform user authentication. According to embodiments of this disclosure, electronic device 1000 may execute (or run) PEFT model 30 only if it is determined that user 1 has permission to execute (or run) PEFT model 30. Input processing module 310 may request authentication from authentication module 320 to verify whether user 1 has permission to execute PEFT model 30.

[0081] The authentication module 320 is a component used to perform user authentication. The authentication module 320 can authenticate user 1 to verify whether user 1 has the permission to execute PEFT model 30.

[0082] According to embodiments of this disclosure, authentication for verifying whether user 1 has permission to execute PEFT model 30 can be performed in a different manner than authentication for using electronic device 1000. This will be described in detail using a first authentication method with relatively low security (e.g., authentication method using account identifier (ID), password, etc.) and a second authentication method with relatively high security (e.g., biometric authentication method). Various authentication methods can be used as both the first and second authentication methods.

[0083] When user 1 starts using electronic device 1000 and unlocks electronic device 1000 according to the first authentication method, authentication module 320 can perform additional authentication on user 1 according to the second authentication method when PEFT model 30 needs to be executed.

[0084] However, when user 1 starts using electronic device 1000 and unlocks electronic device 1000 according to the second authentication method, authentication module 320 can determine that user 1 has the right to execute PEFT model 30 without performing additional authentication on him (i.e., authentication module 320 can send (i.e. provide) an authentication result as "success" to PEFT model activation module 330).

[0085] The PEFT model activation module 330 is a component used to activate the PEFT model 30 based on the authentication result. When the authentication result for user 1 is received from the authentication module 320, the PEFT model activation module 330 determines whether to activate the PEFT model 30 based on the authentication result.

[0086] When authentication for user 1 is successful, the PEFT model activation module 330 can execute the PEFT model 30.

[0087] According to embodiments of this disclosure, the PEFT model activation module 330 can generate the QKV / W matrix required to execute the PEFT model 30. As described above, because the QKV / W generation module included in the PEFT model activation module 330 learns the QKV / W matrix generated during fine-tuning of the PEFT model 30, the QKV / W generation module can generate the QKV / W matrix based on prompts input by user 1. Specifically, during the fine-tuning of the base model 10 using PEFT technology, a generative model can be trained to generate the QKV / W matrix required to execute the PEFT model 30, and the trained generative model (QKV / W generation module) can be included in the PEFT model activation module 330 in an encrypted state. When user authentication is successful, the generative model (QKV / W generation module) included in the PEFT model activation module 330 can be decrypted to generate the QKV / W matrix.

[0088] The PEFT model activation module 330 can temporarily store the generated QKV / W matrix in the cache memory 350, allowing the PEFT model 30 to use the QKV / W matrix. When generating the QKV / W matrix, the PEFT model activation module 330 can also use the user 1's personal data stored in the flash memory 340. The PEFT model activation module 330 can also be implemented as an integral part of the PEFT model 30.

[0089] According to embodiments of this disclosure, flash memory 340 can store a weight matrix for generating a QKV matrix, and PEFT model activation module 330 can cache the weight matrix stored in flash memory 340. Specifically, PEFT model activation module 330 can temporarily store the weight matrix corresponding to PEFT model 30 stored in flash memory 340 in cache memory 350, so that PEFT model 30 can use the weight matrix whenever it needs it.

[0090] The cached weight matrix can be the matrix required to perform the PEFT model 30. That is, the weight matrix generated as a result of fine-tuning the base model 10 using the PEFT technique can be stored in the flash memory 340 and cached by the PEFT model activation module 330.

[0091] According to embodiments of this disclosure, user 1’s personal data can be used in the process of fine-tuning the base model 10 using the PEFT technique. Therefore, user 1’s personal data can be reflected in the weight matrix corresponding to the PEFT model 30.

[0092] According to an embodiment of this disclosure, the weight matrix is ​​encrypted and stored in flash memory 340, decrypted by PEFT model activation module 330, and stored in cache memory 350, so that PEFT model 30 can use the weight matrix.

[0093] As mentioned above, the weight matrix stored in flash memory 340 can be W Q′ W K′ and W V′ These are the matrices used to generate the QKV matrix required to execute the PEFT model 30. Therefore, the PEFT layer 360 can use the weight matrix W temporarily stored in the cache memory 350. Q '、W K 'and W V To generate a QKV matrix, and use the generated QKV matrix to perform an attention mechanism.

[0094] Instead of the weight matrix W used to generate the QKV matrix Q '、W K 'and W V Other types of weight matrices can be stored in flash memory 340 and cached by the PEFT model activation module 330. However, the weight matrix cached by the PEFT model activation module 330 can be a weight matrix corresponding to the PEFT model 30. In other words, the weight matrix cached by the PEFT model activation module 330 can be a weight matrix generated as a result of fine-tuning the base model 10 according to the PEFT technique.

[0095] According to embodiments of this disclosure, the PEFT model activation module 330 can generate a QKV / W matrix based on the user 1's personal data stored in the flash memory 340, and cache the generated QKV / W matrix. That is, the PEFT model activation module 330 can temporarily store the generated QKV / W matrix in the cache memory 350, and allow the PEFT model 30 to use the stored QKV / W matrix whenever needed. The PEFT model activation module 330 can also use the weight matrix W stored in the flash memory 340. Q '、W K 'and W V Generate a QKV matrix based on user 1's personal data.

[0096] The PEFT layer 360 of the PEFT model 30 may include a layer for performing an attention mechanism. Therefore, the attention mechanism can be performed using the QKV / W matrix stored in the cache memory 350, and the result of performing the attention mechanism can be provided to another layer included in the PEFT layer 360, or to one of the first to Nth layers 11, 12 and 13 included in the base model 10.

[0097] Alternatively, a layer that is a combination of at least one of the first layer 11, second layer 12, ..., or Nth layer 13 included in the base model 10 with the PEFT layer 360 may use a QKV / W matrix stored in the cache memory 350 to perform an attention mechanism and provide the result of the attention mechanism to another layer. In various other ways, the combination of the PEFT layer 360 and the base model 10 may use a QKV / W matrix to perform an attention mechanism and perform inference based on the result of the attention mechanism.

[0098] When authentication for user 1 fails, the PEFT model activation module 330 can generate a random weight matrix based on a random seed and provide the generated weight matrix to the PEFT layer 360.

[0099] In this way, if the generative model (QKV / W generation module) is not used to generate the QKV / W matrix during fine-tuning, the effect of fine-tuning is not reflected, and the PEFT model 30 operates as a virtual model. Alternatively, when the PEFT layer 360 uses a random weight matrix other than the weight matrix generated via fine-tuning to perform computations (e.g., computations for generating the QKV matrix), the effect of fine-tuning is not reflected, and the PEFT model 30 operates as a virtual model. Therefore, the security of the PEFT model 30 can be maintained.

[0100] When the processing of the prompt input by user 1 is completed, the PEFT model activation module 330 can delete the data (e.g., QKV / W matrix, etc.) stored in the cache memory 350 for executing the PEFT model 30.

[0101] The following is for reference. Figures 5 to 8 The process of the PEFT model activation module 330 executing the PEFT model 30 based on the authentication result is described in detail.

[0102] Figure 4 This is a diagram illustrating hardware components included in an electronic device according to embodiments of the present disclosure. Reference Figure 4 According to embodiments of this disclosure, electronic device 1000 may include a communication interface 1100, an input / output (I / O) interface 1200, a processor 1300, and a memory 1400. However, the components of electronic device 1000 are not limited to the examples described above, and electronic device 1000 may include more or fewer components than those described. Some or all of the communication interface 1100, I / O interface 1200, processor 1300, and memory 1400 may be implemented as a single chip.

[0103] The communication interface 1100 can be a component for sending signals (control commands, data, etc.) to and receiving signals (control commands, data, etc.) from external devices, either wired or wirelessly, and can be implemented as a communication chipset supporting various communication protocols. The communication interface 1100 can receive signals from the outside and output signals to the processor 1300, or it can send signals output from the processor 1300 to the outside. The electronic device 1000 can communicate with external devices via the communication interface 1100.

[0104] I / O interface 1200 may include an input interface (e.g., touchscreen, keyboard, microphone, etc.) for receiving control commands or information from user 1, and an output interface (e.g., display panel, speaker, etc.) for displaying the results of the operation performed according to the user's commands or the status of electronic device 1000. According to embodiments of this disclosure, electronic device 1000 may receive prompts and input data from user 1 via I / O interface 1200, and when a task is completed, output the results of the task performance via I / O interface 1200 (e.g., answers to questions included in the prompts, images edited according to requests in the prompts, etc.).

[0105] Processor 1300 may be a component that controls a series of processes to cause electronic device 1000 to perform operations according to embodiments of the present disclosure as described below, and may include one or more processors. The one or more processors included in processor 1300 may be circuits such as a system-on-chip (SoC), integrated circuit (IC), etc. The one or more processors included in processor 1300 may be general-purpose processors (such as central processing unit (CPU), microprocessor unit (MPU), application processor (AP), digital signal processor (DSP), etc.), dedicated graphics processors (such as graphics processing unit (GPU) and vision processing unit (VPU)), dedicated AI processors (such as neural processing unit (NPU)), or dedicated communication processors (such as communication processor (CP)). When the one or more processors included in processor 1300 are dedicated AI processors, the corresponding dedicated AI processor may be designed with a hardware architecture specifically designed to process a particular AI model.

[0106] Processor 1300 can write data to or read data stored in memory 1400, and specifically, execute a program or at least one instruction stored in memory 1400 to process data according to predefined operating rules or an AI model. Therefore, processor 1300 can perform the operations described below according to embodiments of this disclosure, and unless otherwise stated, operations described in this disclosure as being performed by electronic device 1000 or components included in electronic device 1000 (i.e., PEFT model 30 to cache memory 350) can be considered as being performed by processor 1300.

[0107] Memory 1400 may be a component for storing various programs or data, and may include storage media (such as read-only memory (ROM), random access memory (RAM), hard disk, compact disc ROM (CD-ROM), digital video disc (DVD), etc.) or combinations of storage media. Memory 1400 may exist alone, or may be configured to be included in processor 1300. Memory 1400 may include volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. Memory 1400 may store programs or at least one instruction for performing operations according to one or more embodiments of the present disclosure as described below. Memory 1400 may provide stored data to processor 1300 upon request from processor 1300. Memory 1400 may include flash memory 340 and cache memory 350, such as... Figure 3 As shown.

[0108] 3. (Example 1) Generate and cache the QKV / W matrix corresponding to the PEFT model.

[0109] Figure 5 and Figure 6 This is a diagram illustrating the process by which an electronic device according to an embodiment of the present disclosure activates a PEFT model by generating and caching a QKV / W matrix based on user authentication results. (See reference...) Figure 5 The PEFT model activation module 330 may include a QKV / W generation module 331 and a QKV / W retrieval module 332. Although Figure 5 The PEFT model activation module 330 is shown to exist separately from the PEFT model 30; however, according to embodiments of this disclosure, the PEFT model activation module 330 can be implemented as being included within the PEFT model 30. First, refer to... Figure 5 Describe the process by which electronic device 1000 activates PEFT model 30 when user authentication is successful.

[0110] As described above, when PEFT model 30 needs to be executed, authentication module 320 can perform authentication and provide the authentication result to QKV / W generation module 331. When authentication is successful, electronic device 1000 can unlock QKV / W generation module 331 included in PEFT model activation module 330. The unlocked QKV / W generation module 331 can generate the QKV / W matrix required to execute (or run) PEFT model 30.

[0111] As described above, the QKV / W generation module 331 can be a generative model (e.g., a language model) trained during the fine-tuning of the base model 10 to the PEFT model 30. The QKV / W generation module 331 can generate the QKV / W matrix required to perform the PEFT model 30 by learning the QKV / W matrix generated during the fine-tuning of the base model 10 to the PEFT model 30.

[0112] According to embodiments of this disclosure, the QKV / W generation module 331 may also be included in the PEFT model 30.

[0113] According to embodiments of this disclosure, flash memory 340 can store user's personal data. QKV / W generation module 331 can obtain the user's personal data from flash memory 340, generate a QKV / W matrix using the personal data, and then store the generated QKV / W matrix in cache memory 350. For example, QKV / W generation module 331 can generate a QKV / W matrix based on prompts input by the user and the personal data stored in flash memory 340.

[0114] The QKV / W retrieval module 332 can obtain the QKV / W matrix stored in the cache memory 350 and provide the QKV / W matrix to the PEFT layer 360. According to embodiments of this disclosure, the QKV / W retrieval module 332 can obtain the QKV / W matrix from the cache memory 350 and provide the QKV / W matrix to the PEFT layer 360 when calculations using the QKV / W matrix are to be performed in the PEFT model 30.

[0115] The PEFT layer 360 can provide personalized services to users by performing calculations using the received QKV / W matrix. When the QKV / W matrix is ​​generated, the PEFT model 30 can use the generated QKV / W matrix to perform subsequent calculations.

[0116] When the processing of the user's request is completed, the QKV / W generation module 331 can delete the QKV / W matrix stored in the cache memory 350, thereby preventing the QKV / W matrix from being leaked to the outside.

[0117] Next, refer to Figure 6 Describe the process by which electronic device 1000 operates PEFT model 30 as a virtual model when user authentication fails.

[0118] When the authentication module 320 attempts to authenticate the user and the authentication fails, the electronic device 1000 does not unlock the QKV / W generation module 331 included in the PEFT model activation module 330. The QKV / W generation module 331, in a locked state, does not perform any operation; therefore, it does not generate the QKV / W matrix required for executing the PEFT model 30.

[0119] When a calculation using the QKV / W matrix needs to be performed in the PEFT model 30, the QKV / W retrieval module 332 can access the cache memory 350 to search for the QKV / W matrix. However, because the QKV / W matrix is ​​not stored in the cache memory 350, the QKV / W retrieval module 332 cannot obtain the QKV / W matrix.

[0120] When the QKV / W retrieval module 332 fails to obtain the QKV / W matrix, the QKV / W retrieval module 332 can generate a random QKV / W matrix based on a random seed and provide the generated random QKV / W matrix to the PEFT layer 360.

[0121] The PEFT layer 360 can use the random QKV / W matrix received from the QKV / W retrieval module 332 to perform subsequent calculations. In this way, the effect of fine-tuning is not reflected in the random QKV / W matrix that is not generated by the QKV / W matrix trained via the fine-tuning module 331; therefore, the PEFT model 30 operates as a virtual model. Thus, the security of the PEFT model 30 can be maintained.

[0122] 4. (Example 2) Caching the weight matrix corresponding to the PEFT model

[0123] Figure 7 and Figure 8 This is a diagram illustrating the process by which an electronic device, according to an embodiment of the present disclosure, activates a PEFT model by caching a weight matrix based on user authentication results.

[0124] exist Figure 7 and Figure 8 In the embodiments of this disclosure shown, the PEFT model activation module 330 can cache the weight matrix W used to generate the QKV matrix. Q '、W K 'and W VThe specific components included in the PEFT model activation module 330, namely the names of the QKV / W generation module 331 and the QKV / W retrieval module 332, are determined accordingly. However, the PEFT model activation module 330 can also activate the PEFT model 30 by caching other types of weight matrices.

[0125] refer to Figure 7 The PEFT model activation module 330 may include a QKV / W generation module 331 and a QKV / W retrieval module 332. First, refer to... Figure 7 Describe the process by which electronic device 1000 activates PEFT model 30 when user authentication is successful.

[0126] As described above, when the PEFT model 30 needs to be executed, the authentication module 320 can perform authentication and provide the authentication result to the QKV / W generation module 331. When authentication is successful, the electronic device 1000 can unlock the QKV / W generation module 331 included in the PEFT model activation module 330. The unlocked QKV / W generation module 331 can obtain the weight matrix stored in the flash memory 340.

[0127] According to embodiments of this disclosure, the weight matrix W is used to generate the QKV matrix required for performing the PEFT model 30. Q '、W K 'and W V The encrypted weight matrix W can be encrypted and stored in flash memory 340. The QKV / W generation module 331 can obtain the encrypted weight matrix W from flash memory 340. Q '、W K '、W V ', for the encrypted weight matrix W Q '、W K '、W V Decrypt the data and output the decrypted weight matrix W. Q '、W K '、W V 'Stored in cache memory 350. Alternatively, the weight matrix W...' Q '、W K 'and W V It can also be stored in the flash memory 340 in an unencrypted state and cached by the QKV / W generation module 331.

[0128] The QKV / W retrieval module 332 can obtain the decrypted weight matrix W stored in the cache memory 350. Q '、W K 'and W V ', and decrypt the weight matrix W Q '、W K 'and W VProvided to PEFT layer 360. According to embodiments of this disclosure, QKV / W retrieval module 332 can obtain the decrypted weight matrix W from cache memory 350. Q '、W K 'and W V ', and to perform the use of weight matrix W in PEFT model 30 Q '、W K 'and W V The weight matrix W is used in the calculation of '. Q '、W K 'and W V Provided to the PEFT layer 360.

[0129] PEFT layer 360 can use the decrypted weight matrix W Q '、W K 'and W V 'Generate a QKV matrix. Specifically, the PEFT layer 360 can do this by adjusting the weight matrix W...' Q '、W K 'and W V 'Applied to generating QKV matrices based on input embeddings generated from prompts by user input.'

[0130] According to embodiments of this disclosure, personal data can be stored in flash memory 340, and the stored personal data can be used to generate a QKV matrix. For example, electronic device 1000 can generate an input embedding based on prompts input by the user and personal data stored in flash memory 340, and PEFT layer 360 can use the weight matrix W Q '、W K 'and W V 'Apply to the input embedding generated in this way to generate the QKV matrix.'

[0131] When generating the QKV matrix, the PEFT model 30 can use the generated QKV matrix to perform subsequent calculations. According to embodiments of this disclosure, the QKV matrix generated by the PEFT layer 360 can also be stored in the cache memory 350.

[0132] When the processing of a user's request is complete, the QKV / W generation module 331 can delete the weight matrix stored in the cache memory 350, thereby preventing the weight matrix from being leaked to the outside.

[0133] Next, refer to Figure 8 Describe the process by which electronic device 1000 operates PEFT model 30 as a virtual model when user authentication fails.

[0134] When authentication module 320 attempts to authenticate the user and authentication fails, electronic device 1000 does not unlock the QKV / W generation module 331 included in PEFT model activation module 330. The QKV / W generation module 331, in a locked state, performs no operation; therefore, the weight matrix W required to execute PEFT model 30 is not executed. Q '、W K 'and W V Keep it encrypted and stored in flash memory 340.

[0135] When it is necessary to perform the use of weight matrix W in PEFT model 30 Q '、W K 'and W V During the calculation of ', the QKV / W retrieval module 332 can access the cache memory 350 to search the weight matrix W. Q '、W K 'and W V However, because of the weight matrix W Q '、W K '、W V 'The weight matrix W is not stored in the cache memory 350, therefore the QKV / W retrieval module 332 cannot obtain it.' Q '、W K '、W V '.

[0136] When the QKV / W retrieval module 332 fails to obtain the weight matrix W Q '、W K 'and W V At this time, the QKV / W retrieval module 332 can generate a random weight matrix based on a random seed and provide the generated random weight matrix to the PEFT layer 360.

[0137] The PEFT layer 360 can generate a QKV matrix by applying the random weight matrix received from the QKV / W retrieval module 332 to the input embedding, and the PEFT model 30 can use the generated QKV matrix to perform subsequent computations. In this way, when the PEFT layer 360 uses a random weight matrix instead of a weight matrix generated via fine-tuning to generate the QKV matrix, the effects of fine-tuning are not reflected in the QKV matrix; therefore, the PEFT model 30 operates as a virtual model. Thus, the security of the PEFT model 30 can be maintained.

[0138] 5. (Example 3) Caching a QKV matrix generated using personal data

[0139] Figure 9 and Figure 10This is a diagram illustrating the process by which an electronic device according to an embodiment of the present disclosure activates a PEFT model by caching a QKV / W matrix based on user authentication results.

[0140] First, refer to Figure 9 Describe the process by which electronic device 1000 activates PEFT model 30 when user authentication is successful.

[0141] When the authentication module 320 performs authentication on the user and the authentication is successful, the electronic device 1000 can unlock the QKV / W generation module 331 included in the PEFT model activation module 330.

[0142] The unlocked QKV / W generation module 331 can obtain the user's personal data stored in the flash memory 340 and use the obtained personal data to generate a QKV matrix. The specific process by which the QKV / W generation module 331 generates a QKV matrix using personal data is as follows.

[0143] Electronic device 1000 can generate input embeddings based on prompts input by the user and personal data stored in flash memory 340. First, electronic device 1000 can generate an input sequence by combining the prompts and personal data. For example, when the prompts input by the user and the personal data stored in flash memory 340 are as follows, electronic device 1000 can generate an input sequence by combining the prompts with the personal data.

[0144] Tip: "Send a greeting message to my friends among the people saved in my contacts."

[0145] Personal data: Phone numbers and group information of people stored in contacts.

[0146] Input sequence: Send greeting messages to the phone numbers of people included in the friends group in your contacts.

[0147] When generating an input sequence, the electronic device 1000 can obtain an embedding matrix by performing an embedding transformation on the input sequence, and the embedding matrix obtained in this way corresponds to the input embedding.

[0148] The QKV / W generation module 331 can generate the pre-stored weight matrix W. Q '、W K 'and W V The input embedding is applied to generate a QKV matrix, and the generated QKV matrix is ​​stored in cache memory 350. In this case, the weight matrix W Q '、W K 'and W V It can be a matrix generated by fine-tuning the base model 10 using the PEFT technique, and can be stored in flash memory 340.

[0149] The QKV / W retrieval module 332 can obtain the QKV matrix stored in the cache memory 350 and provide the QKV matrix to the PEFT layer 360. According to embodiments of this disclosure, the QKV / W retrieval module 332 can obtain the QKV matrix from the cache memory 350 and provide the QKV matrix to the PEFT layer 360 when computations using the QKV matrix are to be performed in the PEFT model 30. The PEFT model 30 can use the received QKV matrix to perform subsequent computations.

[0150] When the processing of a user's request is complete, the QKV / W generation module 331 can maintain security by deleting the QKV matrix stored in the cache memory 350.

[0151] Next, refer to Figure 10 Describe the process by which electronic device 1000 operates PEFT model 30 as a virtual model when user authentication fails.

[0152] When the authentication module 320 attempts to authenticate the user and the authentication fails, the electronic device 1000 does not unlock the QKV / W generation module 331 included in the PEFT model activation module 330. The QKV / W generation module 331, in a locked state, does not perform any operation; therefore, it does not generate the QKV / W matrix required for executing the PEFT model 30.

[0153] When a calculation using the QKV / W matrix needs to be performed in the PEFT model 30, the QKV / W retrieval module 332 can access the cache memory 350 to search for the QKV / W matrix. However, because the QKV / W matrix is ​​not stored in the cache memory 350, the QKV / W retrieval module 332 cannot obtain the QKV / W matrix.

[0154] When the QKV / W retrieval module 332 fails to obtain the QKV matrix, the QKV / W retrieval module 332 can generate a random QKV matrix based on a random seed and provide the generated random QKV matrix to the PEFT layer 360.

[0155] The PEFT model 30 can use a random QKV matrix received from the QKV / W retrieval module 332 to perform subsequent calculations. In this way, the randomly generated QKV matrix does not reflect the effects of fine-tuning or personal data, thus the PEFT model 30 operates as a virtual model. Therefore, the security of the user's personal data and the PEFT model 30 can be maintained.

[0156] 6. Refer to the flowchart to describe the process.

[0157] Figures 11 to 19This is a flowchart illustrating a method for processing user requests using a PEFT model according to embodiments of the present disclosure. In the following, reference is made to... Figures 11 to 19 This describes a method for processing user requests using a PEFT model according to embodiments of the present disclosure. Because Figures 11 to 19 The operations included in the flowchart are by Figure 3 and Figure 4 The electronic device 1000 performs this operation, so even if it is omitted below, it has already been referenced. Figures 3 to 10 The provided description can also be applied to Figures 11 to 19 .

[0158] refer to Figure 11 In operation 1101, the process can begin when the electronic device 1000 receives a prompt from the user. In operation 1102, the electronic device 1000 can determine whether it is necessary to execute (or run) the PEFT model to perform the task based on the prompt input by the user.

[0159] When it is determined that the PEFT model does not need to be executed, the electronic device 1000 can perform operation 1104 to execute the base model. In other words, the electronic device 1000 can perform the user-requested task using only the base model as a pre-trained model, without using the PEFT model to which the base model is fine-tuned.

[0160] When it is determined that a PEFT model needs to be executed (run), electronic device 1000 may perform operation 1103 to perform user authentication. Electronic device 1000 may perform authentication on the user to verify whether the user has permission to execute the PEFT model.

[0161] According to embodiments of this disclosure, authentication for verifying a user's permission to execute a PEFT model can be performed in a different manner than the authentication used for using electronic device 1000. For example, when a user begins using electronic device 1000 and unlocks it according to a first authentication method (e.g., authentication via password), electronic device 1000 can additionally authenticate the user according to a second authentication method (e.g., biometric authentication) when PEFT model execution is required. In this case, the second authentication method can have higher security than the first authentication method. However, when a user begins using electronic device 1000 and unlocks it according to the second authentication method, electronic device 1000 can determine that the user has permission to execute a PEFT model without performing additional authentication.

[0162] In operation 1105, electronic device 1000 can determine whether the authentication was successful.

[0163] Upon successful authentication, the electronic device 1000 can perform operation 1106 to obtain at least one matrix corresponding to the PEFT model. According to embodiments of this disclosure, the electronic device 1000 can use a generative model (QKV / W generation module) trained during fine-tuning of the PEFT model to generate the QKV / W matrix required for executing the PEFT model. The generative model is encrypted and then decrypted upon successful user authentication to generate the QKV / W matrix. Furthermore, according to embodiments of this disclosure, the electronic device 1000 can decrypt the weight matrix used to generate the QKV matrix. That is, in operation 1106, the electronic device 1000 can decrypt the components required for executing the PEFT model.

[0164] The detailed operations included in operation 1106 are as follows: Figure 12 and Figure 14 As shown in the image.

[0165] refer to Figure 12 In operation 1201, electronic device 1000 can unlock the QKV / W generation module. In this case, the QKV / W generation module can be a generative model trained using the QKV / W matrix generated during the fine-tuning of the base model to a PEFT model.

[0166] In operation 1202, the unlocked QKV / W generation module can generate a QKV / W matrix and store the generated QKV / W matrix in the cache memory.

[0167] In operation 1203, the QKV / W retrieval module of electronic device 1000 can provide the PEFT model with the QKV / W matrix stored in the cache memory.

[0168] refer to Figure 14 In operation 1401, electronic device 1000 can unlock the QKV / W generation module.

[0169] In operation 1402, unlocking the QKV / W generation module allows the weight matrix used to generate the QKV matrix to be stored in a cache memory. In this case, the weight matrix can be a weight matrix generated as a result of fine-tuning the pre-trained base model using the PEFT technique.

[0170] The detailed operations included in operation 1402 are as follows: Figure 15 As shown in the image. (Reference) Figure 15 The QKV / W generation module of electronic device 1000 can request a weight matrix from the flash memory of electronic device 1000 for generating the QKV matrix in operation 1501, and obtain the encrypted weight matrix from the flash memory in operation 1502.

[0171] In operation 1503, the QKV / W generation module of electronic device 1000 can decrypt the obtained weight matrix and store the decrypted weight matrix in the cache memory of electronic device 1000.

[0172] Return to reference Figure 14 In operation 1403, the QKV / W retrieval module of electronic device 1000 can provide the PEFT model with a weight matrix stored in the cache memory.

[0173] Return to Figure 11 In operation 1108, the electronic device 1000 can perform a user-requested task by executing a PEFT model using at least one obtained matrix. According to embodiments of this disclosure, the electronic device 1000 can generate a QKV / W matrix using personal data stored in flash memory, and then execute a PEFT model using the generated QKV / W matrix. Alternatively, according to embodiments of this disclosure, the electronic device 1000 can obtain a QKV matrix using personal data stored in flash memory and an obtained weight matrix, and then execute a PEFT model using the obtained QKV matrix.

[0174] The detailed operations included in operation 1108 are as follows: Figure 13 and Figure 16 As shown in the image.

[0175] refer to Figure 13 In operation 1301, the electronic device 1000 can control the PEFT model to perform an attention mechanism using the obtained QKV / W matrix.

[0176] In operation 1302, electronic device 1000 can control the PEFT model to perform inference based on the results of the execution attention mechanism.

[0177] When the user-requested task is completed, in operation 1303, the QKV / W generation module of the electronic device 1000 can delete the QKV / W matrix stored in the cache memory.

[0178] refer to Figure 16 In operation 1601, electronic device 1000 can obtain a QKV matrix by applying a weight matrix to an input embedding generated based on prompts.

[0179] In operation 1602, electronic device 1000 can control the PEFT model to perform an attention mechanism using the obtained QKV matrix.

[0180] In operation 1603, electronic device 1000 can control the PEFT model to perform inference based on the results of the execution attention mechanism.

[0181] When the user-requested task is completed, in operation 1604, the QKV / W generation module of the electronic device 1000 can delete the weight matrix stored in the cache memory. The QKV / W generation module can also delete the QKV matrix from the cache memory if it is also stored there.

[0182] On the other hand, when Figure 11 If authentication fails in operation 1105, which determines whether authentication was successful, electronic device 1000 can execute operation 1107 to perform a PEFT model using a randomly generated QKV / W matrix. In this case, the PEFT model operates as a virtual model.

[0183] refer to Figure 17 In operation 1701, the process may begin when electronic device 1000 receives a prompt from the user. In operation 1702, electronic device 1000 may determine whether to execute a PEFT model to perform the task based on the prompt input by the user.

[0184] When it is determined that the PEFT model does not need to be executed, the electronic device 1000 can perform operation 1704 to execute the base model. In other words, the electronic device 1000 can perform the user-requested task using only the base model as a pre-trained model, without using the PEFT model to which the base model is fine-tuned.

[0185] When it is determined that the PEFT model needs to be executed, the electronic device 1000 can perform operation 1703 to perform user authentication. The electronic device 1000 can authenticate the user to verify whether the user has permission to execute the PEFT model.

[0186] According to embodiments of this disclosure, authentication for verifying a user's permission to execute a PEFT model can be performed in a different manner than the authentication used for using electronic device 1000. For example, when a user begins using electronic device 1000 and unlocks it according to a first authentication method (e.g., authentication via password), electronic device 1000 can additionally authenticate the user according to a second authentication method (e.g., biometric authentication) when PEFT model execution is required. In this case, the second authentication method can have higher security than the first authentication method. However, when a user begins using electronic device 1000 and unlocks it according to the second authentication method, electronic device 1000 can determine that the user has permission to execute a PEFT model without performing additional authentication.

[0187] In operation 1705, electronic device 1000 can determine whether the authentication was successful.

[0188] Upon successful authentication, electronic device 1000 can perform operation 1706 to obtain the QKV matrix corresponding to the PEFT model based on the user's personal data. In other words, upon successful user authentication, electronic device 1000 can generate the QKV matrix for executing the PEFT model. In this case, electronic device 1000 can also obtain the weight matrix corresponding to the PEFT model to generate the QKV matrix. The specific method by which electronic device 1000 obtains the weight matrix corresponding to the PEFT model is described in the reference above. Figure 14 and Figure 15 As stated above.

[0189] The detailed operations included in Operation 1706 are as follows: Figure 18 As shown in the image. (Reference) Figure 18 In operation 1801, electronic device 1000 can unlock the QKV / W generation module.

[0190] In operation 1802, the unlocked QKV / W generation module can obtain personal data from the flash memory of electronic device 1000.

[0191] In operation 1803, the QKV / W generation module of electronic device 1000 can generate a QKV matrix based on personal data and store the generated QKV matrix in a cache memory.

[0192] In operation 1804, the QKV / W retrieval module of electronic device 1000 can provide the PEFT model with the QKV matrix stored in the cache memory.

[0193] Return to reference Figure 17 In operation 1708, electronic device 1000 can use the obtained QKV matrix to perform a PEFT model.

[0194] The detailed operations included in Operation 1708 are as follows: Figure 19 As shown in the image. (Reference) Figure 19 In operation 1901, electronic device 1000 can control the PEFT model to perform an attention mechanism using the obtained QKV matrix.

[0195] In operation 1902, electronic device 1000 can control the PEFT model to perform inference based on the results of the execution attention mechanism.

[0196] In operation 1903, the QKV / W generation module of electronic device 1000 can delete the QKV matrix stored in the cache memory.

[0197] On the other hand, when authentication fails in operation 1705, which determines whether authentication was successful, electronic device 1000 can execute operation 1707 to perform a PEFT model using a randomly generated QKV matrix. In this case, the PEFT model operates as a virtual model.

[0198] According to the embodiments of this disclosure described above, the PEFT model reflecting a user's personal data can provide an enhanced security effect by being executed only when the user's authentication is successful.

[0199] According to embodiments of this disclosure, a method for processing a user request using a PEFT model may include: an electronic device receiving a prompt from a user; the electronic device determining whether a PEFT model needs to be executed to process the prompt; when a PEFT model does not need to be executed, the electronic device executing a base model; when a PEFT model needs to be executed, the electronic device performing user authentication; when user authentication is successful, the electronic device obtaining at least one matrix corresponding to the PEFT model; and the electronic device using the at least one matrix to execute the PEFT model.

[0200] According to embodiments of this disclosure, obtaining at least one matrix corresponding to a PEFT model may include: unlocking the QKV / W generation module; generating a QKV / W matrix; and storing the generated QKV / W matrix in a cache memory, wherein the generation and storage are performed by the QKV / W generation module; and providing the QKV / W matrix stored in the cache memory to the PEFT model by the QKV / W retrieval module.

[0201] According to embodiments of this disclosure, using at least one matrix to perform a PEFT model may include: the PEFT model performing an attention mechanism using the generated QKV / W matrix; the PEFT model performing inference based on the result of performing the attention mechanism; and the QKV / W matrix being deleted from a cache memory by the QKV / W generation module.

[0202] According to embodiments of this disclosure, the QKV / W generation module can be a generative model trained using the QKV / W matrix generated during the process of fine-tuning the base model into a PEFT model.

[0203] According to embodiments of this disclosure, the weight matrix may be a matrix used to generate the QKV matrix required to perform the PEFT model.

[0204] According to embodiments of this disclosure, obtaining at least one matrix may include: unlocking the QKV / W generation module; storing a weight matrix used to generate the QKV matrix in a cache memory by the QKV / W generation module; and providing the weight matrix stored in the cache memory to the PEFT model by the QKV / W retrieval module.

[0205] According to embodiments of this disclosure, storing the weight matrix in a cache memory may include: the QKV / W generation module requesting a weight matrix from the flash memory for generating a QKV matrix; the QKV / W generation module obtaining an encrypted weight matrix from the flash memory; and the QKV / W generation module decrypting the obtained weight matrix and storing the decrypted weight matrix in the cache memory.

[0206] According to embodiments of this disclosure, executing a PEFT model may include: obtaining a QKV matrix by applying a weight matrix to a prompt-generated input embedding; controlling the PEFT model to perform an attention mechanism using the QKV matrix; controlling the PEFT model to perform inference based on the result of performing the attention mechanism; and deleting the weight matrix stored in a cache memory by the QKV / W generation module.

[0207] According to embodiments of this disclosure, obtaining a QKV matrix may include: obtaining personal data of a user stored in flash memory; generating an input embedding based on prompts and personal data; and obtaining the QKV matrix by applying a weight matrix to the input embedding.

[0208] An electronic device according to an embodiment of the present disclosure may include a memory storing a program or at least one instruction and at least one processor, wherein the at least one processor is configured to execute a program or at least one instruction stored in the memory to cause the electronic device to receive a prompt from a user, determine whether it is necessary to execute a PEFT model to process the prompt, execute a base model when it is not necessary to execute a PEFT model, perform user authentication when it is necessary to execute a PEFT model, and obtain at least one matrix corresponding to the PEFT model when the user authentication is successful, and use the at least one matrix to execute the PEFT model.

[0209] According to embodiments of this disclosure, when at least one matrix corresponding to the PEFT model is obtained, the electronic device can be configured to unlock the QKV / W generation module, generate a QKV / W matrix by the QKV / W generation module and store the generated QKV / W matrix in a cache memory, and provide the QKV / W matrix stored in the cache memory to the PEFT model by the QKV / W retrieval module.

[0210] According to embodiments of this disclosure, when using at least one matrix to execute a PEFT model, the electronic device can be configured to have the PEFT model perform an attention mechanism using the generated QKV / W matrix, have the PEFT model perform inference based on the result of performing the attention mechanism, and have the QKV / W matrix stored in a cache memory deleted by the QKV / W generation module.

[0211] According to embodiments of this disclosure, the QKV / W generation module can be a generative model trained using the QKV / W matrix generated during the process of fine-tuning the base model into a PEFT model.

[0212] According to embodiments of this disclosure, the weight matrix may be a matrix used to generate the QKV matrix required to perform the PEFT model.

[0213] According to embodiments of this disclosure, when at least one matrix is ​​obtained, the electronic device can be configured to unlock the QKV / W generation module, which stores the weight matrix used to generate the QKV matrix in a cache memory, and the QKV / W retrieval module provides the weight matrix stored in the cache memory to the PEFT model.

[0214] According to embodiments of this disclosure, when storing the weight matrix in a cache memory, the electronic device can be configured to control the QKV / W generation module to request the weight matrix for generating the QKV matrix from the flash memory, control the QKV / W generation module to obtain the encrypted weight matrix from the flash memory, and control the QKV / W generation module to decrypt the obtained weight matrix and store the decrypted weight matrix in the cache memory.

[0215] According to embodiments of this disclosure, when executing a PEFT model, an electronic device can be configured to obtain a QKV matrix by applying a weight matrix to a prompt-generated input embedding, control the PEFT model to use the QKV matrix to perform an attention mechanism, control the PEFT model to perform inference based on the result of performing the attention mechanism, and control the QKV / W generation module to delete the weight matrix stored in a cache memory.

[0216] According to embodiments of this disclosure, when obtaining a QKV matrix, an electronic device can be configured to obtain personal data of a user stored in flash memory, generate an input embedding based on prompts and personal data, and obtain a QKV matrix by applying a weight matrix to the input embedding.

[0217] One or more embodiments of this disclosure may be implemented or supported by one or more computer programs, which may be created from computer-readable program code and included on a computer-readable medium. As used herein, the terms "application" and "program" may refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in computer-readable program code. "Computer-readable program code" may include various types of computer code, including source code, object code, and executable code. "Computer-readable medium" may include various types of media accessible by a computer, such as ROM, RAM, hard disk drive (HDD), CD, DVD, or various other types of storage.

[0218] Furthermore, machine-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory storage media" refers to tangible devices and may exclude wired, wireless, optical, or other communication links that transmit transient electrical signals or other signals. Moreover, the term "non-transitory storage media" does not distinguish between cases where data is stored semi-permanently in a storage medium and cases where data is temporarily stored in a storage medium. For example, "non-transitory storage media" may include buffers for temporarily storing data. Computer-readable media can be any available medium accessible by a computer and includes volatile and non-volatile media, as well as removable and non-removable media. Computer-readable media include media on which data can be permanently stored and media on which data can be stored and later overwritten, such as rewritable optical discs or erasable memory devices.

[0219] According to embodiments of this disclosure, when provided, methods according to one or more embodiments of this disclosure set forth herein may be included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., CD-ROM), or distributed online via an app store (e.g., downloaded or uploaded), or directly between two user devices (e.g., smartphones). For online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be stored at least temporarily or temporarily generated in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a relay server).

[0220] The foregoing description of this disclosure is for illustrative purposes only, and those skilled in the art will understand that changes in form and detail can be readily made without departing from the technical spirit or essential characteristics of this disclosure. For example, appropriate effects may be achieved even when the above-described techniques are performed in a different order than described above, and / or when the above-described components of a system, structure, device, circuit, etc., are coupled or combined in a different form and mode than described above, or when they are replaced or supplemented by other components or their equivalents. Therefore, the above-described embodiments and all aspects thereof are merely illustrative and not restrictive. For example, each component defined as an integrated component may be implemented in a distributed manner, and similarly, components defined as separate components may be implemented in an integrated form.

[0221] The scope of this disclosure is not defined by its detailed description but by the appended claims, and all changes or modifications to the meaning and scope of the appended claims and their equivalents shall be construed as being included within the scope of this disclosure.

Claims

1. A method for an electronic device to efficiently fine-tune a PEFT model using parameters to process user requests, the method comprising: The electronic device receives prompts from the user; The electronic device determines whether a PEFT model needs to be executed to process the prompt. Based on the determination that the PEFT model does not need to be executed, the basic model is executed by the electronic device. Based on the determination that the PEFT model needs to be executed, user authentication is performed by the electronic device. Based on successful user authentication, the electronic device obtains at least one matrix corresponding to the PEFT model; and The electronic device uses the at least one matrix to perform the PEFT model.

2. The method according to claim 1, wherein, Obtaining the at least one matrix includes: Unlock the query key-value / weight QKV / W generation module; The QKV / W generation module generates a QKV / W matrix and stores it in the electronic device's cache memory; and The QKV / W retrieval module provides the QKV / W matrix to the PEFT model.

3. The method according to claim 1 or 2, wherein, Using the at least one of the matrices to perform a PEFT model includes: The attention mechanism is performed by the PEFT model using the QKV / W matrix; Inference is performed by the PEFT model based on the results of the execution attention mechanism; and The QKV / W matrix is ​​removed from the cache memory by the QKV / W generation module.

4. The method according to any one of claims 1 to 3, wherein, The QKV / W generation module is a generative model trained using the QKV / W matrix generated during the fine-tuning of the base model to the PEFT model.

5. The method according to any one of claims 1 to 4, wherein, Obtaining the at least one matrix includes: Unlock the QKV / W generation module; The QKV / W generation module stores the weight matrix used to generate the QKV matrix in a cache memory; and The weight matrix is ​​provided to the PEFT model by the QKV / W retrieval module.

6. The method according to any one of claims 1 to 5, wherein, Storing the weight matrix in a cache memory includes: The QKV / W generation module requests a weight matrix from the flash memory of the electronic device; The encrypted weight matrix is ​​obtained from flash memory by the QKV / W generation module; and The encrypted weight matrix is ​​decrypted and stored in a cache memory, wherein the decryption and storage are performed by the QKV / W generation module.

7. The method according to any one of claims 1 to 6, wherein, Executing a PEFT model includes: The QKV matrix is ​​obtained by applying the weight matrix to the input embedding generated based on the prompts; Control the PEFT model to use the QKV matrix to perform the attention mechanism; Controlling the PEFT model to perform inference based on the results of the execution attention mechanism; and The weight matrix is ​​removed from the cache memory by the QKV / W generation module.

8. The method according to any one of claims 1 to 7, wherein, Obtaining the QKV matrix includes: To obtain a user's personal data stored in flash memory; Generate an input embedding based on the prompts and the personal data; and The QKV matrix is ​​obtained by applying the weight matrix to the input embedding.

9. An electronic device (1000), comprising: Memory (1400), storing a program or at least one instruction; and At least one processor (1300) is configured to execute the program or the at least one instruction individually or jointly, wherein the program or the at least one instruction, when executed individually or jointly by the at least one processor (1300), causes the electronic device (1000): Get prompts from users, Determine whether it is necessary to perform parameter-efficient fine-tuning of the PEFT model (30) to handle the aforementioned prompt. Based on the determination that the PEFT model (30) does not need to be executed, the basic model (10) is executed. Based on the determination that the PEFT model (30) needs to be executed, user authentication is performed. Based on successful user authentication, at least one matrix corresponding to the PEFT model (30) is obtained, and The PEFT model (30) is performed using at least one of the matrices.

10. The electronic device according to claim 9, further comprising a cache memory (350). in, When the program or the at least one instruction is executed individually or jointly by at least one processor (1300), the electronic device (1000) obtains the at least one matrix in the following manner: Unlock the query key-value / weight QKV / W generation module (331); The QKV / W generation module (331) generates a QKV / W matrix and stores the generated QKV / W matrix in a cache memory (350). The QKV / W matrix is ​​provided to the PEFT model (30) by the QKV / W retrieval module (332).

11. The electronic device according to claim 9 or 10, wherein, When the program or the at least one instruction is executed individually or jointly by at least one processor (1300), the electronic device (1000) executes the PEFT model (30) in such a way as follows: The attention mechanism is performed by the PEFT model (30) using the QKV / W matrix. Inference is performed by the PEFT model (30) based on the results of the execution attention mechanism, and The QKV / W matrix is ​​removed from the cache memory (350) by the QKV / W generation module (331).

12. The electronic device according to any one of claims 9 to 11, wherein, The QKV / W generation module (331) is a generative model trained using the QKV / W matrix generated during the fine-tuning of the base model (10) to the PEFT model (30).

13. The electronic device according to any one of claims 9 to 12, wherein, When the program or the at least one instruction is executed individually or jointly by at least one processor (1300), the electronic device (1000) obtains the at least one matrix in the following manner: Unlock the QKV / W generation module (331). The QKV / W generation module (331) stores the weight matrix used to generate the QKV matrix in a cache memory, and The weight matrix is ​​provided to the PEFT model (30) by the QKV / W retrieval module (332).

14. The electronic device according to any one of claims 9 to 13, wherein, When the program or the at least one instruction is executed individually or jointly by at least one processor (1300), the electronic device (1000) stores the weight matrix in the cache memory (350) in such a way that: The control module (331) requests the weight matrix from the flash memory (340). The control module (331) obtains the encrypted weight matrix from flash memory (340), and The QKV / W generation module (331) is controlled to decrypt the encrypted weight matrix and store the decrypted weight matrix in the cache memory (350).

15. The electronic device according to any one of claims 9 to 14, wherein, When the program or the at least one instruction is executed individually or jointly by at least one processor (1300), the electronic device (1000) executes the PEFT model (30) in the following manner: The QKV matrix is ​​obtained by applying the weight matrix to the input embedding generated based on the prompts. The PEFT model (30) is controlled to use a QKV matrix to perform the attention mechanism. The PEFT model (30) is controlled to perform inference based on the results of the execution attention mechanism, and The control QKV / W generation module (331) removes the weight matrix from the cache memory (350).

Images