Method for processing user request using on-device ai model, and electronic device for performing same

Abstract

A method of processing a user request by using a PEFT model, performed by an electronic device, may comprise the steps in which: the electronic device obtains a prompt from a user; the electronic device determines whether execution of the PEFT model is required to process the prompt; when it is determined that the execution of the PEFT model is not required, the electronic device executes a foundation model; when it is determined that the execution of the PEFT model is required, the electronic device performs user authentication; when the user authentication is successful, the electronic device obtains at least one matrix corresponding to the PEFT model; and the electronic device executes the PEFT model by using the at least one matrix.

Description

A method for processing a user request using an on-device AI model and an electronic device for performing the same.

[0001] The present disclosure relates to a method for processing a user request using an on-device AI model and an electronic device for performing the same, and more particularly, to a method for enhancing security by encrypting a configuration for generating a QKV / W matrix required for executing an on-device AI model.

[0002] Recently, pretraining large models on common datasets and then fine-tuning them according to individual needs has become a popular approach. However, training all parameters of a large model during fine-tuning consumes excessive time and computational resources. To improve efficiency, the Parameter Efficient Fine Tuning (PEFT) technique was developed, which trains only a subset of parameters.

[0003] Fine-tuning large models using PEFT techniques can compromise users' efforts, expertise, and personal information. Therefore, users may not want their fine-tuned models to be used without their permission. Furthermore, maintaining the security of fine-tuned models is crucial for protecting personal information.

[0004] According to one aspect of the present disclosure, a method for an electronic device to process a user request using a PEFT model may include: obtaining a prompt from a user by the electronic device; determining whether execution of a PEFT model is required for the electronic device to process the prompt; executing a foundation model by the electronic device when it is determined that execution of the PEFT model is not required; performing user authentication by the electronic device when it is determined that execution of the PEFT model is required; obtaining at least one matrix corresponding to the PEFT model by the electronic device when the user authentication is successful; and executing the PEFT model using the at least one matrix by the electronic device.

[0005] According to one aspect of the present disclosure, an electronic device includes a memory in which a program or at least one instruction is stored, and at least one processor configured to execute the program or the at least one instruction alone or in cooperation, and the at least one processor alone or in cooperation executes the program or the at least one instruction, whereby the electronic device can obtain a prompt from a user, determine whether execution of a PEFT model is required to process the prompt, and if it is determined that execution of the PEFT model is not required, execute a foundation model, perform user authentication if it is determined that execution of the PEFT model is required, and if the user authentication is successful, obtain at least one matrix corresponding to the PEFT model, and then execute the PEFT model using the at least one matrix.

[0006] According to one aspect of the present disclosure, a method for an electronic device to process a user request using a PEFT model may include the steps of: obtaining a prompt from a user by the electronic device; determining whether execution of a PEFT model is required for the electronic device to process the prompt; executing a foundation model by the electronic device when it is determined that execution of the PEFT model is not required; performing user authentication by the electronic device when it is determined that execution of the PEFT model is required; obtaining at least one matrix corresponding to the PEFT model by the electronic device when the user authentication is successful, the step of generating a QKV / W (Query Key Value / Weight) matrix, storing the generated QKV / W matrix in a cache memory of the electronic device, and providing the QKV / W matrix to the PEFT model; and executing the PEFT model by the electronic device using the at least one matrix.

[0007] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0008] Figure 1 is a diagram for explaining the process of fine-tuning a foundation model according to the PEFT technique.

[0009] Figure 2 is a diagram illustrating a process in which weight matrices for generating a QKV / W matrix are decomposed into low-dimensional matrices and then fine-tuned according to the LoRA method.

[0010] FIG. 3 is a drawing for explaining modules included in an electronic device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a drawing for explaining a hardware configuration included in an electronic device according to one embodiment of the present disclosure.

[0012] FIGS. 5 and 6 are diagrams for explaining a process in which an electronic device according to one embodiment of the present disclosure activates a PEFT model by generating a QKV / W matrix based on a user authentication result.

[0013] FIGS. 7 and 8 are diagrams illustrating a process in which an electronic device according to one embodiment of the present disclosure activates a PEFT model by caching a weight matrix based on a user authentication result.

[0014] FIGS. 9 and 10 are diagrams illustrating a process in which an electronic device according to one embodiment of the present disclosure activates a PEFT model by caching a QKV matrix based on a user authentication result.

[0015] FIGS. 11 to 19 are flowcharts illustrating a method of processing a user request using a PEFT model according to embodiments of the present disclosure.

[0016] In this disclosure, the expressions “at least one of a, b, or c” and “at least one of a, b, and c” may refer to “a,” “b,” “c,” “a and b,” “a and c,” “b and c,” “all of a, b, and c,” or variations thereof.

[0017] In describing this disclosure, descriptions of technical details that are well-known in the technical field to which this disclosure pertains and are not directly related to this disclosure will be omitted. This is to avoid obscuring the gist of this disclosure by omitting unnecessary explanations and to convey it more clearly. Furthermore, the terms described below are defined based on their functions in this disclosure and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the contents of this specification as a whole.

[0018] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0019] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure of the present disclosure is complete and to fully inform those skilled in the art of the present disclosure of the scope of the disclosure. An embodiment of the present disclosure may be defined according to the claims. Like reference numerals denote like elements throughout the specification. In addition, when describing an embodiment of the present disclosure, if a detailed description of a related function or configuration is determined to unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted. In addition, the terms described below are terms defined in consideration of the functions of the present disclosure and may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification.

[0020] Terms such as "unit," "module," "member," and "block" may be implemented in hardware or software. In this specification, multiple "units," "modules," "members," and "blocks" may be implemented as a single component, and a single "unit," "module," "member," or "block" may include multiple components.

[0021] When an element is referred to as being “connected” to another element, this means that the element may be directly connected to the other element, or may be indirectly connected, where an indirect connection may include “connecting via a wireless communications network.”

[0022] Additionally, when a part is said to “include” or “comprise” an element, unless specifically stated otherwise, that part may additionally include other elements and is not exclusive of such other elements.

[0023] Although terms such as “first,” “second,” and “third” may be used to describe various elements in this disclosure, the disclosure should not be limited to these terms. These terms are used only to distinguish one element from another.

[0024] In this disclosure, singular expressions such as “a,” “an,” and “the” are to be construed to include plural forms as well, unless the context clearly dictates otherwise.

[0025] In the methods or processes described herein, identification codes are provided for convenience of explanation and do not indicate the order in which each step or operation is performed. Unless the context clearly dictates otherwise, each step or operation may be performed in a different order than the illustrated order, and one or more steps or operations may be omitted.

[0026] The various operations, acts, blocks, steps, etc. depicted in the flowchart may be performed in the order depicted, in a different order, or simultaneously. Furthermore, some operations, acts, blocks, steps, etc. may be omitted, added, modified, or skipped according to one or more embodiments, without departing from the scope of the present disclosure.

[0027] In one embodiment, each block of the flowchart diagrams and combinations of the flowchart diagrams can be performed by computer program instructions. The computer program instructions can be installed on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, and the instructions, when executed by the processor of the computer or other programmable data processing apparatus, can create means for performing the functions described in the flowchart block(s). The computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing apparatus to implement the functions in a particular manner, and the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions can also be installed on a computer or other programmable data processing apparatus.

[0028] Additionally, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one embodiment, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0029] The term '~ unit' used in one embodiment of the present disclosure may represent software or a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the '~ unit' may perform a specific role. Meanwhile, the '~ unit' is not limited to software or hardware. The '~ unit' may be configured to be on an addressable storage medium and may be configured to play one or more processors. In one embodiment, the '~ unit' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided through a specific component or a specific '~ unit' may be combined to reduce the number of components or separated into additional components. In addition, in one embodiment, the '~ unit' may include one or more processors.

[0030] Below, the meanings of terms used in this disclosure are explained.

[0031] A "foundation model" may refer to a model trained using a large-scale dataset, making it universally applicable across a variety of tasks. Furthermore, a "pre-trained model" may refer to a model that has previously learned general knowledge or patterns using a large-scale dataset. In this disclosure, the terms "foundation model" and "pre-trained model" may be used interchangeably, and they can be trained to become models suitable for specific tasks through fine-tuning.

[0032] The term 'PEFT model' may refer to a model fine-tuned according to the PEFT (Parameter Efficient Fine Tuning) technique. Similarly, the term 'LoRA model' may refer to a model fine-tuned according to the LoRA (Low Rank Adaptation) technique. The PEFT technique is one of the techniques for performing fine tuning, and is an efficient technique that can save computational resources and processing time by updating only some parameters instead of all parameters of a pre-trained model. Terms such as 'personalized model' or 'user-specific AI model' may be used instead of 'PEFT model'.

[0033] A "PEFT layer" can refer to a layer added to a foundation model as a result of fine-tuning using the PEFT technique. In other words, a PEFT layer can refer to a layer that a PEFT model additionally includes in relation to the foundation model. A PEFT layer can include additional parameters learned during fine-tuning.

[0034] If we organize the relationship between the PEFT model and the PEFT layer, we can say that the PEFT model is the result of adding the PEFT layer to the foundation model.

[0035] The term 'QKV matrix (Query Key Value Matrix)' can encompass the Q matrix, K matrix, and V matrix used in the attention mechanism. The QKV matrix can be obtained by applying a weight matrix to the input embedding.

[0036] The term 'weight matrix' can refer to a matrix for generating a QKV matrix. The weight matrix can be generated (updated) during the fine-tuning process of the foundation model. A QKV matrix can be generated by multiplying the input embedding by the weight matrix. For example, the weight matrix W for the query is added to the input embedding X. Q Multiplying by creates a Q matrix, and the remaining K and V matrices are also weighted by the input embedding X, respectively, with the weight matrix W for the keys. K and weight matrix W for values V can be obtained by multiplying .

[0037] The 'QKV matrix' and the 'weight matrix' can be combined to form a 'QKV / W matrix (Query Key Value / Weight Matrix)'.

[0038] A 'prompt' is a sentence or keyword for interaction between a user and a model, and may be text for the user to send a question or command to the model. In other words, a prompt may mean text or other forms of input that instruct the model what type of output it should generate. In the present disclosure, 'execute a prompt' or 'execute a generative model according to a prompt' may mean an action in which the generative model performs a task according to the request of the prompt, that is, an action in which the generative model performs a calculation and generates a result corresponding to the prompt when a prompt is input to the generative model. A prompt may include 'intent' and 'details', which will be described in detail below. Terms such as 'instruction' may also be used instead of 'prompt'.

[0039] "Input data" can refer to the actual data that a model must process or analyze. Input data can take various forms, such as text, images, or audio. For example, if a user requests translation by prompting the model with "Translate the following sentence into Korean," the text to be translated can be the input data. Or, if a user requests editing of an image by prompting the model with "Erase the clouds in the sky," the image to be edited can be the input data. Terms such as "source data" or "input values" can also be used instead of "input data."

[0040] An "input sequence" refers to the actual input supplied to a model, and can refer to the entire input that the model must process. In other words, an input sequence can refer to the entire data passed to the model's input layer, and can include not only text prompts but also other forms of input data, such as images and audio. In other words, an input sequence can be a combination of prompts and input data. For example, an input sequence for a text-to-image model can include image data to be edited and a prompt (text) instructing the editing.

[0041] For example, if a user inputs the sentence "He always inspires me" as input data along with the prompt "Translate the following sentence into Korean," the input sequence could be "Translate the following sentence into Korean. He always inspires me." Alternatively, if a user inputs the image to be edited along with the prompt "Erase the clouds in the sky," the input sequence could be a combination of "Erase the clouds in the sky in the photo" and the image. Alternatively, the input image could be converted to text, and the input sequence could be the combination of the text and the prompt. Instead of "input sequence," terms such as "complete input," "input stream," or "input series" could also be used.

[0042] The term "input embedding" may refer to an embedding matrix corresponding to an input sequence. In other words, the input embedding may be an embedding matrix obtained by performing an embedding transformation on the tokens included in the input sequence. The input sequence refers to the entire input input to the model, and may include only text prompts entered by the user, or may include data such as images or audio entered along with the prompts.

[0043] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0044] The present disclosure relates to a method for processing a user request using a PEFT model and an electronic device for performing the same, and embodiments of the present disclosure have the feature of enhancing security by encrypting a configuration required for executing the PEFT model.

[0045] The electronic device according to embodiments of the present disclosure executes the PEFT model only when user authentication is successful, in order to enhance the security of the user's personal data (e.g., privacy-related information) and the structure of the model, as described in detail below.

[0046] The PEFT model may reflect a user's personal data. A PEFT model may include PEFT layers alongside the layers of the foundation model (pre-trained model). The parameters included in the PEFT layers may be generated (updated) during the fine-tuning process of the foundation model, reflecting the user's personal data.

[0047] The PEFT model may use the user's personal data stored on the electronic device during execution. For example, the PEFT layer included in the PEFT model may use the user's personal data to generate a QKV / W matrix.

[0048] Since the user's personal data may already be reflected in the PEFT model or the user's personal data may be used in the process of executing the PEFT model, the electronic device according to the embodiments of the present disclosure can enhance data security by executing the PEFT model only when user authentication is successful.

[0049] The features of the embodiments of the present disclosure are briefly summarized as follows.

[0050] (A) Determine whether execution of the PEFT model is required to process the user's request.

[0051] - If the execution of the PEFT model is required, the electronic device performs user authentication.

[0052] - If the execution of the PEFT model is not required, the electronic device uses the Foundation model to process the user's request.

[0053] (B) If user authentication is successful, the PEFT model is executed to process the user's request.

[0054] - The electronic device encrypts the configuration required to execute the PEFT model, and upon successful user authentication, decrypts the configuration and executes the PEFT model.

[0055] - For example, the configuration for generating the QKV / W matrix required for executing the PEFT model is locked and waits, and when the user authentication is successful, the configuration is unlocked and the QKV / W matrix is ​​generated.

[0056] - Or, for example, the weight matrix corresponding to the PEFT model, that is, the weight matrix for generating the QKV matrix required for executing the PEFT model, is encrypted and stored in flash memory, and when user authentication is successful, it is decrypted and stored in cache memory and used to generate the QKV matrix.

[0057] (C) If user authentication fails, execute a dummy model to process the user's request.

[0058] - If user authentication fails, the electronic device generates a random weight matrix based on a random seed and uses the generated weight matrix to generate a QKV matrix (in this case, the PEFT model acts as a dummy model).

[0059] Before describing specific embodiments of the present disclosure, a method for fine-tuning a foundation model using the PEFT technique and the additional layers generated as a result will first be described with reference to FIGS. 1 and 2 . Next, the configuration and overall operation of an electronic device according to an embodiment of the present disclosure will be described with reference to FIGS. 3 and 4 , and specific embodiments will be described with reference to FIGS. 5 to 8 .

[0060] 1. Method of performing fine tuning according to PEFT technique (Fig. 1 and Fig. 2)

[0061] Figure 1 is a diagram for explaining the process of fine-tuning a foundation model according to the PEFT technique.

[0062] Referring to FIG. 1, the foundation model (10) may include N layers (11, 12, 13), and each of the layers (11, 12, 13) may include a weight matrix. For example, at least one of the layers (11, 12, 13) may include weight matrices (W) for generating a QKV matrix. Q , W K , W V ) may be included.

[0063] As previously described, the foundation model (10) may be a pre-trained neural network model using a general dataset. Users can fine-tune the foundation model (10) to suit a specific task, and utilize the Parameter Efficient Fine Tuning (PEFT) technique to efficiently perform fine-tuning.

[0064] PEFT is a method for fine-tuning a pre-trained, large-scale model to suit a specific task. Rather than adjusting (updating) all of the model's parameters, it adjusts only some. In other words, PEFT is a fine-tuning method that fixes the parameters of an existing, pre-trained, large-scale model, adds a new, trainable neural network structure, and then updates only the newly added portion.

[0065] If the foundation model (10) is a large-scale model, a lot of computational resources and memory are required to fine-tune all the parameters included in the foundation model (10). The PEFT technique is a proposed method to overcome this limitation.

[0066] When the foundation model (10) is fine-tuned according to the PEFT technique, a first PEFT model (20) or a second PEFT model (30) can be generated. There are various types of PEFT techniques, such as the adapter tuning method, the LoRA (Low Rank Adaptation) method, the prefix tuning method, and the prompt tuning method. In the embodiment illustrated in Fig. 1, it is assumed that the first PEFT model (20) is generated by the adapter tuning method, and the second PEFT model (30) is generated by the LoRA method.

[0067] The adapter tuning method adds a small network (adapter) to the existing model (foundation model), and trains only the parameters of the added adapter while maintaining the parameters of the existing model.

[0068] Referring to Fig. 1, an adapter layer (21) is added between the first layer (11) and the second layer (12) of the first PEFT model (20) generated by the adapter tuning method. According to the adapter tuning method, the parameters of the layers (11, 12, 13) of the existing foundation model (10) can be frozen, and only the parameters of the adapter layer (21) can be updated.

[0069] According to one embodiment of the present disclosure, the adapter layer (21) may include weight matrices for generating a QKV matrix required for executing the first PEFT model (20).

[0070] The LoRA method decomposes the high-dimensional weight matrices contained in the layers of the existing model (foundation model) into low-dimensional ones and trains the decomposed low-dimensional weight matrices. In other words, according to the LoRA method, the parameters of the existing model (foundation model) are frozen, and newly defined low-rank matrices are updated only for the parameters of those matrices.

[0071] When fine tuning is performed according to the LoRA method, layers that decompose at least one layer among the layers (11, 12, 13) included in the foundation model (10) into a low-dimensional space may be added. Referring to FIG. 1, the second PEFT model (30) may include the second-first layer (31) and the second-second layer (32) generated by decomposing the second layer (12) of the foundation model (10) into a low-dimensional space. Of course, not only the second layer (12), but also other layers (11, 13) included in the foundation model (10) may be decomposed into a low-dimensional space and included in the second PEFT model (30). According to the LoRA method, the parameters of the layers (11, 12, 13) of the existing foundation model (10) may be frozen, and only the parameters of the second-first layer (31) and the second-second layer (32) may be updated.

[0072] According to one embodiment of the present disclosure, the second-1 layer (31) and the second-2 layer (32) may include weight matrices for generating a QKV matrix required for executing the second PEFT model (30). For example, the second layer (12) may include weight matrices (W) for generating a QKV matrix. Q , W K , W V ) are included, and the 2-1 layer (31) and the 2-2 layer (32) include weight matrices (W Q , W K , W V ) may include matrices that are decomposed into low dimensions. This is described in detail with reference to Fig. 2.

[0073] Figure 2 is a diagram illustrating a process in which weight matrices for generating a QKV matrix are decomposed into low-dimensional matrices and then fine-tuned according to the LoRA method.

[0074] Figure 2 shows the weight matrices W for generating the QKV matrix. Q (210), WK (220), W V (230) is shown. Weight matrices W Q (210), W K (220), W V (230) may be included in at least one layer among the layers (11, 12, 13) of the foundation model (10).

[0075] Referring to Figure 2, the weight matrices W for generating the QKV matrix Q (210), W K (220), W V (230) Each can be decomposed into low-dimensional matrices. W to generate the Q matrix Q (210) are two low-dimensional matrices W QA (211) and W QB (212) can be decomposed into W. Similarly, to generate the K matrix, K (220) are two low-dimensional matrices W KA (221) and W KB (222) can be decomposed into W to generate the V matrix. V (230) are two low-dimensional matrices W VA (231) and W VB (232) can be decomposed into.

[0076] When fine-tuning the model according to the LoRA method, the weight matrices W included in the existing model (foundation model) Q (210), W K (220), W V (230) is frozen, and the decomposed low-dimensional matrices W QA (211), W QB (212), W KA (221), W KB (222), W VA (231) and W VB (232) Only can be updated.

[0077] As a result of performing fine tuning, the updated low-dimensional matrices are each WQA '(213), W QB '(214), W KA '(223), W KB '(224), W VA '(233) and W VB '(234) is said to be W QA '(213) and W QB The result of multiplying (214) is W Q ' and W KA '(223) and W KB The result of multiplying (224) is W K ' and W VA '(233) and W VB The result of multiplying (234) is W V ' is said.

[0078] W Q ', W K ' and W V ' can be said to be a weight matrix corresponding to the PEFT model, and W is the weight matrix of the foundation model. Q (210), W K (220) and W V (230) can be used in the generation of QKV matrices. For example, W Q (210), W K (220) and W V (230) W Q ', W K ' and W V By applying the matrices obtained by adding each of ' to the input embedding, a QKV matrix can be created.

[0079] According to one embodiment of the present disclosure, a QKV / W generation module, which is a generative model (e.g., a language model) learned in the process of performing fine tuning, can generate a QKV / W matrix. For example, the QKV / W required for executing the PEFT model can be generated by having the generative model (QKV / W generation module) learn the QKV / W matrices generated in the process of fine-tuning a foundation model into a PEFT model. The QKV / W generation module can be encrypted and then decrypted upon successful user authentication to generate a QKV / W matrix.

[0080] According to one embodiment of the present disclosure, the updated weight matrices W are obtained by performing fine tuning on the foundation model according to the LoRA method. Q ', W K ' and W V ' is encrypted and can be decrypted and used when executing the PEFT model. The detailed process is explained below with reference to Figures 3, 5, and 6.

[0081] 2. Description of the configuration and overall operation of the electronic device that runs the PEFT model (Figs. 3 and 4)

[0082] FIG. 3 is a diagram illustrating modules included in an electronic device according to an embodiment of the present disclosure. Referring to FIG. 3, an electronic device (1000) according to an embodiment of the present disclosure may include an input processing module (310), an authentication module (320), a PEFT model activation module (330), a flash memory (340), and a cache memory (350). In addition, the electronic device (1000) may include a PEFT model (30), and the PEFT model (30) may include a foundation model (10) and a PEFT layer (360).

[0083] The components (30, 310, 320, 330, 340, 350) included in the electronic device (1000) of FIG. 3 are components classified based on their functions or roles. The components (30, 310, 320, 330, 340, 350) of the electronic device (1000) of FIG. 3 may be software components implemented by the processor (1300) of the electronic device (1000) to be described later with reference to FIG. 4 executing a program stored in the memory (1400), or may be virtual components for which no matching hardware device actually exists. In other words, the operations performed by the processor (1300) of the electronic device (1000) by executing a program or instruction stored in the memory (1400) may be classified into a plurality of groups by function or purpose, and the entities performing the operations included in each classified group may be expressed as the components (30, 310, 320, 330, 340, 350) of FIG. 3. Accordingly, the operations described as being performed by the components (30, 310, 320, 330, 340, 350) of the electronic device (1000) illustrated in FIG. 3 may actually be viewed as being performed by the processor (1300) of the electronic device (1000) by executing a program or instruction stored in the memory (1400).

[0084] In FIG. 3, one electronic device (1000) is illustrated as including all of the components (30, 310, 320, 330, 340, 350), but this is not limited thereto, and at least some of the components (30, 310, 320, 330, 340, 350) may be implemented to be included in a separate device, or one of the components may be implemented to be included in another component. In this way, the components (30, 310, 320, 330, 340, 350) included in the electronic device (1000) according to one embodiment of the present disclosure may be hardware components or software components, and may be implemented in the form of various electronic devices (e.g., one electronic device or a combination of two or more electronic devices).

[0085] An electronic device (1000) according to one embodiment of the present disclosure may be a user's terminal (e.g., a smartphone, a laptop, a desktop, etc.), but is not limited thereto, and may also be a server that performs communication with the user's terminal.

[0086] The input processing module (310) is configured to determine a model to be executed based on the input of the user (1). The electronic device (1000) can receive a prompt requesting the execution of a specific function (e.g., AI function) from the user (1) through the input / output interface (1200) described later in FIG. 4. For example, the electronic device (1000) can receive a prompt in text form from the user (1) through a touch screen, or can receive a prompt in voice form from the user (1) through a microphone. In addition, the electronic device (1000) can obtain a prompt from the user (1) in various ways.

[0087] The input processing module (310) can determine whether the use of the PEFT model (30) is required to perform a task based on a prompt input by the user (1). In other words, the input processing module (310) can determine whether the use of the PEFT model (30) is required to process a prompt obtained from the user (1). For example, the input processing module (310) can determine whether the use of the PEFT model (30) is required based on a prompt obtained from the user (1). Alternatively, for example, the user (1) may request the use of the PEFT model (30) when inputting a prompt, and the input processing module (310) may determine that the use of the PEFT model (30) is required based on this request.

[0088] A user (1) may store and use multiple PEFT models in advance in the electronic device (1000) according to the intended use. In FIG. 3, the electronic device (1000) is illustrated as including one PEFT model (30), but may also include one or more PEFT models, and the input processing module (310) may determine which PEFT model among the multiple PEFT models is necessary to process the user's (1) prompt.

[0089] If the input processing module (310) determines that the PEFT model (30) is not necessary to process the user's (1) prompt, it can request the foundation model (10) to execute the function requested by the user (1). That is, the input processing module (310) can perform the operation by having the input embedding generated based on the user's (1) prompt pass only through the layers (11, 12, 13) of the foundation model (10).

[0090] As previously described with reference to FIGS. 1 and 2, when fine tuning is performed on the foundation model (10) according to the PEFT technique, the PEFT model can be generated in such a way that new layers (PEFT layers) are added while the layers (11, 12, 13) of the foundation model (10) are maintained as they are. Accordingly, the input processing module (310) can process the input embedding using only the layers (11, 12, 13) of the foundation model (10) excluding the PEFT layer (360).

[0091] If the input processing module (310) determines that the PEFT model (30) is required to process the user's (1) prompt, it may request the authentication module (320) to perform user authentication. In order to strengthen the security of the PEFT model (30), the electronic device (1000) according to one embodiment of the present disclosure may execute the PEFT model (30) only when it is confirmed that the user (1) has the authority to execute the PEFT model (30). The input processing module (310) may request authentication from the authentication module (320) to confirm whether the user (1) has the authority to execute the PEFT model (30).

[0092] The authentication module (320) is a configuration for performing user authentication. The authentication module (320) can perform authentication on the user (1) to confirm whether the user (1) has the authority to execute the PEFT model (30).

[0093] According to one embodiment of the present disclosure, authentication to verify whether a user (1) has the authority to execute the PEFT model (30) may be performed in a manner different from authentication for use of the electronic device (1000). This will be described in detail using a first authentication method (e.g., authentication using an account ID, password, etc.) with relatively low security and a second authentication method (e.g., biometric authentication) with relatively high security. Various authentication methods may be used as the first and second authentication methods.

[0094] When a user (1) unlocks the electronic device (1000) according to the first authentication method when starting to use the electronic device (1000), the authentication module (320) can additionally perform authentication on the user (1) according to the second authentication method when execution of the PEFT model (30) is required.

[0095] However, if the user (1) unlocks the electronic device (1000) according to the second authentication method when the user (1) starts using the electronic device (1000), the authentication module (320) can determine that the user (1) has the authority to execute the PEFT model (30) without performing additional authentication. (i.e., transmit (provide) the authentication result as 'success' to the PEFT model activation module (330))

[0096] The PEFT model activation module (330) is configured to activate the PEFT model (30) based on the authentication result. When the PEFT model activation module (330) receives the authentication result for the user (1) from the authentication module (320), it determines whether to activate the PEFT model (30) based on the authentication result.

[0097] If authentication for the user (1) is successful, the PEFT model activation module (330) can perform a process for executing the PEFT model (30).

[0098] According to one embodiment of the present disclosure, the PEFT model activation module (330) can generate the QKV / W matrix required to execute the PEFT model (30). The QKV / W generation module included in the PEFT model activation module (330) can generate the QKV / W matrix from a prompt input by the user (1) because it learns the QKV / W matrices generated in the process of fine-tuning the PEFT model (30) as described above. Specifically, in the process of fine-tuning the foundation model (10) using the PEFT technique, the generation model can be trained to generate the QKV / W matrix required to execute the PEFT model (30), and the generated model (QKV / W generation module) trained in this way can be included in the PEFT model activation module (330) in an encrypted state. If the user authentication is successful, the generated model (QKV / W generation module) included in the PEFT model activation module (330) can be decrypted to generate the QKV / W matrix.

[0099] The PEFT model activation module (330) can temporarily store the generated QKV / W matrix in the cache memory (350) so that the PEFT model (30) can use it. The PEFT model activation module (330) can also use the personal data of the user (1) stored in the flash memory (340) when generating the QKV / W matrix. The PEFT model activation module (330) can also be implemented to be included in the PEFT model (30).

[0100] According to one embodiment of the present disclosure, a weight matrix for generating a QKV matrix may be stored in the flash memory (340), and the PEFT model activation module (330) may cache the weight matrix stored in the flash memory (340). Specifically, the PEFT model activation module (330) may temporarily store a weight matrix corresponding to a PEFT model (30) stored in the flash memory (340) in the cache memory (350), and use it whenever the PEFT model (30) requires it.

[0101] The weight matrix being cached may be a matrix required for executing the PEFT model (30). That is, the weight matrix generated as a result of performing fine tuning on the foundation model (10) using the PEFT technique may be stored in flash memory (340) and then cached by the PEFT model activation module (330).

[0102] According to one embodiment of the present disclosure, personal data of a user (1) can be used in the process of fine-tuning a foundation model (10) using the PEFT technique, and thus, personal data of the user (1) can be reflected in a weight matrix corresponding to the PEFT model (30).

[0103] According to one embodiment of the present disclosure, the weight matrix is ​​encrypted and stored in a flash memory (340), decrypted by a PEFT model activation module (330), and stored in a cache memory (350) so that it can be used by the PEFT model (30).

[0104] As explained above, the weight matrix stored in the flash memory (340) is a matrix W for generating the QKV matrix required for executing the PEFT model (30). Q ', W K ' and W V ' may be. Therefore, the PEFT layer (360) temporarily stores the weight matrix W in the cache memory (350). Q ', W K ' and W V ' can be used to generate a QKV matrix, and the attention mechanism can be performed using the generated QKV matrix.

[0105] Weight matrix W for generating QKV matrix Q ', W K ' and W V ' Instead, another type of weight matrix may be stored in the flash memory (340) and cached by the PEFT model activation module (330). However, the weight matrix cached by the PEFT model activation module (330) may be a weight matrix corresponding to the PEFT model (30). In other words, the weight matrix cached by the PEFT model activation module (330) may be a weight matrix generated as a result of the foundation model (10) being fine-tuned according to the PEFT technique.

[0106] According to one embodiment of the present disclosure, the PEFT model activation module (330) can generate a QKV / W matrix based on the personal data of the user (1) stored in the flash memory (340), and cache the generated QKV / W matrix. That is, the PEFT model activation module (330) can temporarily store the generated QKV / W matrix in the cache memory (350) and use it whenever the PEFT model (30) needs it. The PEFT model activation module (330) can generate a weight matrix W stored in the flash memory (340). Q ', W K ' and W V ', a QKV matrix can be generated from the personal data of the user (1).

[0107] The PEFT layer (360) of the PEFT model (30) may include a layer for performing an attention mechanism, and thus may perform an attention mechanism using a QKV / W matrix stored in a cache memory (350), and provide the result to another layer included in the PEFT layer (360), or transmit it to any one of the layers (11, 12, 13) included in the foundation model (10).

[0108] Alternatively, a layer in which at least one of the layers (11, 12, 13) included in the foundation model (10) is combined with a PEFT layer (360) may perform an attention mechanism using the QKV / W matrix stored in the cache memory (350) and provide the result to another layer. In addition, a combination of a PEFT layer (360) and a foundation model (10) may perform an attention mechanism using the QKV / W matrix and perform inference based on the result in various ways.

[0109] If authentication for the user (1) fails, the PEFT model activation module (330) can generate a random weight matrix based on a random seed and transmit the generated weight matrix to the PEFT layer (360).

[0110] In this way, if the QKV / W matrix is ​​not generated using the generative model (QKV / W generation module) learned during the fine tuning process, the effect of the fine tuning is not reflected, and the PEFT model (30) operates as a dummy model. Alternatively, if the PEFT layer (360) performs an operation (e.g., an operation to generate a QKV matrix) using an arbitrary weight matrix other than the weight matrix generated through the fine tuning, the effect of the fine tuning is not reflected, and the PEFT model (30) operates as a dummy model. Therefore, the security of the PEFT model (30) can be maintained.

[0111] When processing of the prompt entered by the user (1) is completed, the PEFT model activation module (330) can delete data (e.g., QKV / W matrix, etc.) for execution of the PEFT model (30) stored in the cache memory (350).

[0112] The specific process by which the PEFT model activation module (330) executes the PEFT model (30) according to the authentication result is described in detail below with reference to FIGS. 5 to 8.

[0113] FIG. 4 is a diagram illustrating a hardware configuration included in an electronic device according to one embodiment of the present disclosure. Referring to FIG. 4, an electronic device (1000) according to one embodiment may include a communication interface (1100), an input / output interface (1200), a processor (1300), and a memory (1400). However, the components of the electronic device (1000) are not limited to the above-described examples, and the electronic device (1000) may include more or fewer components than the above-described components. Some or all of the communication interface (1100), the input / output interface (1200), the processor (1300), and the memory (1400) may be implemented in the form of a single chip.

[0114] The communication interface (1100) is a configuration for transmitting and receiving signals (such as control commands and data) with an external device via wire or wirelessly, and may be implemented to include a communication chipset that supports various communication protocols. The communication interface (1100) may receive signals from the outside and output them to the processor (1300), or transmit signals output from the processor (1300) to the outside. The electronic device (1000) may communicate with external devices via the communication interface (1100).

[0115] The input / output interface (1200) may include an input interface (e.g., a touch screen, a keyboard, a microphone, etc.) for receiving commands or information from a user (1), and an output interface (e.g., a display panel, a speaker, etc.) for displaying the results of execution of an operation according to a user's command or the status of the electronic device (1000). According to one embodiment of the present disclosure, the electronic device (1000) may receive a prompt and input data from the user (1) through the input / output interface (1200), and when a task is completed, may output the results of performing the task (e.g., an answer to a question included in a prompt, an image edited according to a request of a prompt, etc.) through the input / output interface (1200).

[0116] The processor (1300) controls a series of processes to operate the electronic device (1000) according to the embodiments described below, and may be composed of one or more processors. The one or more processors included in the processor (1300) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The one or more processors included in the processor (1300) may be a general-purpose processor such as a Central Processing Unit (CPU), a Micro Processor Unit (MPU), an Application Processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a Graphics Processing Unit (GPU), a Vision Processing Unit (VPU), an artificial intelligence-only processor such as a Neural Processing Unit (NPU), or a communication-only processor such as a Communication Processor (CP). When the one or more processors included in the processor (1300) are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0117] The processor (1300) can write data to the memory (1400) or read data stored in the memory (1400), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (1400). Accordingly, the processor (1300) can perform operations described in the following embodiments, and operations described as being performed by the electronic device (1000) or components (30, 310, 320, 330, 340, 350) included in the electronic device (1000) in the present disclosure can be regarded as being performed by the processor (1300) unless otherwise specifically described.

[0118] The memory (1400) is a configuration for storing various programs or data, and may be configured as a storage medium such as a ROM, a RAM, a hard disk, a CD-ROM, and a DVD, or a combination of storage media. The memory (1400) may not exist separately and may be configured to be included in the processor (1300). The memory (1400) may be configured as a volatile memory, a non-volatile memory, or a combination of volatile memory and non-volatile memory. A program or at least one instruction for performing operations according to embodiments described below may be stored in the memory (1400). The memory (1400) may also provide stored data to the processor (1300) at the request of the processor (1300). The memory (1400) may include a flash memory (340) and a cache memory (350), as illustrated in FIG. 3.

[0119] 3. (First embodiment) Generate and cache a QKV / W matrix corresponding to the PEFT model.

[0120] FIGS. 5 and 6 are diagrams for explaining a process of activating a PEFT model by generating and caching a QKV / W matrix based on a user authentication result by an electronic device according to an embodiment of the present disclosure. Referring to FIG. 5, a PEFT model activation module (330) may include a QKV / W generation module (331) and a QKV / W search module (332). In FIG. 5, the PEFT model activation module (330) is illustrated as existing separately from the PEFT model (30), but according to an embodiment of the present disclosure, the PEFT model activation module (330) may be implemented to be included in the PEFT model (30). First, referring to FIG. 5, a process of activating a PEFT model (30) by an electronic device (1000) when user authentication is successful will be described.

[0121] As described above, if execution of the PEFT model (30) is required, the authentication module (320) can perform authentication and provide the authentication result to the QKV / W generation module (331). If authentication is successful, the electronic device (1000) can unlock the QKV / W generation module (331) included in the PEFT model activation module (330). The unlocked QKV / W generation module (331) can generate the QKV / W matrix required for execution of the PEFT model (30).

[0122] As described above, the QKV / W generation module (331) may be a generation model (e.g., language model) learned in the process of fine-tuning the foundation model (10) into the PEFT model (30). The QKV / W generation module (331) may generate the QKV / W matrix required for executing the PEFT model (30) by learning the QKV / W matrices generated in the process of fine-tuning the foundation model (10) into the PEFT model (30).

[0123] According to one embodiment of the present disclosure, the QKV / W generation module (331) may be included in the PEFT model (30).

[0124] According to one embodiment of the present disclosure, a user's personal data may be stored in the flash memory (340). The QKV / W generation module (331) may obtain the user's personal data from the flash memory (340), use the data to generate a QKV / W matrix, and then store the generated QKV / W matrix in the cache memory (350). For example, the QKV / W generation module (331) may generate a QKV / W matrix based on a prompt entered by the user and the personal data stored in the flash memory (340).

[0125] The QKV / W search module (332) can obtain the QKV / W matrix stored in the cache memory (350) and provide it to the PEFT layer (360). According to one embodiment of the present disclosure, the QKV / W search module (332) can obtain the QKV / W matrix from the cache memory (350) and provide it to the PEFT layer (360) at a time when an operation using the QKV / W matrix must be performed in the PEFT model (30).

[0126] The PEFT layer (360) can provide personalized services to users by performing operations using the received QKV / W matrix. Once the QKV / W matrix is ​​generated, the PEFT model (30) can perform subsequent operation processes using the generated QKV / W matrix.

[0127] When processing of a user's request is completed, the QKV / W generation module (331) can prevent the QKV / W matrix from being leaked to the outside by deleting the QKV / W matrix stored in the cache memory (350).

[0128] This time, referring to FIG. 6, a process in which an electronic device (1000) operates a PEFT model (30) as a dummy model when user authentication fails is described.

[0129] If the authentication module (320) performs authentication on the user and the authentication fails, the electronic device (1000) does not unlock the QKV / W generation module (331) included in the PEFT model activation module (330). The locked QKV / W generation module (331) does not perform any operation, and therefore, the QKV / W matrix required to execute the PEFT model (30) is not generated.

[0130] At a point in time when an operation using a QKV / W matrix must be performed in the PEFT model (30), the QKV / W search module (332) can access the cache memory (350) to search for the QKV / W matrix. However, since the QKV / W matrix is ​​not stored in the cache memory (350), the QKV / W search module (332) cannot obtain the QKV / W matrix.

[0131] If the QKV / W search module (332) fails to obtain a QKV / W matrix, it can generate a random QKV / W matrix based on a random seed and provide the generated random QKV / W matrix to the PEFT layer (360).

[0132] The PEFT layer (360) can perform subsequent computational processes using any QKV / W matrix received from the QKV / W search module (332). In this way, the effect of fine tuning is not reflected in any QKV / W matrix other than the QKV / W matrix generated by the QKV / W generation module (331) learned through fine tuning, and thus the PEFT model (30) operates as a dummy model. Therefore, the security of the PEFT model (30) can be maintained.

[0133] 4. (Example 2) Caching the weight matrix corresponding to the PEFT model

[0134] FIGS. 7 and 8 are diagrams illustrating a process in which an electronic device according to one embodiment of the present disclosure activates a PEFT model by caching a weight matrix based on a user authentication result.

[0135] In the embodiment illustrated in FIGS. 7 and 8, the PEFT model activation module (330) generates a weight matrix W for generating a QKV matrix. Q ', W K ' and W V ' is cached, and the names of the detailed components (331, 332) included in the PEFT model activation module (330) are set accordingly, but differently, the PEFT model activation module (330) can also activate the PEFT model (30) by caching a different type of weight matrix.

[0136] Referring to FIG. 7, the PEFT model activation module (330) may include a QKV / W generation module (331) and a QKV / W search module (332). First, referring to FIG. 7, a process for an electronic device (1000) to activate a PEFT model (30) when user authentication is successful will be described.

[0137] As described above, if execution of the PEFT model (30) is required, the authentication module (320) can perform authentication and provide the authentication result to the QKV / W generation module (331). If authentication is successful, the electronic device (1000) can unlock the QKV / W generation module (331) included in the PEFT model activation module (330). The unlocked QKV / W generation module (331) can obtain the weight matrix stored in the flash memory (340).

[0138] According to one embodiment of the present disclosure, the flash memory (340) includes a weight matrix W for generating a QKV matrix required for executing the PEFT model (30). Q ', W K ' and W V' can be encrypted and stored. The QKV / W generation module (331) generates the encrypted weight matrix W from the flash memory (340). Q ', W K ' and W V After obtaining and decrypting ', it can be stored in the cache memory (350). Of course, the weight matrix W Q ', W K ' and W V ' may be stored in the flash memory (340) in an unencrypted state and cached by the QKV / W generation module (331).

[0139] The QKV / W search module (332) decrypts the weight matrix W stored in the cache memory (350). Q ', W K ' and W V ' can be obtained and provided to the PEFT layer (360). According to one embodiment of the present disclosure, the QKV / W search module (332) obtains the weight matrix W from the PEFT model (30). Q ', W K ' and W V At the point when an operation using ' must be performed, the weight matrix W decoded from the cache memory (350) Q ', W K ' and W V ' can be obtained and provided to the PEFT layer (360).

[0140] The PEFT layer (360) decrypts the weight matrix W Q ', W K ' and W V ' can be used to generate a QKV matrix. In detail, the PEFT layer (360) adds a weight matrix W to the input embedding generated based on the prompt entered by the user. Q ', W K ' and W V By applying ', we can generate a QKV matrix.

[0141] According to one embodiment of the present disclosure, personal data is stored in the flash memory (340), and the stored personal data may be used to generate a QKV matrix. For example, the electronic device (1000) generates an input embedding based on a prompt entered by a user and the personal data stored in the flash memory (340), and the PEFT layer (360) adds a weight matrix W to the input embedding thus generated. Q ', W K ' and W V We can also generate a QKV matrix by applying '.

[0142] Once the QKV matrix is ​​generated, the PEFT model (30) can perform subsequent computational processes using the generated QKV matrix. According to one embodiment of the present disclosure, the QKV matrix generated by the PEFT layer (360) can also be stored in the cache memory (350).

[0143] When processing of a user's request is completed, the QKV / W generation module (331) can prevent the weight matrix from being leaked to the outside by deleting the weight matrix stored in the cache memory (350).

[0144] This time, referring to FIG. 8, a process in which an electronic device (1000) operates a PEFT model (30) as a dummy model when user authentication fails is described.

[0145] If the authentication module (320) performs authentication on the user and the authentication fails, the electronic device (1000) does not unlock the QKV / W generation module (331) included in the PEFT model activation module (330). The locked QKV / W generation module (331) does not perform any operation, and therefore, the weight matrix W required to execute the PEFT model (30) Q ', W K ' and W V ' is encrypted and stored in flash memory (340).

[0146] Weight matrix W in PEFT model (30) Q ', W K ' and W V At the point when an operation using ' must be performed, the QKV / W search module (332) accesses the cache memory (350) to retrieve the weight matrix W Q ', W K ' and W V ' can be searched. However, the weight matrix W is stored in the cache memory (350). Q ', W K ' and W V Since ' is not stored, the QKV / W search module (332) uses the weight matrix W Q ', W K ' and W V ' is not obtained.

[0147] The QKV / W search module (332) uses a weight matrix W Q ', W K ' and W V If the acquisition of ' fails, a random weight matrix can be generated based on a random seed, and the generated random weight matrix can be provided to the PEFT layer (360).

[0148] The PEFT layer (360) generates a QKV matrix by applying an arbitrary weight matrix received from the QKV / W search module (332) to the input embedding, and the PEFT model (30) can perform subsequent computational processes using the generated QKV matrix. In this way, if the PEFT layer (360) generates a QKV matrix using an arbitrary weight matrix rather than a weight matrix generated through fine tuning, the effect of fine tuning is not reflected in the QKV matrix, and thus the PEFT model (30) operates as a dummy model. Therefore, the security of the PEFT model (30) can be maintained.

[0149] 5. (Example 3) Caching the QKV matrix generated using personal data

[0150] FIGS. 9 and 10 are diagrams illustrating a process in which an electronic device according to one embodiment of the present disclosure activates a PEFT model by caching a QKV matrix based on a user authentication result.

[0151] First, referring to FIG. 9, a process for activating the PEFT model (30) by an electronic device (1000) when user authentication is successful is described.

[0152] If the authentication module (320) performs authentication on the user and the authentication is successful, the electronic device (1000) can unlock the QKV / W generation module (331) included in the PEFT model activation module (330).

[0153] The unlocked QKV / W generation module (331) can obtain the user's personal data stored in the flash memory (340) and generate a QKV matrix using the obtained personal data. The specific process by which the QKV / W generation module (331) generates a QKV matrix using the personal data is as follows.

[0154] The electronic device (1000) can generate an input embedding based on a prompt entered by the user and personal data stored in the flash memory (340). First, the electronic device (1000) can generate an input sequence by combining the prompt and the personal data. For example, if the prompt entered by the user and the personal data stored in the flash memory (340) are as follows, the electronic device (1000) can generate an input sequence by combining them.

[0155] Prompt: "Send a greeting message to my friends in my contacts."

[0156] Personal data: phone numbers and group information of people stored in your contacts

[0157] Input sequence: Send a greeting message to the phone numbers of people in your contacts' friend group.

[0158] When an input sequence is generated, the electronic device (1000) can obtain an embedding matrix by performing an embedding transformation on the input sequence, and the embedding matrix thus obtained corresponds to the input embedding.

[0159] The QKV / W generation module (331) generates a pre-stored weight matrix W Q ', W K ' and W V By applying ' to the input embedding, a QKV matrix can be generated, and the generated QKV matrix can be stored in the cache memory (350). At this time, the weight matrix W Q ', W K ' and W V ' may be a matrix generated as a result of performing fine tuning using the PEFT technique on the foundation model (10) and may be stored in flash memory (340).

[0160] The QKV / W search module (332) can obtain the QKV matrix stored in the cache memory (350) and provide it to the PEFT layer (360). According to one embodiment of the present disclosure, the QKV / W search module (332) can obtain the QKV matrix from the cache memory (350) and provide it to the PEFT layer (360) at a time when an operation using the QKV matrix should be performed in the PEFT model (30). The PEFT model (30) can perform subsequent operation processes using the received QKV matrix.

[0161] Once processing of the user's request is completed, the QKV / W generation module (331) can maintain security by deleting the QKV matrix stored in the cache memory (350).

[0162] This time, referring to FIG. 10, a process in which an electronic device (1000) operates a PEFT model (30) as a dummy model when user authentication fails is described.

[0163] If the authentication module (320) performs authentication on the user and the authentication fails, the electronic device (1000) does not unlock the QKV / W generation module (331) included in the PEFT model activation module (330). The locked QKV / W generation module (331) does not perform any operation, and therefore, the QKV matrix required for executing the PEFT model (30) is not generated.

[0164] At a point in time when an operation using a QKV matrix must be performed in the PEFT model (30), the QKV / W search module (332) can access the cache memory (350) to search for the QKV matrix. However, since the QKV matrix is ​​not stored in the cache memory (350), the QKV / W search module (332) cannot obtain the QKV matrix.

[0165] If the QKV / W search module (332) fails to obtain a QKV matrix, it can generate a random QKV matrix based on a random seed and provide the generated random QKV matrix to the PEFT layer (360).

[0166] The PEFT model (30) can perform subsequent computational processes using an arbitrary QKV matrix received from the QKV / W search module (332). The randomly generated QKV matrix does not reflect the effects of personal data or fine tuning, and thus the PEFT model (30) operates as a dummy model. Therefore, the security of the user's personal data and the PEFT model (30) can be maintained.

[0167] 6. Describe the process by referring to the flowcharts.

[0168] FIGS. 11 to 19 are flowcharts illustrating a method for processing a user request using a PEFT model according to embodiments of the present disclosure. Hereinafter, a method for processing a user request using a PEFT model according to embodiments of the present disclosure will be described with reference to the flowcharts of FIGS. 11 to 19. The steps included in the flowcharts of FIGS. 11 to 19 are performed by the electronic device (1000) of FIGS. 3 and 4, and therefore, even if the contents described above with reference to FIGS. 3 to 10 are omitted below, they can be equally applied to FIGS. 11 to 19.

[0169] Referring to FIG. 11, the process may begin at step 1101 when the electronic device (1000) obtains a prompt from the user. At step 1102, the electronic device (1000) may determine whether execution of the PEFT model is required to perform a task according to the prompt entered by the user.

[0170] If the judgment result indicates that execution of the PEFT model is not necessary, the electronic device (1000) may proceed to step 1104 to execute the foundation model. That is, the electronic device (1000) may perform the task requested by the user using only the pre-trained foundation model, without using the fine-tuned PEFT model.

[0171] If the judgment result indicates that the PEFT model needs to be executed, the electronic device (1000) may proceed to step 1103 to perform user authentication. The electronic device (1000) may perform user authentication to confirm whether the user has the authority to execute the PEFT model.

[0172] According to one embodiment of the present disclosure, authentication to determine whether a user has the authority to execute a PEFT model may be performed in a different manner from authentication for use of the electronic device (1000). For example, if a user unlocks the electronic device (1000) using a first authentication method (e.g., password authentication) when starting to use the electronic device (1000), the electronic device (1000) may additionally authenticate the user using a second authentication method (e.g., biometric authentication) when execution of the PEFT model is required. In this case, the second authentication method may be a method with higher security than the first authentication method. However, if the user unlocks the electronic device (1000) using the second authentication method when starting to use the electronic device (1000), the electronic device (1000) may determine that the user has the authority to execute the PEFT model without performing additional authentication.

[0173] At step 1105, the electronic device (1000) can determine whether authentication was successful.

[0174] If the authentication is successful, the electronic device (1000) can proceed to step 1106 to obtain at least one matrix corresponding to the PEFT model. According to one embodiment of the present disclosure, the electronic device (1000) can generate a QKV / W matrix required for executing the PEFT model by using a generation model (QKV / W generation module) learned in the fine tuning process of the PEFT model. The generation model is encrypted and can be decrypted when user authentication is successful to generate the QKV / W matrix. In addition, according to one embodiment of the present disclosure, the electronic device (1000) can decrypt a weight matrix for generating the QKV matrix. That is, in step 1106, the electronic device (1000) can decrypt a configuration required for executing the PEFT model.

[0175] The detailed steps included in step 1106 are illustrated in Figures 12 and 14.

[0176] Referring to FIG. 12, in step 1201, the electronic device (1000) can unlock the QKV / W generation module. At this time, the QKV / W generation module may be a generation model learned using QKV / W matrices generated in the process of fine-tuning the foundation model into a PEFT model.

[0177] The unlocked QKV / W generation module at step 1202 can generate a QKV / W matrix and store the generated QKV / W matrix in cache memory.

[0178] At step 1203, the QKV / W search module of the electronic device (1000) can provide the QKV / W matrix stored in the cache memory to the PEFT model.

[0179] Referring to FIG. 14, at step 1401, the electronic device (1000) can unlock the QKV / W generation module.

[0180] In step 1402, the unlocked QKV / W generation module can store the weight matrix for generating the QKV matrix in the cache memory. At this time, the weight matrix may be a weight matrix generated as a result of fine-tuning a pre-trained foundation model using the PEFT technique.

[0181] Detailed steps included in step 1402 are illustrated in FIG. 15. Referring to FIG. 15, the QKV / W generation module of the electronic device (1000) can request a weight matrix for generating a QKV matrix from the flash memory of the electronic device (1000) in step 1501, and obtain an encrypted weight matrix from the flash memory in step 1502.

[0182] At step 1503, the QKV / W generation module of the electronic device (1000) can decrypt the acquired weight matrix and then store it in the cache memory of the electronic device (1000).

[0183] Returning to FIG. 14 again, at step 1403, the QKV / W search module of the electronic device (1000) can provide the weight matrix stored in the cache memory to the PEFT model.

[0184] Returning to FIG. 11, at step 1108, the electronic device (1000) may perform a task requested by the user by executing a PEFT model using at least one acquired matrix. According to one embodiment of the present disclosure, the electronic device (1000) may generate a QKV / W matrix using personal data stored in a flash memory, and execute the PEFT model using the generated QKV / W matrix. Alternatively, according to one embodiment of the present disclosure, the electronic device (1000) may obtain a QKV matrix using personal data stored in a flash memory and an acquired weight matrix, and execute the PEFT model using the acquired QKV matrix.

[0185] The detailed steps included in step 1108 are illustrated in Figures 13 and 16.

[0186] Referring to FIG. 13, in step 1301, the electronic device (1000) can control the PEFT model to perform the attention mechanism using the acquired QKV / W matrix.

[0187] At step 1302, the electronic device (1000) can control the PEFT model to perform inference based on the result of performing the attention mechanism.

[0188] When the user-requested task is completed, the QKV / W / W generation module of the electronic device (1000) can delete the QKV / W matrix stored in the cache memory at step 1303.

[0189] Referring to FIG. 16, in step 1601, the electronic device (1000) can obtain a QKV matrix by applying a weight matrix to an input embedding generated based on a prompt.

[0190] At step 1602, the electronic device (1000) can control the PEFT model to perform the attention mechanism using the acquired QKV matrix.

[0191] At step 1603, the electronic device (1000) can control the PEFT model to perform inference based on the result of performing the attention mechanism.

[0192] When the user-requested task is completed, the QKV / W generation module of the electronic device (1000) can delete the weight matrix stored in the cache memory at step 1604. If the QKV matrix is ​​also stored in the cache memory, the QKV / W generation module can also delete the QKV matrix from the cache memory.

[0193] Meanwhile, if authentication fails in step 1105 of FIG. 11, the electronic device (1000) can execute the PEFT model using a randomly generated QKV / W matrix, proceeding to step 1107. At this time, the PEFT model operates as a dummy model.

[0194] Referring to FIG. 17, the process may begin at step 1701 when the electronic device (1000) obtains a prompt from the user. At step 1702, the electronic device (1000) may determine whether execution of the PEFT model is required to perform a task according to the prompt entered by the user.

[0195] If the judgment result indicates that execution of the PEFT model is not necessary, the electronic device (1000) can proceed to step 1704 and execute the foundation model. That is, the electronic device (1000) can perform the task requested by the user using only the pre-trained foundation model, without using the fine-tuned PEFT model.

[0196] If the judgment result indicates that the PEFT model needs to be executed, the electronic device (1000) may proceed to step 1703 to perform user authentication. The electronic device (1000) may perform user authentication to confirm whether the user has the authority to execute the PEFT model.

[0197] According to one embodiment of the present disclosure, authentication to determine whether a user has the authority to execute a PEFT model may be performed in a different manner from authentication for use of the electronic device (1000). For example, if a user unlocks the electronic device (1000) using a first authentication method (e.g., password authentication) when starting to use the electronic device (1000), the electronic device (1000) may additionally authenticate the user using a second authentication method (e.g., biometric authentication) when execution of the PEFT model is required. In this case, the second authentication method may be a method with higher security than the first authentication method. However, if the user unlocks the electronic device (1000) using the second authentication method when starting to use the electronic device (1000), the electronic device (1000) may determine that the user has the authority to execute the PEFT model without performing additional authentication.

[0198] At step 1705, the electronic device (1000) can determine whether authentication was successful.

[0199] If the authentication is successful, the electronic device (1000) can proceed to step 1706 and obtain a QKV matrix corresponding to the PEFT model based on the user's personal data. That is, if the user authentication is successful, the electronic device (1000) can generate a QKV matrix for executing the PEFT model. At this time, the electronic device (1000) can also obtain a weight matrix corresponding to the PEFT model in order to generate the QKV matrix. A specific method by which the electronic device (1000) obtains the weight matrix corresponding to the PEFT model is as described above with reference to FIGS. 14 and 15.

[0200] The detailed steps included in step 1706 are illustrated in FIG. 18. Referring to FIG. 18, in step 1801, the electronic device (1000) can unlock the QKV / W generation module.

[0201] The unlocked QKV / W generation module at step 1802 can obtain personal data from the flash memory of the electronic device (1000).

[0202] At step 1803, the QKV / W generation module of the electronic device (1000) can generate a QKV matrix based on personal data and then store it in cache memory.

[0203] At step 1804, the QKV / W search module of the electronic device (1000) can provide the QKV matrix stored in the cache memory to the PEFT model.

[0204] Returning to FIG. 17 again, at step 1708, the electronic device (1000) can execute the PEFT model using the acquired QKV matrix.

[0205] The detailed steps included in step 1708 are illustrated in FIG. 19. Referring to FIG. 19, in step 1901, the electronic device (1000) can control the PEFT model to perform the attention mechanism using the acquired QKV matrix.

[0206] At step 1902, the electronic device (1000) can control the PEFT model to perform inference based on the result of performing the attention mechanism.

[0207] At step 1903, the QKV / W generation module of the electronic device (1000) can delete the QKV matrix stored in the cache memory.

[0208] Meanwhile, if authentication fails after determining whether authentication was successful at step 1705, the process proceeds to step 1707, where the electronic device (1000) can execute the PEFT model using a randomly generated QKV matrix. At this time, the PEFT model operates as a dummy model.

[0209] According to the embodiments described above, the PEFT model reflecting the user's personal data can be expected to have the effect of strengthening security by being executed only when the user's authentication is successful.

[0210] A method for processing a user request using a PEFT model according to an embodiment of the present disclosure may include a step of an electronic device obtaining a prompt from a user, a step of the electronic device determining whether execution of a PEFT model is required to process the prompt, a step of the electronic device executing a foundation model if execution of the PEFT model is not required and performing user authentication if execution of the PEFT model is required, a step of the electronic device obtaining at least one matrix corresponding to the PEFT model if the user authentication is successful, and a step of the electronic device executing the PEFT model using the at least one matrix.

[0211] According to one embodiment, the step of obtaining at least one matrix corresponding to the PEFT model may include the steps of unlocking a QKV / W generation module, the step of allowing the QKV / W generation module to generate a QKV / W matrix, storing the generated QKV / W matrix in a cache memory, and the step of allowing a QKV / W retrieval module to provide the QKV / W matrix stored in the cache memory to the PEFT model.

[0212] According to one embodiment, the step of executing the PEFT model using the at least one matrix may include the step of the PEFT model performing an attention mechanism using the generated QKV / W matrix, the step of the PEFT model performing inference based on a result of performing the attention mechanism, and the step of the QKV / W generation module deleting the QKV / W matrix stored in the cache memory.

[0213] According to one embodiment, the QKV / W generation module may be a generative model learned using QKV / W matrices generated in the process of fine-tuning the foundation model into the PEFT model.

[0214] According to one embodiment, the weight matrix may be a matrix for generating a QKV matrix required for executing the PEFT model.

[0215] According to one embodiment, the step of obtaining the at least one weight matrix may include the steps of unlocking a QKV / W generation module, storing a weight matrix for generating a QKV matrix in a cache memory by the QKV / W generation module, and providing the weight matrix stored in the cache memory by a QKV / W retrieval module to a PEFT model.

[0216] According to one embodiment, the step of storing in the cache memory may include a step in which the QKV / W generation module requests a weight matrix for generating the QKV matrix from a flash memory, a step in which the QKV / W generation module obtains an encrypted weight matrix from the flash memory, and a step in which the QKV / W generation module decrypts the obtained weight matrix and then stores it in the cache memory.

[0217] According to one embodiment, the step of executing the PEFT model may include the step of obtaining the QKV matrix by applying the at least one weight matrix to an input embedding generated based on the prompt, the step of controlling the PEFT model to perform an attention mechanism using the QKV matrix, the step of controlling the PEFT model to perform inference based on a result of performing the attention mechanism, and the step of the QKV / W generation module deleting a weight matrix stored in the cache memory.

[0218] According to one embodiment, the step of obtaining the QKV matrix may include the steps of obtaining personal data of a user stored in a flash memory, generating an input embedding based on the prompt and the personal data, and obtaining the QKV matrix by applying the at least one weight matrix to the input embedding.

[0219] An electronic device according to one embodiment of the present disclosure includes a memory in which a program or at least one instruction is stored, and at least one processor, and the at least one processor executes the program or at least one instruction stored in the memory, whereby the electronic device obtains a prompt from a user, determines whether execution of a PEFT model is required to process the prompt, executes a foundation model if execution of the PEFT model is not required, performs user authentication if execution of the PEFT model is required, and, if the user authentication is successful, obtains at least one matrix corresponding to the PEFT model, and then executes the PEFT model using the at least one matrix.

[0220] According to one embodiment, the electronic device may unlock a QKV / W generation module to obtain at least one matrix corresponding to the PEFT model, cause the QKV / W generation module to generate a QKV / W matrix, store the generated QKV / W matrix in a cache memory, and then cause a QKV / W retrieval module to provide the QKV / W matrix stored in the cache memory to the PEFT model.

[0221] According to one embodiment, when the electronic device executes the PEFT model using the at least one matrix, the PEFT model performs an attention mechanism using the obtained QKV / W matrix, and after the PEFT model performs inference based on a result of performing the attention mechanism, the QKV / W generation module can delete the QKV / W matrix stored in the cache memory.

[0222] According to one embodiment, the QKV / W generation module may be a generative model learned using QKV / W matrices generated in the process of fine-tuning the foundation model into the PEFT model.

[0223] According to one embodiment, the weight matrix may be a matrix for generating a QKV matrix required for executing the PEFT model.

[0224] According to one embodiment, in obtaining the at least one weight matrix, the electronic device unlocks a QKV / W generation module, stores a weight matrix for generating a QKV matrix in a cache memory by the QKV / W generation module, and then a QKV / W retrieval module provides the weight matrix stored in the cache memory to a PEFT model.

[0225] According to one embodiment, when storing the weight matrix in the cache memory, the electronic device can control the QKV / W generation module to request a weight matrix for generating the QKV matrix from a flash memory, and after the QKV / W generation module obtains an encrypted weight matrix from the flash memory, the QKV / W generation module to decrypt the obtained weight matrix and then store it in the cache memory.

[0226] According to one embodiment, the electronic device may obtain the QKV matrix by applying the at least one weight matrix to an input embedding generated based on the prompt when executing the PEFT model, control the PEFT model to perform an attention mechanism using the QKV matrix, control the PEFT model to perform inference based on a result of performing the attention mechanism, and then control the QKV / W generation module to delete the weight matrix stored in the cache memory.

[0227] According to one embodiment, the electronic device may obtain the QKV matrix by obtaining personal data of a user stored in a flash memory, generating an input embedding based on the prompt and the personal data, and then applying at least one weight matrix to the input embedding.

[0228] One or more embodiments of the present disclosure may be implemented or supported by one or more computer programs, which may be formed from computer-readable program code and embodied in a computer-readable medium. In the present disclosure, "application" and "program" may refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in computer-readable program code. "Computer-readable program code" may include various types of computer code, including source code, object code, and executable code. "Computer-readable medium" may include various types of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), a hard disk drive (HDD), a compact disc (CD), a digital video disc (DVD), or various types of memory.

[0229] Additionally, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a 'non-transitory storage medium' is a tangible device and may exclude wired, wireless, optical, or other communication links that transmit temporary electrical or other signals. Meanwhile, this 'non-transitory storage medium' does not distinguish between cases where data is permanently stored in the storage medium and cases where it is temporarily stored. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored. A computer-readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and removable and non-removable media. A computer-readable medium includes a medium on which data can be permanently stored and a medium on which data can be stored and later overwritten, such as a rewritable optical disk or an erasable memory device.

[0230] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0231] The above description of the present disclosure is for illustrative purposes only, and those skilled in the art will appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present disclosure. For example, suitable results can be achieved even if the described techniques are performed in a different order than the described method, and / or components of the systems, structures, devices, circuits, etc. described are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form.

[0232] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. In a method for an electronic device to process a user request using the PEFT model, A step in which the electronic device obtains a prompt from a user; A step of determining whether the electronic device requires execution of the PEFT model to process the prompt; If it is determined that execution of the above PEFT model is not necessary, the electronic device executes a foundation model; A step of the electronic device performing user authentication when it is determined that execution of the above PEFT model is necessary; If the user authentication is successful, the electronic device obtains at least one matrix corresponding to the PEFT model; and A method comprising the step of executing the PEFT model using the at least one matrix of the electronic device.

2. In paragraph 1, The step of obtaining at least one matrix comprises: Steps to unlock the QKV / W generation module; The step of the QKV / W generation module generating a QKV / W matrix and storing the QKV / W matrix in the cache memory of the electronic device; and A method characterized in that the QKV / W search module comprises a step of providing the QKV / W matrix to the PEFT model.

3. In either of paragraphs 1 and 2, The step of executing the PEFT model using at least one matrix is ​​as follows: A step in which the PEFT model performs an attention mechanism using the QKV / W matrix; A step in which the PEFT model performs inference based on the result of performing the above attention mechanism; and A method characterized in that the QKV / W generation module includes a step of deleting the QKV / W matrix from the cache memory.

4. In any one of paragraphs 1 to 3, A method characterized in that the above QKV / W generation module is a generative model learned using QKV / W matrices generated in the process of fine-tuning the foundation model into the PEFT model.

5. In any one of paragraphs 1 to 4, The step of obtaining at least one matrix comprises: Step for unlocking the QKV / W generation module; A step of storing a weight matrix for generating a QKV matrix in the cache memory by the QKV / W generation module; and A method characterized in that the QKV / W retrieval module comprises a step of providing the weight matrix to the PEFT model.

6. In any one of paragraphs 1 to 5, The step of storing in the above cache memory is: A step in which the QKV / W generation module requests the weight matrix from the flash memory of the electronic device; The step of the QKV / W generation module obtaining an encrypted weight matrix from the flash memory; and A method characterized in that the QKV / W generation module includes a step of decrypting the encrypted weight matrix and then storing it in the cache memory.

7. In any one of paragraphs 1 to 6, The steps for running the above PEFT model are: A step of obtaining the QKV matrix by applying the at least one weight matrix to an input embedding generated based on the above prompt; A step of controlling the PEFT model to perform an attention mechanism using the QKV matrix; A step of controlling the PEFT model to perform inference based on the result of performing the above attention mechanism; and A method characterized in that the QKV / W generation module includes a step of deleting a weight matrix stored in the cache memory.

8. In any one of paragraphs 1 to 7, The step of obtaining the above QKV matrix is: Step of obtaining the user's personal data stored in flash memory; generating an input embedding based on the above prompt and the personal data; and A method characterized by comprising the step of obtaining the QKV matrix by applying the at least one weight matrix to the input embedding.

9. In the electronic device (1000), a memory (1400) in which a program or at least one instruction is stored; and At least one processor (1300) configured to execute the above program or at least one instruction alone or cooperatively, The electronic device (1000) executes the program or the at least one instruction by the at least one processor (1300) alone or in cooperation. Obtain a prompt from the user, Determine whether execution of the PEFT model (30) is required to process the above prompt, If it is determined that the execution of the above PEFT model (30) is not necessary, the foundation model (10) is executed, If it is determined that the execution of the above PEFT model (30) is necessary, user authentication is performed, If the above user authentication is successful, at least one matrix corresponding to the PEFT model (30) is obtained, An electronic device that executes the PEFT model (30) using at least one matrix.

10. In paragraph 9, The above electronic device (1000) further includes a cache memory (350), The electronic device (1000) obtains the at least one matrix by executing the program or the at least one instruction, either alone or in cooperation with the at least one processor (1300). Unlock the QKV / W generation module (331), After the above QKV / W generation module (331) generates a QKV / W matrix and stores the generated QKV / W matrix in the cache memory (350), An electronic device characterized in that a QKV / W retrieval module (332) provides the QKV / W matrix to the PEFT model (30).

11. In any one of paragraphs 9 and 10, The electronic device (1000) executes the PEFT model (30) by executing the program or the at least one instruction, either alone or in cooperation with the at least one processor (1300). The PEFT model (30) performs the attention mechanism using the above QKV / W matrix, After the PEFT model (30) performs inference based on the result of performing the above attention mechanism, An electronic device characterized in that the QKV / W generation module (331) deletes the QKV / W matrix from the cache memory (350).

12. In any one of paragraphs 9 to 11, An electronic device characterized in that the above QKV / W generation module (331) is a generative model learned using QKV / W matrices generated in the process of fine-tuning the foundation model (10) with the PEFT model (30).

13. In any one of paragraphs 9 to 12, The electronic device (1000) obtains the at least one weight matrix by executing the program or the at least one instruction, either alone or in cooperation with the at least one processor (1300). Unlock the QKV / W generation module (331), After the weight matrix for generating the QKV matrix is ​​stored in the cache memory (350) by the QKV / W generation module (331), An electronic device characterized in that the QKV / W search module (332) provides the weight matrix to the PEFT model (30).

14. In any one of paragraphs 9 to 13, The electronic device (1000) stores the weight matrix in the cache memory (350) by executing the program or the at least one instruction, either alone or in cooperation with the at least one processor (1300). The above QKV / W generation module (331) requests the weight matrix from the flash memory (340), After the above QKV / W generation module (331) obtains the encrypted weight matrix from the flash memory (340), An electronic device characterized in that the QKV / W generation module (331) controls the encrypted weight matrix to be decrypted and then stored in the cache memory (350).

15. In any one of paragraphs 9 to 14, The electronic device (1000) executes the PEFT model (30) by executing the program or the at least one instruction, either alone or in cooperation with the at least one processor (1300). Obtaining the QKV matrix by applying the at least one weight matrix to the input embedding generated based on the above prompt, Using the above QKV matrix, the PEFT model (30) is controlled to perform an attention mechanism, After controlling the PEFT model (30) to perform inference based on the result of performing the above attention mechanism, An electronic device characterized in that the QKV / W generation module (331) controls the deletion of a weight matrix from the cache memory (350).

Images