Method and system for large language model unlearning, and control method for large language model unlearning system

Abstract

The present invention relates to a method and a system for large language model unlearning, and a control method for the large language model unlearning system, which specify and compare forgetting data and retaining data for a pre-trained large language model and then measure the importance of parameters for each of the forgetting data and the retaining data by using a Fisher information matrix, so as to specify specific high-importance parameters, and initialize low-rank adaptation (LoRA) weights on the basis of the specific parameters such that the large language model to which the LoRA weights are applied can be optimized, thereby efficiently removing data to be eliminated, such as sensitive data, without affecting the inference and generation capability of the large language model.

Description

A method and system for large-scale language model unlearning, and a control method for a large-scale language model unlearning system.

[0001] The present invention relates to a large-scale language model unlearning method and system, and a control method for the large-scale language model unlearning system. Furthermore, the present invention relates to a large-scale language model unlearning method and system capable of optimizing large-scale language models, and a control method for the large-scale language model unlearning system.

[0002] The dictionary definition of artificial intelligence is "the technology that embodies human learning, reasoning, perception, and natural language comprehension abilities through computer programs." This type of artificial intelligence has made remarkable progress thanks to deep learning.

[0003] In particular, thanks to the development of artificial intelligence, various language models have been developed, and these language models have reached the level of not only recognizing text and understanding its meaning, but also extracting and classifying information from data containing large amounts of text, such as documents, and even directly generating text.

[0004] These language models are actively used in various fields, such as search engines, document writing (e.g., resume writing, report writing, posting writing, etc.), free conversation on various topics, data parsing from a given text (e.g., data summarization, classification, etc.), providing expert knowledge, programming, and converting a given sentence into sentences in an appropriate style, etc., and they exist in various fields that can perform text-based tasks.

[0005] In this regard, large language models (LLMs) have recently emerged, capable of understanding and generating human language through pre-training on massive amounts of text data. Unlike traditional manually built chatbots, which only offer limited responses, LLMs demonstrate the technological prowess to communicate naturally, almost human-like, and provide fast, accurate information, revolutionizing the artificial intelligence market.

[0006] While these large-scale language models demonstrate powerful reasoning and memory capabilities, they are constantly exposed to privacy and copyright infringement risks as they learn from human-provided text.

[0007] To prevent this, an efficient optimization method is needed that can effectively remove knowledge about sensitive data while improving the overall performance of large-scale language models.

[0008] The present invention provides a large-scale language model unlearning method and system capable of effectively removing data to be removed and increasing efficiency for data to be maintained, and a control method for a large-scale language model unlearning system.

[0009] More specifically, the present invention provides a large-scale language model unlearning method and system capable of removing knowledge of data to be removed while maintaining knowledge of data to be maintained, and a control method of a large-scale language model unlearning system.

[0010] Furthermore, the present invention provides a large-scale language model unlearning method and system capable of efficiently removing data to be removed without affecting the inference and generation capabilities of a large-scale language model, and a control method for a large-scale language model unlearning system.

[0011] In addition, the present invention provides a large-scale language model optimization method and system capable of improving the learning efficiency of a large-scale language model, and a control method of a large-scale language model optimization system.

[0012] In addition, the present invention provides a method and system for optimizing a large-scale language model that can improve the learning and inference speed of a large-scale language model.

[0013] Furthermore, the present invention provides a large-scale language model optimization method and system capable of improving the inference performance of a large-scale language model and achieving cost-effective learning, and a control method of a large-scale language model optimization system.

[0014] In order to solve the problem discussed above, a computerized large-scale language model unlearning method according to the present invention may include the steps of: specifying forget data and maintenance data for a previously trained large-scale language model (LLM) from a training data set stored in a memory; comparing the forget data and the maintenance data to specify a specific parameter having a high importance for the forget data among the parameters of the large-scale language model; initializing a weight of LoRA (Low-Rank Adaptation) based on the specific parameter having a high importance for the forget data; and performing unlearning on the large-scale language model to which the weight of LoRA is applied.

[0015] In an embodiment, the method may further include a step of measuring parameter importance for each of the forgotten data and the maintained data using a Fisher Information Matrix.

[0016] In an embodiment, the Fisher information matrix may be a measure indicating whether at least one parameter of the large-scale language model is important for a text sample included in the forgotten data or the maintained data.

[0017] In an embodiment, a parameter having a high absolute value of a slope for the forgotten data may be specified as having a relatively high importance in the forgotten data, and a parameter having a high absolute value of a slope for the maintained data may be specified as having a relatively high importance in the maintained data.

[0018] In the embodiment, in the measuring step, for each parameter of the large-scale language model, a Fisher information matrix for the forgotten data and a Fisher information matrix for the maintained data are measured using the forgotten data and the maintained data, and the parameter importance can be measured using the Fisher information matrix for the forgotten data and the Fisher information matrix for the maintained data.

[0019] In an embodiment, the parameter importance can be measured using a relative Fisher information matrix between the Fisher information matrix for the forgotten data and the Fisher information matrix for the maintained data.

[0020] In an embodiment, the relative Fisher information matrix can be calculated using a Fisher information matrix for the forgotten data and a Fisher information matrix for the maintained data.

[0021] In an embodiment, based on the relative Fisher information matrix, a parameter having high parameter importance for the forgotten data can be specified as the specific parameter.

[0022] In an embodiment, the specific parameter may be specified as a parameter having high importance for the forgotten data and low importance for the maintained data.

[0023] In an embodiment, the initialization may be performed by calculating the relative importance of parameters for each of the forgotten data and the maintained data.

[0024] In the embodiment, in the step of performing the initialization, the weight of the LoRA can be initialized with a focus on the specific parameter with high importance in the forgotten data.

[0025] In an embodiment, among the forgotten data and the maintained data, important information about the forgotten data can be specified and reflected in the initialization.

[0026] In the embodiment, in the step of performing the unlearning, unlearning can be performed on the forgotten data using a loss function set for unlearning in the large-scale language model.

[0027] In an embodiment, in the step of performing the unlearning, unlearning may be performed on the large-scale language model using the loss function so that the prediction probability for the maintenance data of the large-scale language model increases while the prediction probability for the forgotten data decreases.

[0028] In the embodiment, in the step of performing the unlearning, the prediction probability of the large-scale language model for a true token included in the forgotten data may be reduced so as to reduce the prediction probability for the forgotten data, and the prediction probability of the large-scale language model for a replacement token having the highest probability among all tokens excluding the true token may be increased.

[0029] In an embodiment, the replacement token may correspond to any one token that is most likely to replace the correct token among all tokens except the correct token.

[0030] In an embodiment, while unlearning is performed on the large-scale language model, the parameters of the large-scale language model are fixed, and only the weights of the LoRA can be updated.

[0031] A control method of a large-scale language model unlearning system according to the present invention may include the steps of: receiving a user input requesting deletion of specific data from a training data set used for training a large-scale language model (LLM); specifying data corresponding to the specific data in the training data set as forgotten data based on the received user input, and specifying the remaining data excluding the forgotten data as maintained data; comparing the forgotten data and the maintained data to specify a specific parameter having a high importance for the forgotten data among the parameters of the large-scale language model; initializing a weight of LoRA (Low-Rank Adaptation) based on the specific parameter having a high importance for the forgotten data; and performing unlearning on the large-scale language model to which the LoRA weight is applied.

[0032] A large-scale language model unlearning system according to the present invention, comprising a memory configured to store executable instructions, and one or more processors configured to perform operations by executing one or more instructions, can specify forget data and maintenance data for a previously trained large-scale language model (LLM) from a training data set stored in the memory, compare the forget data and the maintenance data to specify a specific parameter having a high importance for the forget data among parameters of the large-scale language model, initialize a weight of LoRA (Low-Rank Adaptation) based on the specific parameter having a high importance for the forget data, and perform unlearning on the large-scale language model to which the weight of LoRA is applied.

[0033] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, and may include instructions for performing a step of specifying forget data and maintenance data for a large-scale language model (LLM) that has been trained in a training data set, a step of comparing the forget data and the maintenance data to specify a specific parameter having a high importance for the forget data among the parameters of the large-scale language model, a step of initializing a weight of LoRA (Low-Rank Adaptation) based on the specific parameter having a high importance for the forget data, and a step of performing unlearning on the large-scale language model to which the weight of LoRA is applied.

[0034] A computerized method for optimizing a large-scale language model according to the present invention, comprising the steps of: specifying forget data and maintenance data for a large-scale language model trained with a training data set; specifying, using the forget data and the maintenance data, a specific parameter having a high importance for the forget data among parameters of the trained large-scale language model; initializing a weight of a preset adapter based on the importance of the specific parameter; and performing unlearning on the trained large-scale language model to which the weight of the initialized adapter is applied.

[0035] In an embodiment, the learned large-scale language model may be a model for which learning is performed on the learning data set based on a preset attention mechanism.

[0036] In an embodiment, the learned large-scale language model can be learned based on the attention mechanism for long-context modeling.

[0037] In an embodiment, when at least one text sequence included in the training data set is input to the large-scale language model, the trained large-scale language model can perform an attention operation only on some query-key pairs selected according to preset criteria among all query-key pairs included in the input text sequence.

[0038] In an embodiment, the learned large-scale language model may be trained to process tokens included in the text sequence in sliding-window attention units, blockwise selection and processing, or processing based on importance during the process of processing the text sequence through the attention mechanism.

[0039] In an embodiment, the learned large-scale language model may be learned using at least one of a low-precision training technique and a mixed precision training technique.

[0040] In an embodiment, the learned large-scale language model may correspond to a teacher model configured to distill knowledge learned through learning on the learning data set into at least one model corresponding to a student model.

[0041] In an embodiment, the method further includes a step of measuring parameter importance for each of the forgotten data and the maintained data using an empirical Fisher Information Matrix, wherein the empirical Fisher Information Matrix may be a measure indicating whether at least one parameter of the learned large-scale language model is important for a text sample included in the forgotten data or the maintained data.

[0042] In the embodiment, in the measuring step, for each parameter of the learned large-scale language model, an empirical Fisher information matrix for the forgotten data and an empirical Fisher information matrix for the maintained data are measured using the forgotten data and the maintained data, and the parameter importance can be measured using the empirical Fisher information matrix for the forgotten data and the maintained data.

[0043] In an embodiment, the parameter importance can be measured using a relative Fisher information matrix between an empirical Fisher information matrix for the forgotten data and an empirical Fisher information matrix for the maintained data.

[0044] In an embodiment, the relative Fisher information matrix can be calculated using a Fisher information matrix for the forgotten data and a Fisher information matrix for the maintained data.

[0045] In the embodiment, in the step of specifying the specific parameter, a parameter having high parameter importance for the forgotten data can be specified as the specific parameter.

[0046] In an embodiment, the specific parameter may be specified as a parameter having high importance for the forgotten data and low importance for the maintained data.

[0047] In an embodiment, the initialization may be performed by calculating the relative importance of parameters for each of the forgotten data and the maintained data.

[0048] In the embodiment, in the step of performing the initialization, the weight of the adapter can be initialized with a focus on the specific parameter having high importance in the forgotten data.

[0049] In the embodiment, in the step of performing the unlearning, unlearning can be performed on the forgotten data using a loss function set for unlearning in the learned large-scale language model.

[0050] In the embodiment, in the step of performing the unlearning, unlearning can be performed on the learned large-scale language model using the loss function so that the prediction probability for the maintenance data of the learned large-scale language model increases while the prediction probability for the forgotten data decreases.

[0051] In an embodiment, while unlearning is performed on the large-scale language model, the parameters of the large-scale language model are fixed, and only the weights of the LoRA adapter can be updated.

[0052] A large-scale language model optimization system according to the present invention, comprising a memory configured to store executable instructions, and one or more processors configured to perform operations by executing one or more instructions, specifies forget data and maintenance data for a large-scale language model trained with a training data set, and, using the forget data and the maintenance data, specifies a specific parameter having a high importance for the forget data among parameters of the trained large-scale language model, and initializes a weight of a preset adapter based on the importance of the specific parameter, and performs unlearning on the trained large-scale language model to which the weight of the initialized adapter is applied.

[0053] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, and may include instructions for performing the steps of: specifying forget data and maintenance data for a large-scale language model learned using a training data set; using the forget data and the maintenance data, specifying a specific parameter having a high importance for the forget data among the parameters of the learned large-scale language model; initializing a weight of a preset adapter based on the importance of the specific parameter; and performing unlearning on the learned large-scale language model to which the weight of the initialized adapter is applied.

[0054] As described above, the large-scale language model unlearning method and system according to the present invention, and the control method for the large-scale language model unlearning system, can measure Fisher information for each of the forgotten data and the maintained data, and perform unlearning of a large-scale language model based on the measured results. Therefore, the present invention can effectively remove knowledge about the forgotten data while maintaining knowledge about the maintained data by selecting and preferentially adjusting only the parameters (or weights) that are relatively important for the forgotten data. Through this, the present invention can provide the effect of maintaining the existing performance of the model by simultaneously unlearning the forgotten data more quickly and minimizing the impact on the maintained data.

[0055] Furthermore, the large-scale language model unlearning method and system according to the present invention, and the control method for the large-scale language model unlearning system, can perform an initialization process that analyzes the relative importance of forgotten data and retained data, and selectively adjusts parameters important only to forgotten data based on the analyzed results. Through this, the present invention can reduce unnecessary operations during the unlearning process, lower computational costs, and efficiently perform unlearning in terms of time and resources without retraining the entire model.

[0056] Furthermore, according to the large-scale language model unlearning method and system according to the present invention, and the control method of the large-scale language model unlearning system, by focusing the gradient update only on the minimum viable replacements for the correct token, the data to be removed can be effectively removed while maintaining the language generation ability and inference performance of the existing model. Through this, the present invention provides an environment in which performance degradation that may occur during the information deletion process of the model can be minimized and privacy protection and copyright issues can be resolved. In other words, the present invention prevents unnecessary loss diffusion and performs effective unlearning by adjusting only the minimum viable replacement tokens, thereby maintaining the model's natural sentence generation ability even in situations where unlearning must be performed on specific data.

[0057] In this way, the present invention prioritizes adjusting important parameters in forgotten data and minimizes information loss in retained data. This allows for the removal of specific data without retraining the entire model, significantly reducing computational and operational costs. Furthermore, the present invention enables the rapid unlearning of data desired by the user, thereby contributing to data protection law compliance, reduced corporate operating costs, and enhanced AI system reliability. It can be utilized effectively in a variety of industries and services.

[0058] Figure 1 is a conceptual diagram explaining unlearning of a large-scale language model.

[0059] Figure 2 is a conceptual diagram illustrating a large-scale language model unlearning system according to the present invention.

[0060] FIGS. 3 and 4 are flowcharts for explaining a large-scale language model unlearning method according to the present invention.

[0061] Figures 5 and 6 are conceptual diagrams for explaining a large-scale language model unlearning method according to the present invention.

[0062] Figures 7, 8, 9, 10 and 11 are formulas related to a large-scale language model unlearning method according to the present invention.

[0063] Figure 12 is a flowchart for explaining a control method of a large-scale language model unlearning system according to the present invention.

[0064] Figure 13 is a conceptual diagram for explaining a control method of a large-scale language model unlearning system according to the present invention.

[0065] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0066] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0067] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0068] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0069] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0070] The present invention relates to a large-scale language model unlearning method and system, and a control method for the large-scale language model unlearning system. The present invention relates to a large-scale language model unlearning method and system capable of effectively removing desired data and increasing the efficiency of retaining desired data, and a control method for the large-scale language model unlearning system.

[0071] Large language models (LLMs) possess powerful inference and memory capabilities through pre-training on massive amounts of text data. However, LLMs are constantly exposed to privacy and copyright infringement risks as they learn from human-provided text.

[0072] To prevent this, unlearning is necessary, which removes sensitive data (i.e., data to be removed) that poses risks of privacy and copyright infringement. Unlearning can refer to the process of intentionally removing (or deleting) or modifying information, patterns (or knowledge) that a model has previously learned (or previously learned). For example, unlearning can be a method for removing or modifying incorrect information, inappropriate biases, or unintended data that a model has learned.

[0073] In this regard, referring to Figure 1, the purpose of large-scale language model unlearning is to fine-tune a pre-trained large-scale language model to remove (or delete) knowledge about a data set to be removed (e.g., “Forget set”). For example, the data set to be removed (or “Forget set,” or “Delete set”) may include text data that has been requested to be deleted by a user. During the unlearning process, the large-scale language model must forget the knowledge about the data included in the data set to be removed, and retain the knowledge about the data included in the data set to be retained (e.g., “Retain set”), and must also maintain the inference and generation capabilities acquired in advance.

[0074] Accordingly, the present invention aims to provide a large-scale language model unlearning method and system capable of efficiently removing data to be removed and maintaining knowledge about data to be retained without affecting the inference and generation capabilities of a large-scale language model.

[0075] The present invention can be usefully utilized in various situations. More specifically, the large-scale language model unlearning method and system according to the present invention can be applied and utilized effectively in various industries and services. For example, the large-scale language model unlearning method and system according to the present invention can be applied and utilized effectively in systems (or applications, software, websites, programs, etc.) based on large-scale language models.

[0076] In this way, the present invention can be usefully utilized in various industries and services that require unlearning of large-scale language models (e.g., natural language generation-related services, conversational AI and chatbots, text generation AI and content creation, customized education and language learning, social media and online platforms, harmful content filtering, medical and healthcare, finance and law, games and virtual environments, etc.).

[0077] Meanwhile, as illustrated in FIG. 2, a large-scale language model unlearning system according to the present invention (hereinafter referred to as “unlearning system”, 100) may include at least one of an input unit (110), an output unit (120), a storage unit (130), a control unit (140), and a large-scale language model (150).

[0078] The unlearning system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may function as the storage unit (130) described above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.

[0079] Although not shown, the unlearning system (100) according to the present invention may include one or more processors, which may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., a digital signal processor, a tensor processing unit (TPU), a graphics processing unit (GPU), a neural network processing unit (NPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a quantum processing unit (or quantum processor, QPU), etc.). The one or more processors may be configured to execute instructions stored (or included) in the storage unit (130), computer-readable instructions, and / or other instructions described herein. The unlearning method and system according to the present invention may allow the memory and at least one processor to cooperate to perform data processing as described below. The processor may perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage unit (130).

[0080] Furthermore, the language unlearning system (100) according to the present invention can perform data processing and calculation processes utilizing quantum gates, quantum entanglement, and quantum superposition states, taking into account implementation in a quantum computer environment. For example, the present invention can perform parallel operations based on qubits, and such quantum operations can operate complementarily with existing classical computers.

[0081] These quantum computers may include qubit-based parallel computing and high-speed data processing devices leveraging quantum entanglement, enabling hardware-based computational optimization using FPGAs and ASICs. Furthermore, quantum computers can utilize quantum processors capable of qubit-based parallel computing, and data processing efficiency can be improved through hybrid architectures with existing classical computers.

[0082] Meanwhile, the input unit (110) may be configured as a means for data input and may be configured in various types. For example, the input unit (110) may be configured to receive user input. The input unit (110) may be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on the input being made by the user through the input unit configuration provided in the user terminal (10).

[0083] The user terminal (10) may include at least one of a mobile phone, a smart phone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head mounted display (HMD)).

[0084] In addition, in the present invention, the input unit (110) does not necessarily mean a hardware means, and can be understood as a passage for receiving input from a user.

[0085] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, computer mouse, keyboard, keypad, touch pad, trackball, joystick, voice recognition module, or other similar device. However, the present invention does not limit the type of input unit (110).

[0086] Here, the user input may include documents, text, images (or videos), voice, etc. In this case, the unlearning system (100) may further include a module that converts voice into text.

[0087] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked with the unlearning system (100) according to the present invention. For example, the output unit (120) can output a page (or service page) linked with the unlearning system (100) according to the present invention to the display unit of the user terminal (10). In addition, the output unit (120) does not necessarily mean a hardware means, and can be understood as a channel for outputting results to the user.

[0088] Next, the storage unit (130, or memory) serves to store various data related to the present invention, and may include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of the one or more processors.

[0089] The one or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage (130) may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage device), while in other examples, the storage (130) may be implemented using two or more physical devices.

[0090] The storage unit (130) may include computer-readable instructions and additional data. The storage unit (130) may include storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the devices and networks.

[0091] Furthermore, at least a portion of the storage unit (130) may be a cloud storage or a cloud server. The storage unit (130) may store at least a portion of data corresponding to user input received from the input unit (110) and learning data (or a learning data set).

[0092] That is, it can be understood that the storage unit (130) is sufficient as a space in which information necessary for the operation of the unlearning system (100) according to the present invention is stored, and there are no restrictions on the physical space.

[0093] Furthermore, the storage unit (130) may store a computer program including computer program instructions. Furthermore, the storage unit (130) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (140) when loaded onto the processor of the system (100).

[0094] Next, the control unit (140) may perform a role of controlling the overall operation of the unlearning system (100) related to the present invention. The control unit (140) may process signals, data, information, etc. input or output through the components of the unlearning system (100) discussed above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (140) may be physically implemented by the processor discussed above.

[0095] Meanwhile, the large-scale language model (150) may be a pre-trained (or pre-trained) model using a training data set. The large-scale language model (150) may perform pre-training on large-scale text data (or text corpus, text data, text samples, text sequences, token sequences, language data, etc.) included (or configured) in the training data set. Such a large-scale language model (150) may model the likelihood of a sequence by predicting the next token when a token sequence (or text sequence) of a certain length (T) is given.

[0096] In one embodiment, after pre-training of a large-scale language model (150) is completed, assume that the user requests that specific data (or a specific data set) be deleted from the training data set used for training the large-scale language model (150). In the present invention, the specific data that the user wishes to remove (unlearn) may be referred to as “forgotten data” or a “forgotten data set.” In addition, in the present invention, even after unlearning of the forgotten data (210) is completed, data containing knowledge that the large-scale language model (150) should not forget may be referred to as “maintained data” or a “maintained data set.”

[0097] In this case, in the unlearning process, the large-scale language model (150) can unlearn the text sequence by maximizing the next token prediction loss of at least one text sequence included in the forgotten data (210) to assign a low probability to the forgotten data (210).

[0098] Here, maximizing the prediction loss can mean maximizing the next token prediction loss through gradient ascent, which is the opposite of gradient descent. Unlike gradient descent (or gradient descent, gradient descent, etc.), which increases the prediction probability (or prediction score, generation probability, generation score, etc.) of the model's correct answer by minimizing the loss function, gradient ascent (or gradient ascent, gradient ascent) decreases the prediction probability of the correct answer by maximizing the loss function. In this regard, the log-likelihood (or log-likelihood) can be implemented as a cross-entropy loss (or cross-entropy loss) for incremental tokens. This gradient ascent can be a model that adjusts to maximize the cross-entropy loss.

[0099] In this way, the gradient ascent method discussed above trains the model to decrease the probability of the correct token and increase the probability of the remaining tokens. In other words, the model trains to increase the probability of all non-correct tokens.

[0100] Therefore, maximizing the prediction loss may mean adjusting the probability distribution of the large-scale language model (150) to induce the large-scale language model (150) not to learn or not to predict certain text sequences. For example, the prediction loss for certain tokens may be maximized so that the prediction probability for certain tokens is reduced so that the large-scale language model (150) predicts certain tokens with a lower probability than those that it had predicted with a high probability, thereby adjusting the probability distribution so that the large-scale language model (150) makes inaccurate predictions for certain tokens (i.e., suppresses generation of forgotten data, suppresses generation of inappropriate sentences, etc.).

[0101] However, in the present invention, in order to prevent i) the loss from not converging but increasing or diverging without a finite boundary, ii) the logit for all other tokens from increasing, resulting in unnecessary additional forgetting, or iii) unstable results occurring during the optimization process, the probability distribution of the model is adjusted in a direction that increases the prediction probability (e.g., loss function) of the token (alternative token) with the highest probability among all tokens excluding the correct token while decreasing the prediction probability of the correct token, without increasing the prediction probability of all tokens excluding the correct token. That is, the present invention uses a method of adjusting the model by focusing the gradient update only on the minimum alternative token with a high possibility of replacing the correct token. More specific details about this will be described later.

[0102] Meanwhile, the present invention provides a method and system for large-scale language model unlearning that can effectively remove knowledge about data to be removed while maintaining knowledge about data to be retained, as well as a control method for a large-scale language model unlearning system. More specifically, the present invention provides a method for large-scale language model unlearning that can efficiently remove data to be removed without affecting the inference and generation capabilities of the large-scale language model. Hereinafter, the large-scale language model unlearning method according to the present invention will be examined in more detail.

[0103] First, in the present invention, a process of specifying forgetting data and maintenance data for a previously trained large-scale language model (LLM) from a training data set stored in memory can be performed (S310, see FIG. 3).

[0104] The control unit (140) can specify, among the pre-learned data for a large-scale language model (150), forget data (e.g., “forget set”) corresponding to data for which the learned result is to be removed, and retain data (e.g., “retain set”) corresponding to data for which the learned result is to be maintained.

[0105] As illustrated in FIG. 4, the control unit (140) can specify, in the learning data set (200) used in the learning process of the large-scale language model (150), learning data to be unlearned so that recognition is impossible through the large-scale language model (150) as forgetting data (210), and can specify learning data to be maintained in a state where recognition is possible through the large-scale language model (150) as maintenance data (220).

[0106] For example, the control unit (140) can specify text samples (or text data, text sequences, token sequences, language data, etc.) that are to be removed (i.e., targets for removal) to make them unrecognizable through the large-scale language model (150) from the learning data set (200) used for learning the large-scale language model (150) as forgotten data (210).

[0107] As another example, the control unit (140) can specify, as maintenance data (220), text samples that are to be maintained in a state that is recognizable through the large-scale language model (150) in the training data set (200) used for training the large-scale language model (150).

[0108] In this regard, various methods for specifying forgotten data and retained data in the present invention may exist. In the present invention, forgetting data and retained data may be specified based on user input (or request) or by the unlearning system (100) itself.

[0109] In one embodiment, after pre-training of a large-scale language model (150) is completed, if a user requests deletion of specific data from a training data set (200) used for training the large-scale language model (150), the control unit (140) may specify the specific data requested by the user (or received from the user terminal (10)) as forgotten data (210) and specify the remaining data excluding the specified forgotten data (210) as maintained data (220).

[0110] In another embodiment, the unlearning system (100) may analyze the training data set (200) used in the training process of the large-scale language model (150) based on preset criteria (or conditions). Here, the preset criteria may be criteria set in relation to the user's personal information (e.g., name, address, phone number, email, etc.) or copyright infringement elements. As a result of the analysis, if data related to the preset criteria is detected (or filtered) in the training data set (200) used in the training of the large-scale language model (150), the unlearning system (100) may specify the detected data as forgotten data (210) and specify the remaining data excluding the specified forgotten data (210) as maintained data (220).

[0111] In another embodiment, when the unlearning system (100) performs fine tuning (or fine tuning) on ​​a large-scale language model (150) trained based on large-scale text data so that only text data corresponding to a specific item (or type) can be recognized, the learning data related to the data corresponding to the specific item can be specified as maintenance data (220), and the learning data related to the data corresponding to an item different from the specific item can be specified as forget data (210).

[0112] At this time, the unlearning system (100) may be understood as performing unlearning on data corresponding to the above-mentioned other items, and here, the specific item or other items may be understood as meaning a category (or type) for data recognizable through a large-scale language model (150).

[0113] Additionally, the maintenance data (220) may include learning data related to items to be newly learned in addition to data previously learned for the large-scale language model (150).

[0114] However, in the present invention, the method (or method) for specifying the forgetting data (210) and the maintenance data (220) is not necessarily limited to the cases described above, and may be specified in various ways.

[0115] Furthermore, in the present invention, the learning data set (200) can be represented as in (a) of FIG. 7, the forgetting data (210) can be represented as in (b) of FIG. 7, and the maintenance data (220) can be represented as in (c) of FIG. 7.

[0116] Next, in the present invention, a process may be performed to compare forgotten data and maintained data to identify specific parameters with high importance for forgotten data among the parameters of a large-scale language model (S320, see FIG. 3).

[0117] Here, comparing the forgotten data and the maintained data can also be understood as comparing and analyzing the relative importance of parameters for the forgotten data (210) and the maintained data (220).

[0118] Parameters can refer to learnable values, including the model's weights and biases. Parameters are adjustable values ​​that a model uses to learn data and make predictions (or inferences). For example, in an artificial neural network, the weights of each layer can be considered parameters. These parameters are optimized through the learning process, allowing the model to learn patterns from data and make predictions.

[0119] In this regard, parameter (or parameter, weight, etc.) changes due to adaptation of a large-scale language model (150) may inherently have a low-rank (or low-dimensional, low-rank, etc.) structure. More specifically, parameter changes of a large-scale language model (150) due to adaptation of the large-scale language model (150) may be approximated as low-rank matrices based on the assumption that they have a low rank.

[0120] Here, adaptation of a large-scale language model is a process of changing or adjusting a pre-trained model to suit a specific purpose (or task), and may include, for example, training, fine-tuning, and unlearning.

[0121] In addition, the fact that the parameter changes of the large-scale language model (150) have a low rank (or low-rank structure) may mean that when the parameters (e.g., the weight matrix) of the large-scale language model (150) change, the change occurs in a subspace with a relatively low dimensionality in the entire parameter space. This may mean that not all weights change independently, but rather the change occurs along a specific low-rank structure. In other words, it may mean that the main change occurs in a specific low-rank (e.g., low-rank) part rather than the entire weight matrix of the large-scale language model (150). For example, when the model learns data from a specific domain, it may also be understood that not all neurons are updated equally, but only some neurons play a major role.

[0122] As discussed above, assuming that the parameter change due to the adaptation of the large-scale language model (150) is low-rank, LoRA (Low-Rank Adaptation) models the parameter change of each linear weight (e.g., linear layer (or linear layer, Linear Layer) weight (or weight matrix) of the large-scale language model (150)) of the large-scale language model (150) as the product of multiple (two) low-rank matrices. Here, each linear weight can be represented as in (d) of FIG. 7, and the parameter change can be represented as in (e) of FIG. 7. In addition, multiple (two) low-rank matrices (A and B) can be represented as in (f) of FIG. 7, and the rank of the LoLA adapter can be represented as in (g) of FIG. That is, when input is given to a large-scale language model (150), the output of the adapted linear layer can be represented as in (h) of Fig. 7.

[0123] In this case, while fine-tuning the large-scale language model (150) is performed, the existing weights of the pre-trained large-scale language model (150) are fixed, and only the low-rank matrices (A and B) can be updated through gradient descent. To ensure that the initial attachment of the LoRA adapter does not change the output of the large-scale language model (150), LoRA defaults to initializing the first low-rank matrix (A) with a Kaiming-uniform distribution and setting the second low-rank matrix (B) to a zero matrix. After the fine-tuning of the large-scale language model (150) is completed, the LoRA adapter can be merged with the existing weights (see (i) of FIG. 7).

[0124] That is, LoRA can mean a method (or technique, method, etc.) that updates a model through low-rank changes instead of adjusting the entire weight by modeling the change in a weight matrix as a product of two low-rank matrices. With reference to this, the present invention can perform unlearning on a large-scale language model (150) in a way that reduces the amount of computation and increases efficiency based on the assumption that parameter changes of the large-scale language model (150) mainly occur in the low-rank region, without retraining the large-scale language model (150).

[0125] Meanwhile, the present invention can perform weighted low-rank decomposition by measuring Fisher information for each parameter (or weight) of a large-scale language model (150) and initializing adapter weights (A and B) using the measured results. In the present invention, this process can also be referred to as “Fisher-weighted LoRA Initialization (FLoRA).”

[0126] The FLoRA process according to the present invention aims to enable a large-scale language model (150) to quickly unlearn the forgotten data (210) and minimize performance degradation for the maintained data (220) by setting parameters that are more important for the forgotten data (210) to be adjusted first. To this end, the present invention quantifies the parameter importance for each of the forgotten data (210) and the maintained data (220) using a Fisher information matrix, and initializes the large-scale language model (150) based on this.

[0127] The control unit (140) can measure the parameter importance for each of the forgotten data (210) and the maintained data (220) using the Fisher Information Matrix. The control unit (140) can measure how important each weight (or specific weight) of the large-scale language model (150) is for each of the forgotten data (210) and the maintained data (220) using the Fisher Information Matrix.

[0128] Here, the Fisher information matrix may refer to the amount of information that the learning data set (200) provides to the parameters of the large-scale language model (150). The Fisher information matrix may be represented as in (a) of Fig. 8.

[0129] This Fisher information matrix can be computed as the second central moment of the first partial derivative of the log-likelihood (see the left side of (c) of FIG. 8). However, since it is computationally infeasible to integrate over the space of the training data set (200), the present invention can utilize the empirical Fisher information (or empirical Fisher information matrix). The empirical Fisher information can be represented as in (b) of FIG. 8. For large-scale language models, the empirical Fisher information can be computed as the root mean square of the propagated gradients for language modeling purposes (e.g., cross-entropy loss) (see (c) of FIG. 8). However, in this specification, the terms “Fisher information matrix (or Fisher information)” and “empirical Fisher information matrix (or empirical Fisher information)” may be used interchangeably.

[0130] A Fisher information matrix may refer to a value indicating how important a specific parameter of a model is for a given data. More specifically, a Fisher information matrix may refer to a measure (or value) indicating whether at least one parameter (or target parameter, specific parameter, target parameter, etc.) of a large-scale language model is important for a text sample (e.g., a sentence, document, token sequence, paragraph, etc.) included in the forgotten data (210) or the maintained data (220). For example, a Fisher information matrix may be a measure indicating whether a specific parameter of a large-scale language model (150) is important for a specific data, and the measure indicating the importance may be expressed as a slope.

[0131] In this case, parameters with a large absolute slope can be identified (or determined, judged, considered, etc.) as being important in the data. In other words, parameters with a large slope for specific data can be identified as playing a significant role in generating the data.

[0132] Accordingly, a parameter having a high absolute value of the slope for the forgotten data (210) can be specified as having a relatively high importance in the forgotten data (210), and a parameter having a high absolute value of the slope for the maintained data (220) can be specified as having a relatively high importance in the maintained data.

[0133] In this regard, the control unit (140) can measure the Fisher information matrix for each of the forget data (210) and the maintenance data (220).

[0134] Specifically, the control unit (140) can measure a Fisher information matrix for the forgotten data (210) and a Fisher information matrix for the maintained data (220) using the forgotten data (210) and the maintained data (220) for each parameter of the large-scale language model (150). The control unit (140) can obtain a Fisher information matrix for the forgotten data (210) measured using the forgotten data (210) and a Fisher information matrix for the maintained data (220) measured using the maintained data (220) for each parameter of the large-scale language model (150). Here, the Fisher information matrix measured for the forgotten data (210) can be represented as in (d) of FIG. 8, and the Fisher information matrix measured for the maintained data (220) can be represented as in (e) of FIG. 8.

[0135] In another embodiment, the control unit (140) may set (or select) at least one target parameter among the parameters of the large-scale language model, and measure a Fisher information matrix for the forgotten data (210) and a Fisher information matrix for the maintained data (220) using the forgotten data (210) and the maintained data (220) for the target parameter. The control unit (140) may obtain a Fisher information matrix for the forgotten data (210) measured using the forgotten data (210) and a Fisher information matrix for the maintained data (220) measured using the maintained data (220) for the target parameter. In this case, the target parameter may be randomly set (or selected) or may be set based on a preset criterion (e.g., a parameter with a high learning weight, a parameter with a high probability distribution of generating forgotten data, etc.).

[0136] In addition, the control unit (140) can measure (or analyze, quantify) the parameter importance for each of the forgotten data (210) and the maintained data (220) by using the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220) (S401, see FIG. 4).

[0137] Here, parameter importance can be measured using the relative Fisher information matrix between the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220). The control unit (140) can use the relative Fisher information matrix between the forgotten data (210) and the maintained data (220) as an importance index to preferentially specify (or select, identify, etc.) parameters that have high importance for the forgotten data (210) but low importance for the maintained data (220). The relative Fisher information matrix can be represented as shown in (f) of FIG. 8.

[0138] This relative Fisher information matrix can be calculated using the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220). In this case, calculating the relative Fisher information matrix can also be understood as calculating the relative importance of parameters for the forgotten data (210) and the maintained data (220).

[0139] The control unit (140) can calculate a relative Fisher information matrix between the Fisher information matrices measured for each of the forgotten data (210) and the maintained data (220), and measure (or analyze, quantify) the parameter importance for each parameter of the large-scale language model (150) based on the calculated result.

[0140] As a result of the parameter importance measurement, the control unit (140) can specify (or determine, select, etc.) at least one parameter among the parameters of the large-scale language model (150) that has a high Fisher information matrix measured for the forgotten data (210) as a specific parameter. The control unit (140) can specify a parameter with high parameter importance for the forgotten data (210) as a specific parameter (S402, see FIG. 4). In this case, the specific parameter can be understood as a parameter with a high absolute value of the slope for the forgotten data (210) and thus has relatively high importance in the forgotten data (210).

[0141] That is, in the unlearning process according to the present invention, a high Fisher information for the forgotten data (210) may indicate that the next token prediction loss in the forgotten data (210) induces a large absolute slope in the corresponding parameter. Therefore, in the present invention, such a parameter may be specified as a specific parameter important for generating a sequence of the forgotten data (210). In this specification, the specific parameter may also be referred to as a "specific weight," "important parameter," or "important weight."

[0142] In another embodiment, the control unit (140) may calculate a relative Fisher information matrix between the Fisher information matrix for the measured forgotten data (210) and the Fisher information matrix for the maintained data (220) for the target parameter, and based on the calculated result, measure (or analyze) which data the target parameter has a higher importance for. As a result of measuring the target parameter importance, if the target parameter has a high importance for the forgotten data (210), the control unit (140) may specify the target parameter as a specific parameter. Alternatively, as a result of measuring the target parameter importance, if the target parameter has a high importance for the maintained data (220), the control unit (140) may exclude the target parameter from the initialization target and perform the above process again to specify a parameter having a high importance for the forgotten data (210).

[0143] Accordingly, the control unit (140) may specify, as a specific parameter, a parameter among the parameters of the large-scale language model (150) in which the Fisher information for the forgotten data (210) is relatively higher than the Fisher information for the maintained data (220). For example, the control unit (140) may preferentially specify a specific parameter in which the Fisher information for the forgotten data (210) is high but the Fisher information for the maintained data (220) is low.

[0144] That is, the control unit (140) can compare the forgotten data (Fisher information for the forgotten data) and the maintained data (e.g., Fisher information for the maintained data) to specify a parameter that has a high importance (or importance) for the forgotten data (210) and a low importance (or importance) for the maintained data (220), as a specific parameter that should be set to be preferentially adjusted during the unlearning process.

[0145] Meanwhile, in the present invention, a process of initializing the weight of LoRA (Low-Rank Adaptation) based on a specific parameter with high importance for forgotten data can be performed (S330, see FIG. 3).

[0146] In the present invention, the initialization process may be a process of calculating the relative importance of parameters for forgotten data (210) and maintained data (220) and performing LoRA initialization based on the calculated results. Furthermore, the initialization process may be a process of initializing LoRA weights (A, B) such that parameters important to forgotten data (210) are greater.

[0147] Additionally, the initialization process may be a process in which the weights (A, B) of the LoRA adapter are initialized based on (or as a standard) a specific parameter with high importance for the forgotten data (210), so that the unlearning process is centered on the specific parameter, thereby allowing the weights to focus on removing knowledge of the forgotten data (210).

[0148] Additionally, the initialization process may be a process of applying LoRA to specific parameters with high importance for the forgotten data (210) (e.g., decomposing the specific parameters into LoRA adapter weights (A, B)) and initializing the LoRA weights (A, B) by reflecting information about the specific parameters (Fisher information, relative Fisher information, etc.). For example, the existing weights (W) of the pre-trained large-scale language model (150) are fixed so as not to change the output of the model when LoRA is applied.

[0149] Additionally, the initialization process may be a process of initializing the weight of LoRA (or the weight of the LoRA adapter) based on the importance (or importance information) of a specific parameter with high importance for the forgotten data (210), thereby effectively removing knowledge about the forgotten data (210).

[0150] In addition, the initialization process may be a process of initializing the weights (low-rank matrix) of LoRA so that the importance of specific parameters with high importance for the forgotten data (210) is reflected. In addition, the initialization process may be a process of initializing the weights of LoRA based on the relative importance of specific parameters with high importance for the forgotten data (210), thereby inducing unlearning to be performed quickly and efficiently centered on specific parameters through adjustment of the corresponding parameters.

[0151] Additionally, the initialization process may be a process of initializing the LoRA adapter (i.e., the low-rank matrix (A, B) of LoRA) in a direction that reflects the importance of specific parameters with high importance for the forgotten data (210), thereby enabling unlearning to be performed effectively (at this time, the weights of the pre-learned large-scale language model are fixed).

[0152] That is, the control unit (140) can initialize the weight of LoRA centered on specific parameters with high importance in the forget data (210) (S403, see FIG. 4).

[0153] In one embodiment, initializing the LoRA weights with parameters important for generating the forgotten data (210) can be advantageous during the unlearning process, as the gradient modifies only the parameters important for the forgotten data (210) and leaves the remaining parameters unchanged. Given the relative importance of each parameter of the large-scale language model (150), the solution to the Weighted Low-Rank Approximation (WLRA) problem can be expressed as the initialization of the LoRA adapter weights. This can be represented as shown in (a) of FIG. 9 .

[0154] At this time, the present invention assumes that the parameters of each row of the large-scale language model (150) have equal importance, and the weighted low-rank approximation problem can be redefined using the square root of the row-wise sum of the relative Fisher information matrix. This can be represented as in (b) of Fig. 9.

[0155] Here, a vector with all elements equal to 1 can be represented as in (c) of Fig. 9, and a function that converts a vector into a diagonal matrix and a matrix-vector product can be represented as in (d) of Fig. 9. In this way, the row-wise weighted low-rank approximation (or weighted low-dimensional approximation based on Fisher information) problem has a closed-form solution, which can be derived by applying the singular value decomposition (SVD) of the rank (r), as shown in (e) of Fig. 9. At this time, the optimal low-rank matrices (A and B) in the present invention can be calculated as shown in (f) of Fig. 9.

[0156] Here, the solution may include optimal LoRA weights obtained by low-rank approximation of existing model weights (W) containing important information in the forgotten data (210). For example, the solution may be a low-rank approximation of a weight matrix that has high importance for the forgotten data (210) but low importance for the maintained data (220).

[0157] After calculating the solution, the control unit (140) can use the calculated optimal low-dimensional matrix as the initial LoRA weight (see (g) of FIG. 9). After LoRA initialization, the control unit (140) can update the layers of the large-scale language model (150) to prevent the output of the large-scale language model (150) from being distorted. That is, the unlearning system (100) extracts specific parameters that are important for the forgotten data (210) but not for the maintained data (220), so that LoRA tuning can focus on removing knowledge about the forgotten data (210).

[0158] In this way, in the initialization process according to the present invention, among the forgotten data (210) and the maintained data (220), important information about the forgotten data (210) can be specified and reflected in the initialization process. As described above, in the present invention, a weighted low-rank approximation based on Fisher information can be performed so that only specific parameters important to the forgotten data (210) are reflected in the low-dimensional matrix of LoRA. The control unit (140) applies a weighted low-rank approximation based on Fisher information, selects parameters containing relatively important information from the forgotten data (210), and initializes the matrices (A and B) of LoRA, so that learning about information to be deleted can be performed faster and more precisely.

[0159] Meanwhile, in the present invention, a process of performing unlearning on a large-scale language model to which LoRA weights are applied can be performed (S340, see FIG. 3).

[0160] The control unit (150) can perform unlearning on a large-scale language model (150) based on the results of LoRA weight initialization. That is, after the initialization process is completed, unlearning can be performed on a large-scale language model (150) to which the initialized LoRA weights are applied.

[0161] In this case, while unlearning is performed on a large-scale language model (150), the parameters of the pre-trained large-scale language model (150) are fixed, and only the weights of LoRA can be updated. At this time, when unlearning on the forgotten data (210), a set of maintenance data (220) containing general knowledge can be used together.

[0162] The control unit (140) can perform unlearning on a large-scale language model (150) using a preset loss function to effectively remove data to be removed (forgotten data) and increase the efficiency of data to be retained (maintained data). For example, in the present invention, the preset loss function for unlearning the large-scale language model (150) may be a final loss function using Inverted Hinge Loss (IHL). This loss function may be represented as shown in (h) of FIG. 9 .

[0163] Unlearning of a large-scale language model (150) can proceed by sampling a text corpus (or data) from each data set and minimizing the final loss function through stochastic gradient descent via backpropagation.

[0164] Specifically, as illustrated in FIG. 6, the large-scale language model (150) can perform unlearning on the forgotten data (210) using a loss function preset for unlearning (S404, see FIG. 4). At this time, the control unit (140) can perform unlearning on the large-scale language model using the loss function so as to increase the prediction probability for the maintained data (220) of the large-scale language model (150) while decreasing the prediction probability for the forgotten data (210). That is, the control unit (140) can perform unlearning on the large-scale language model using the loss function so as to efficiently remove the data to be removed (forgotten data) and maintain the knowledge of the data to be maintained (maintained data) without affecting the inference and generation capabilities of the large-scale language model (150).

[0165] As discussed above, the present invention effectively performs unlearning by reducing the prediction scores for actual tokens (or correct tokens) while increasing the prediction scores for only a small number of other tokens. To this end, the Inverted Hinge Loss used in the preset loss function of the present invention can be represented as shown in (a) of Figure 10.

[0166] First, the control unit (140) can perform unlearning on the large-scale language model (150) in a direction of reducing the prediction probability (or prediction score) for the forgotten data (210) using a loss function so that the large-scale language model (150) eliminates (or does not generate (or recognize, infer, etc.)) knowledge about the forgotten data (210).

[0167] More specifically, the control unit (140) can adjust the probability distribution of the large-scale language model (150) in a direction that reduces the prediction probability of the large-scale language model (150) for the correct token (true token) included in the forgotten data (210) so that the prediction probability of the forgotten data (210) of the large-scale language model (150) is reduced (see FIG. 6). The probability of the correct token can be represented as in (b) of FIG. 10.

[0168] Additionally, the control unit (140) may maximize the log probability for the maintained data (220) using a loss function so that the large-scale language model (150) maintains knowledge of the maintained data (220) (or maintains the ability to generate (or recognize, infer, etc.) the maintained data (220). For example, the control unit (140) may maximize the log probability for the maintained data (220) to maintain (or increase) the ability of the large-scale language model (150) to generate the maintained data (220), thereby balancing the model so that it does not forget more than necessary.

[0169] More specifically, the control unit (140) can adjust the probability distribution of the large-scale language model (150) to increase the prediction probability of the large-scale language model for the replacement token with the highest probability among all tokens excluding the true token, so that the large-scale language model maintains the ability to generate maintenance data (220) (see FIG. 6). Here, the replacement token may correspond to any token with the highest probability of replacing the true token among all tokens excluding the true token. Such a replacement token may be represented as in (c) of FIG. 10.

[0170] Additionally, all tokens are tokens excluding correct tokens from the vocabulary set of a pre-trained large-scale language model (150). For example, the vocabulary set may refer to a set of words or subword units that the model can use. For another example, all tokens may include at least one token from the maintenance data set (220).

[0171] The minimum probability difference between the correct token and the replacement token is represented as in (d) of Fig. 10, and ensuring that the loss value is limited to 0 or more can be represented as in (e) of Fig. 10. The Inverted Hinge Loss used in the preset loss function according to the present invention (see (a) of Fig. 10) can converge the loss to 0 when the probability of the correct token becomes sufficiently smaller than that of the replacement token with the highest possibility of replacement (e.g., when unlearning is complete). In this case, the case when unlearning is complete can be represented as in (f) of Fig. 10, and the case when unlearning is not yet complete can be represented as in (g) of Fig. 10.

[0172] In one embodiment, considering the probability of the correct token defined through the softmax function, the Inverted Hinge Loss derivative for the logit value of the language model for a specific word v at time t (see (a) of FIG. 11) can be represented as in (b) of FIG. 11.

[0173] Additionally, during the gradient calculation process of Inverted Hinge Loss, if unlearning is in progress (see (c) of Figure 11), the probability of the correct token can be decreased and the probability of the replacement token can be increased. At this time, since the absolute value of the gradient is greater than or equal to that of the replacement token, the probability of the correct token can decrease more quickly. Although the adjustment is made toward increasing the probability of the replacement token, the increase can be slower than that of the correct token (because the absolute value is smaller than that of the correct token). In addition, the probability of other tokens can increase slowly in proportion to the difference in probability between the correct token and the replacement token.

[0174] Furthermore, in the process of calculating the gradient of the Inverted Hinge Loss, if unlearning is completed (see (d) of Fig. 11), the loss can converge to 0 if the value for “the difference between the probability of the correct token and the probability of the replacement token + 1” is less than 0.

[0175] That is, the present invention reduces the probability of correct tokens and increases the probability of replacement tokens. Once the unlearning of a large-scale language model (150) for forgotten data (210) is completed, the loss converges to zero, stably terminating the unlearning process. This allows for efficient gradient updates without affecting the inference and generation capabilities of the large-scale language model.

[0176] Meanwhile, the present invention discussed above can be applied and utilized effectively in a variety of industries and services. In one embodiment, the present invention can be applied and utilized effectively in at least one of the following: opt-out applications, natural language generation-related services, conversational AI and chatbots, text generation AI and content creation, personalized education and language learning, social media and online platforms, harmful content filtering, medical and healthcare, finance and law, gaming, and virtual environments.

[0177] In relation to the large-scale language model unlearning method and system described above, the present invention may include, in the inference stage, a process of receiving a user input requesting deletion of specific data from a training data set used for training a large-scale language model (LLM) (S1210), a process of specifying data corresponding to the specific data in the training data set as forgotten data based on the received user input and specifying the remaining data excluding the forgotten data as maintained data (S1220), a process of comparing the forgotten data and the maintained data and specifying specific parameters having high importance for the forgotten data among the parameters of the large-scale language model (S1230), a process of initializing the weights of LoRA (Low-Rank Adaptation) based on the specific parameters having high importance for the forgotten data (S1240), and a process of performing unlearning on the large-scale language model to which the weights of LoRA are applied (S1250).

[0178] As illustrated in FIG. 13, the control unit (140) can receive, from the user terminal (10), a user input (e.g., “Please delete the data I request...”) requesting deletion of specific data (1400) from the training data set (200) used for training the large-scale language model (150). This user input can be received in various ways. For example, it can be received in various ways, such as the user selecting specific data through a user interface, the user inputting a text query regarding specific data to be deleted, the user inputting a voice query regarding specific data to be deleted, etc., and is not necessarily limited to the cases described above.

[0179] The control unit (140) may analyze specific data corresponding to the received user input, and based on the analyzed result, may specify data corresponding to specific data (1400) among the learning data set (200) used for learning a large-scale language model (150) as forgotten data (210). In addition, the control unit (140) may specify the remaining data excluding the forgotten data (210) as maintained data (220).

[0180] Once the forgetting data (210) and the maintenance data (220) are specified, the control unit (140) can measure a Fisher information matrix for the forgetting data (210) and a Fisher information matrix for the maintenance data (220) for each parameter of the large-scale language model (150) using the forgetting data (210) and the maintenance data (220). Then, the control unit (140) can obtain a Fisher information matrix for the forgetting data (210) measured using the forgetting data (210) and a Fisher information matrix for the maintenance data (220) measured using the maintenance data (220) based on the measured results. Furthermore, the control unit (140) can measure (or analyze, quantify) parameter importance using the obtained Fisher information matrix for the forgetting data (210) and the Fisher information matrix for the maintenance data (220). As a result of the parameter importance measurement, the control unit (140) can specify a parameter having a high parameter importance for forgotten data and a low parameter importance for maintained data (220) as a specific parameter.

[0181] Finally, the control unit (140) can initialize the weights of LoRA (Low-Rank Adaptation) based on specific parameters with high importance for the forgotten data. The control unit (150) can perform unlearning on the large-scale language model (150) to which the LoRA weights are applied. More specifically, the control unit (140) can perform unlearning on the large-scale language model using a loss function so as to increase the prediction probability for the maintenance data (220) of the large-scale language model (150) while decreasing the prediction probability for the forgotten data (210). This can be understood as decreasing the prediction probability of the large-scale language model for the correct token included in the forgotten data so as to decrease the prediction probability of the large-scale language model for the true token, and increasing the prediction probability of the large-scale language model (150) for the replacement token with the highest probability among all tokens except the correct token, so as to decrease the prediction probability of the forgotten data (210).

[0182] Furthermore, when unlearning is completed for the forgotten data (210), it is possible to determine (or judge) whether the forgotten data (210) of the large-scale language model (150) has been forgotten. The determination of whether the forgotten data (210) has been forgotten can be determined based on various criteria. For example, whether the forgotten data (210) has been forgotten can be determined independently by the administrator or user of the unlearning system (100) or the artificial intelligence model (160) based on preset criteria (e.g., user history information, user personal information (e.g., name, address, phone number, email, etc.), criteria set in relation to copyright infringement elements, etc.).

[0183] In one embodiment, to determine whether forgotten data (210) has been forgotten, a query related to the forgotten data (210) may be input into a large-scale language model (150). When unlearning of knowledge about the forgotten data (210) is complete, the large-scale language model (150) may provide a response to the user indicating that unlearning of the forgotten data (210) has been completed.

[0184] In this way, the unlearning system (100) can delete knowledge about specific data (1400) that the user has requested to be deleted, while retaining knowledge about data that the user wishes to retain. In other words, the unlearning system (100) can delete knowledge about specific data (1400) corresponding to the user's deletion request from the learning results of the large-scale language model (150), while efficiently retaining knowledge about data (220) that must be retained from the learning results.

[0185] Meanwhile, according to another embodiment of the present invention, the present invention relates to a large-scale language model optimization method and system capable of improving the learning efficiency for a large-scale language model, and a control method for a large-scale language model optimization system. More specifically, the present invention relates to a large-scale language model optimization method and system capable of improving the learning and inference speed (or performance) of a large-scale language model and achieving cost-effective learning. Furthermore, the present invention relates to a large-scale language model optimization method and system capable of effectively removing data to be removed and increasing the efficiency for data to be retained. The “large-scale language model unlearning method and system, and the control method for a large-scale language model unlearning system” discussed above may also be referred to as “large-scale language model optimization method and system, and the control method for a large-scale language model optimization system.”

[0186] Meanwhile, large language models (LLMs) possess powerful inference and memory capabilities through pre-training on massive amounts of text data. However, LLMs are constantly exposed to privacy and copyright infringement risks as they learn from human-provided text.

[0187] To prevent this, unlearning is necessary, which removes sensitive data (i.e., data to be removed) that poses risks of privacy and copyright infringement. Unlearning can refer to the process of intentionally removing (or deleting) or modifying information, patterns (or knowledge) that a model has previously learned (or previously learned). For example, unlearning can be a method for removing or modifying incorrect information, inappropriate biases, or unintended data that a model has learned.

[0188] In this regard, referring to Figure 1, the purpose of large-scale language model unlearning is to fine-tune a pre-trained large-scale language model to remove (or delete) knowledge about a data set to be removed (e.g., “Forget set”). For example, the data set to be removed (or “Forget set,” or “Delete set”) may include text data that has been requested to be deleted by a user. During the unlearning process, the large-scale language model must forget the knowledge about the data included in the data set to be removed, and retain the knowledge about the data included in the data set to be retained (e.g., “Retain set”), and must also maintain the inference and generation capabilities acquired in advance.

[0189] Accordingly, the present invention aims to provide a method and system for optimizing a large-scale language model, which can efficiently remove data to be removed and maintain knowledge about data to be retained, without affecting the inference and generation capabilities of the large-scale language model.

[0190] The present invention can be usefully utilized in various situations. More specifically, the large-scale language model optimization method and system according to the present invention can be applied and utilized effectively in various industries and services. For example, the large-scale language model optimization method and system according to the present invention can be applied and utilized effectively in systems (or applications, software, websites, programs, etc.) based on large-scale language models.

[0191] In this way, the present invention can be usefully utilized in various industries and services that require learning and / or unlearning of large-scale language models (e.g., natural language generation-related services, conversational AI and chatbots, text generation AI and content creation, customized education and language learning, social media and online platforms, harmful content filtering, medical and healthcare, finance and law, games and virtual environments, etc.).

[0192] Meanwhile, as illustrated in FIG. 2, a large-scale language model optimization system according to the present invention (hereinafter referred to as “optimization system”, 100) may include at least one of an input unit (110), an output unit (120), a storage unit (130), a control unit (140), and a large-scale language model (150).

[0193] The optimization system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may function as the storage unit (130) described above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.

[0194] Although not shown, the optimization system (100) according to the present invention may include one or more processors, which may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., a digital signal processor, a tensor processing unit (TPU), a graphics processing unit (GPU), a neural network processing unit (NPU), an application-specific integrated circuit, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a quantum processing unit (or quantum processor, QPU), etc.). The one or more processors may be configured to execute instructions stored (or included) in the storage (130), computer-readable instructions, and / or other instructions described herein. The large-scale language model optimization method and system according to the present invention may enable the memory and at least one processor to cooperate to perform data processing as described below. The processor may perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage (130).

[0195] Furthermore, the language optimization system (100) according to the present invention can perform data processing and calculation processes utilizing quantum gates, quantum entanglement, and quantum superposition states, taking into account implementation in a quantum computer environment. For example, the present invention can perform parallel operations based on qubits, and such quantum operations can operate complementarily with existing classical computers.

[0196] These quantum computers may include qubit-based parallel computing and high-speed data processing devices leveraging quantum entanglement, enabling hardware-based computational optimization using FPGAs and ASICs. Furthermore, quantum computers can utilize quantum processors capable of qubit-based parallel computing, and data processing efficiency can be improved through hybrid architectures with existing classical computers.

[0197] Meanwhile, the input unit (110) may be configured as a means for data input and may be configured in various types. For example, the input unit (110) may be configured to receive user input. The input unit (110) may be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on the input being made by the user through the input unit configuration provided in the user terminal (10).

[0198] The user terminal (10) may include at least one of a mobile phone, a smart phone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head mounted display (HMD)).

[0199] In addition, in the present invention, the input unit (110) does not necessarily mean a hardware means, and can be understood as a passage for receiving input from a user.

[0200] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, computer mouse, keyboard, keypad, touch pad, trackball, joystick, voice recognition module, or other similar device. However, the present invention does not limit the type of input unit (110).

[0201] Here, the user input may include documents, text, images (or videos), voice, etc. In this case, the optimization system (100) may further include a module that converts voice into text.

[0202] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked with the optimization system (100) according to the present invention. For example, the output unit (120) can output a page (or service page) linked with the optimization system (100) according to the present invention to the display unit of the user terminal (10). In addition, the output unit (120) does not necessarily mean a hardware means, and can be understood as a channel for outputting results to the user.

[0203] Next, the storage unit (130, or memory) serves to store various data related to the present invention, and may include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of the one or more processors.

[0204] The one or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage (130) may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage device), while in other examples, the storage (130) may be implemented using two or more physical devices.

[0205] The storage unit (130) may include computer-readable instructions and additional data. The storage unit (130) may include storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the devices and networks.

[0206] Furthermore, at least a portion of the storage unit (130) may be a cloud storage or a cloud server. The storage unit (130) may store at least a portion of data corresponding to user input received from the input unit (110) and learning data (or learning data set (200)).

[0207] That is, it can be understood that the storage unit (130) is sufficient as a space in which information necessary for the operation of the optimization system (100) according to the present invention is stored, and there are no restrictions on the physical space.

[0208] Furthermore, the storage unit (130) may store a computer program including computer program instructions. Furthermore, the storage unit (130) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (140) when loaded onto the processor of the system (100).

[0209] Next, the control unit (140) may perform a role of controlling the overall operation of the optimization system (100) related to the present invention. The control unit (140) may process signals, data, information, etc. input or output through the components of the optimization system (100) discussed above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (140) may be physically implemented by the processor discussed above.

[0210] Meanwhile, the large-scale language model (150) may be a pre-trained (or pre-trained) model using a training data set (200). The large-scale language model (150) may perform pre-training on large-scale text data (or text corpus, text data, text samples, text sequences, token sequences, language data, etc.) included (or configured) in the training data set (200). Such a large-scale language model (150) may model the likelihood of a sequence by predicting the next token when a token sequence (or text sequence) of a certain length (T) is given.

[0211] The large-scale language model (150) according to the present invention can be based on various structures. For example, the large-scale language model (150) according to the present invention can be a transformer-based model. Such a transformer can be configured with an encoder-decoder structure that receives an input sequence from an encoder and outputs an output sequence from a decoder.

[0212] Here, the structure of the transformer can be largely composed of positional encoding, multi-head attention, and a feed-forward neural network (FFN). Positional encoding generates a vector for each position so that the model can recognize the order information of the input tokens and adds this to the token embedding, thereby allowing the transformer to consider positional information without a sequential structure. Multi-head attention is a structure in which multiple attention heads calculate the relationship between each token in parallel. Each head calculates attention by multiplying the query, key, and value matrices for the input embedding, and combines these results to richly reflect the overall contextual information. The feed-forward neural network is a multi-layer perceptron that is applied independently to each token, processes the attention results, and generates an output that is passed on to the next transformer layer.

[0213] In another embodiment, the large-scale language model (150) according to the present invention may be a Mixture of Experts (MoE)-based model. A Mixture of Experts (MoE) is an efficient model architecture in which multiple experts (Experts, or models) are combined to perform a specific task. Each expert is a specialized model for solving a given problem, and may be structured to include a gate that determines which expert to select from among multiple experts. More specifically, a Mixture of Experts (MoE) can be understood as a method in which multiple models (experts) are configured, each expert learns a specific task, and only the experts specialized in the specific task are selectively activated, thereby reducing computational costs and improving the computational (or operational) efficiency and performance of the model.

[0214] Here, experts are independent models, each with a separate set of parameters and performing specialized learning for a specific task. Furthermore, a gate (or gate network) determines which expert to select based on input data. The gate uses the softmax function to calculate the probability distribution of experts to select from among multiple experts and selects the most appropriate expert based on the calculated probability. Furthermore, the outputs from each expert are weighted and combined based on the probability calculated by the gate to determine the final output. The outputs of the selected experts are combined and weighted based on their probabilities.

[0215] For example, experts included in a large-scale language model (150) based on a Mixture of Experts (MoE) may exist in various forms, such as experts specialized in natural language processing tasks, experts specialized in voice data processing, and experts specialized in image processing.

[0216] In another embodiment, the present invention can train a large-scale language model (150) using a pipeline parallelism technique. The large-scale language model (150) may be a model trained through pipeline parallelism. Pipeline parallelism may be a distributed learning technique that divides the model's computation into layers and processes them in parallel across multiple devices (e.g., GPUs) when training or inferring a large-scale deep learning model.

[0217] The present invention divides the computation of a large-scale language model (150) into multiple processing stages (e.g., layers or modules) based on a pipeline parallel processing technique, and each stage can be processed in parallel. That is, the present invention can reduce bottlenecks and increase processing efficiency by sequentially processing the layers of a large-scale language model on multiple processing devices (e.g., GPUs). This pipeline parallel processing allows computations to be performed on each device for the next input while one input is being processed, thereby utilizing computational resources more efficiently. This allows for increased learning speed and efficiency of the large-scale language model (150) while saving memory capacity. Furthermore, by utilizing multiple GPUs, the resources of each GPU can be optimally utilized, thereby increasing GPU utilization and reducing resource waste.

[0218] Furthermore, the large-scale language model (150) according to the present invention may be a pre-learned and / or fine-tuned model using at least one of supervised learning, reinforcement learning, and supervised fine tuning.

[0219] The large-scale language model (150) learned in the present invention may correspond to at least one of a large-scale language model based on supervised learning, a large-scale language model (150) based on reinforcement learning, and a large-scale language model (150) adjusted through supervised fine-tuning.

[0220] Supervised learning can be a method of training a model based on input data and its corresponding correct labels (output). More specifically, supervised learning trains a model using input-output pairs (labeled data). During the learning process, input data and their corresponding correct answers (labels) are given, and the model learns patterns based on these data, enabling it to make predictions about new data. For example, supervised learning encompasses regression and classification problems, and serves as a guide for predicting output values ​​based on training data.

[0221] Reinforcement learning is the process of learning a policy that maximizes (or maximizes) rewards as an agent interacts with its environment. Reinforcement learning is primarily used to solve sequential decision problems, where the agent selects an action based on a state, receives a reward as a result of that action, and learns based on this. Here, the agent makes decisions based on information gained through interaction with the environment, and the environment provides feedback (rewards) in response. The agent improves its behavior to maximize long-term rewards.

[0222] For example, in the present invention, when a large-scale language model (150) is trained based on reinforcement learning, reinforcement learning may be performed using at least one of the Proximal Policy Optimization (PPO) algorithm and / or inference-oriented reinforcement learning.

[0223] Here, Proximal Policy Optimization (PPO) is an efficient and stable algorithm for policy optimization in reinforcement learning, based on the policy gradient technique. The primary goal of Proximal Policy Optimization (PPO) is to optimize policy parameters (i.e., the action selection probability distribution) to maximize reward in a given environment.

[0224] Meanwhile, let's assume that after the pre-training of the large-scale language model (150) is completed, the user requests to delete specific data (or a specific data set) from the training data set (200) used for training the large-scale language model (150). In the present invention, the specific data that the user wishes to remove (unlearn) may be named "forgotten data" or "forgotten data set." In addition, in the present invention, even after the unlearning of the forgotten data (210) is completed, data containing knowledge that the large-scale language model (150) should not forget may be named "maintained data" or "maintained data set."

[0225] In this case, in the unlearning process, the large-scale language model (150) can unlearn the text sequence by maximizing the next token prediction loss of at least one text sequence included in the forgotten data (210) to assign a low probability to the forgotten data (210).

[0226] Here, maximizing the prediction loss can mean maximizing the next token prediction loss through gradient ascent, which is the opposite of gradient descent. Unlike gradient descent (or gradient descent, gradient descent, etc.), which increases the prediction probability (or prediction score, generation probability, generation score, etc.) of the model's correct answer by minimizing the loss function, gradient ascent (or gradient ascent, gradient ascent) decreases the prediction probability of the correct answer by maximizing the loss function. In this regard, the log-likelihood (or log-likelihood) can be implemented as a cross-entropy loss (or cross-entropy loss) for incremental tokens. This gradient ascent can be a model that adjusts to maximize the cross-entropy loss.

[0227] In this way, the gradient ascent method discussed above trains the model to decrease the probability of the correct token and increase the probability of the remaining tokens. In other words, the model trains to increase the probability of all non-correct tokens.

[0228] Therefore, maximizing the prediction loss may mean adjusting the probability distribution of the large-scale language model (150) to induce the large-scale language model (150) not to learn or not to predict certain text sequences. For example, the prediction loss for certain tokens may be maximized so that the prediction probability for certain tokens is reduced so that the large-scale language model (150) predicts certain tokens with a lower probability than those that it had predicted with a high probability, thereby adjusting the probability distribution so that the large-scale language model (150) makes inaccurate predictions for certain tokens (i.e., suppresses generation of forgotten data, suppresses generation of inappropriate sentences, etc.).

[0229] However, in the present invention, in order to prevent i) the loss from not converging but increasing or diverging without a finite boundary, ii) the logit for all other tokens from increasing, resulting in unnecessary additional forgetting, or iii) unstable results occurring during the optimization process, the probability distribution of the model is adjusted in a direction that increases the prediction probability (e.g., loss function) of the token (alternative token) with the highest probability among all tokens excluding the correct token while decreasing the prediction probability of the correct token, without increasing the prediction probability of all tokens excluding the correct token. That is, the present invention uses a method of adjusting the model by focusing the gradient update only on the minimum alternative token with a high possibility of replacing the correct token. More specific details about this will be described later.

[0230] Meanwhile, the present invention provides a large-scale language model optimization method and system capable of improving the learning efficiency of a large-scale language model, and a control method for the large-scale language model optimization system. More specifically, the present invention provides a large-scale language model optimization method and system capable of improving the learning and inference speed (or performance) of a large-scale language model and achieving cost-effective learning. Furthermore, the present invention provides a large-scale language model optimization method and system capable of effectively removing data to be removed and increasing the efficiency of data to be retained. Hereinafter, the large-scale language model optimization method according to the present invention will be examined in more detail.

[0231] In the present invention, a process can be performed in which a learning data set is specified, a large-scale language model (LLM) is trained using the specified learning data set, and a large-scale language model trained using the learning data set is obtained based on the training.

[0232] The control unit (150) can specify a learning data set to be used for training a large-scale language model (150). In this case, the method (or method, standard, etc.) for specifying the learning data set (or learning data) in the present invention may vary.

[0233] In one embodiment, the control unit (150) may, based on a user's input of specific data, specify specific data corresponding to the user input as learning data to be used for learning the large-scale language model (150).

[0234] In another embodiment, the control unit (150) may collect data from various sources (e.g., databases, web crawling, APIs, servers linked to the optimization system (100), external servers, etc.) and store the collected data in the storage unit (130). In addition, the control unit (150) may specify at least some of the data stored in the storage unit (130) as learning data to be used for learning the large-scale language model (150).

[0235] However, the method of specifying the learning data set (200) in the present invention is not limited to the examples mentioned above, and can be specified by various methods in addition to the examples mentioned.

[0236] This learning data set (200) may include various data. For example, the learning data set (200) may include at least one of large-scale supervised learning (SL) data, large-scale reinforcement learning (SL) data, language understanding and reasoning related data, question and answer data, reading comprehension data, directive interpretation data, language modeling data, language data of various countries, mathematical data, scientific data, code (or coding) data, and learning prompts (or learning or templates).

[0237] Based on the specified learning data set (200), the control unit (150) can train a large-scale language model (150) to be learned using the specified learning data set (200). The control unit (150) can process the specified learning data set (200) as input to the large-scale language model (150).

[0238] At this time, in the learning process of the large-scale language model (150), the control unit (150) can train the large-scale language model (150) based on (or using) an attention mechanism.

[0239] An attention mechanism is a mechanism used in language modeling. It assigns weights to each element in a given input sequence (e.g., a text sequence tokenized through a tokenization process) and evaluates the importance of each element based on the weights. These weights are automatically adjusted as the model learns, determining how much each element contributes to the output. This attention mechanism allows the model to focus on the most relevant input elements at each output generation point, rather than treating the entire input equally.

[0240] More specifically, the attention mechanism is structured around three elements: query, key, and value. The attention mechanism calculates the similarity (or score) between each query and each key, determines a weight for each value based on the similarity, and then combines the values ​​(weighted sum) using the weights to generate the final attention output. For example, the process of computing (or calculating) attention involves computing the attention score (or similarity score) between each query and each key using the dot product and / or cosine similarity, and applying softmax to the computed attention score to transform it into a probability distribution. Then, the weights are multiplied by the softmax result for the value vectors, and the sum is used to generate the final attention output.

[0241] That is, in the attention mechanism, each query token calculates a relevance score for all previous keys to generate a weighted sum of values. In this case, the attention operation for a sequence input with a specific length can be represented as in (e) of Fig. 11. At this time, Attn in (e) of Fig. 11 is an attention function, and the definition of the attention function, which includes the attention weight between the query and the key and the dimension of the key vector, can be represented as in (f) of Fig. 11.

[0242] In this regard, the control unit (150) can train a large-scale language model (150) for a training data set (200) based on a preset attention mechanism. In the present invention, the types of preset attention mechanisms that can be used (or utilized, used, etc.) in the training process of the large-scale language model (150) may vary. For example, the preset attention mechanism may include at least one of sparse attention, sliding-window attention, multi-head attention, self-attention, global attention, local attention, and cross-attention.

[0243] A large-scale language model (150) can perform learning on a learning data set (200) based on a preset mechanism for long-context modeling.

[0244] Specifically, a large-scale language model (150) can perform learning on a learning data set (200) based on a sparse attention mechanism to process long contexts (or long contexts, long contexts, etc.).

[0245] A sparse attention mechanism may be a mechanism that, instead of computing interactions between all elements within an input sequence (or input text sequence), pays attention only to some selected token pairs according to predefined rules (or fixed methods) and / or dynamic methods (or learning-based methods).

[0246] More specifically, sparse attention can be a technique that minimizes (reduces) computational complexity and memory usage by selectively performing attention operations on only a few important token pairs, rather than performing operations on all input token pairs. In other words, sparse attention can increase computational efficiency and optimize memory usage by activating only a portion (the sparse portion) of the entire attention map.

[0247] The control unit (150) may train the large-scale language model (150) using a sparse attention mechanism to enable the large-scale language model (150) to process long contexts included in the training data set (200) well. In this case, when at least one text sequence included in the training data set (200) is input, the large-scale language model (150) may selectively perform an attention operation only on some query-key pairs selected according to a preset criterion (or a preset selection criterion) among all query-key pairs included in the input text sequence (or tokenized text). That is, the large-scale language model (150) may perform an attention operation by selectively connecting only a certain percentage of the entire query-key pairs of the text sequence.

[0248] The preset criteria may include various criteria related to at least one of predefined rules (or fixed methods) and dynamic methods (or learning-based methods). For example, the preset criteria may include at least one of the following: i) each token only pays attention to adjacent tokens (e.g., 2-3 before or after it); ii) only connect tokens spaced apart by a certain interval (e.g., always the third token); iii) divide tokens into blocks and pay attention only between specific blocks; iv) give full attention only to special tokens (e.g., [CLS], first token of a sentence, etc.); v) select only the k most similar key tokens for each query token; vi) cluster similar tokens and pay attention only within the cluster. However, the preset criteria are not necessarily limited to the examples mentioned and may naturally include various other criteria.

[0249] In this way, the present invention enables a large-scale language model (150) to efficiently process long sequences (long contexts) by calculating attention only with a selected portion of keys instead of calculating with all keys, thereby significantly reducing memory usage and improving computational efficiency.

[0250] Meanwhile, a large-scale language model (150) can be trained to process the text sequence included in the learning data set (200) using various methods (or techniques) in the process of processing the text sequence through an attention mechanism.

[0251] When a text sequence is input to a large-scale language model (150), the control unit (150) can be trained to process tokens included in the input text sequence in sliding-window attention units, blockwise selection and processing, or processing based on importance.

[0252] These processing methods (or methods, techniques, etc.) can be understood as being used for sparse computation, which considers only selected token pairs rather than the entire token set. These processing methods can be used in various models aimed at improving computational efficiency, reducing model weight, and achieving sparse attention.

[0253] In one embodiment, the large-scale language model (150) can process tokens included in an input text sequence in sliding window units. In this case, the large-scale language model (150) can perform learning on a training data set (200) based on sliding window attention.

[0254] A sliding window attention mechanism can be a mechanism that performs attention by dividing the input sequence into windows of a specific size (or a preset size). Conventional attention mechanisms calculate attention for all input elements, which increases computational cost as the input sequence lengthens. To address this issue, sliding window attention defines a window of a fixed size in the input sequence when processing a long input sequence (or text sequence) and performs attention only on the tokens contained within the defined window.

[0255] More specifically, sliding window attention divides an input sequence into small windows and calculates attention for each divided window. The size of the window can be fixed or flexibly adjusted by the system (100). Such sliding window attention is performed on only a portion of an input sequence or input image, so that instead of calculating attention for the entire input, attention is applied only within a limited range. In other words, sliding window attention can efficiently process long sequences by limiting the attention operation of a large-scale language model (150) to a limited window around each token.

[0256] The control unit (150) can train the large-scale language model (150) using a sliding window attention mechanism to enable the large-scale language model (150) to properly process long text sequences included in the training data set (200). In this case, when at least one text sequence included in the training data set (200) is input, the large-scale language model (150) can set (or select) a sliding window of a preset size based on the input text sequence. At this time, the window slides along the input sequence, and the large-scale language model (150) can perform an attention operation on each window. That is, the large-scale language model (150) can reduce the amount of computation by performing the attention operation only within a fixed window, instead of calculating attention for all inputs at once. For example, if the length of the input text sequence is 100 and the window size is set to 10, each location will perform attention only with its 10 neighbors centered on it.

[0257] In this way, the present invention reduces memory usage and optimizes computational costs by dividing the sequence into smaller pieces rather than processing the entire sequence at once when processing long sentences or text. Thus, the present invention conserves memory and computational resources when processing long sentences or text, enabling the construction of a memory-efficient, large-scale language model (150).

[0258] In another embodiment, the large-scale language model (150) can select and process tokens included in an input text sequence in block units. This can be understood as a method of performing attention operations by selecting tokens of the input sequence in block units. Instead of calculating attention between all tokens, the input sequence is divided into multiple blocks, and only important tokens within or between each block are selected (or screened) to perform attention operations.

[0259] When at least one text sequence included in a learning data set (200) is input, the large-scale language model (150) can divide the input text sequence into a plurality of blocks having a specific size (or a preset size), select only important tokens within each block or between each block, and perform an attention operation on the selected tokens. For example, assuming that the length of a text sequence is 1000, if it is divided into a plurality of blocks (e.g., 10) having a preset size, each of the plurality of blocks can include 100 tokens. The large-scale language model (150) can evaluate the importance of tokens included in each of the plurality of blocks, and select at least one token with high importance among the tokens included in each of the plurality of blocks based on the evaluated importance, and perform an attention operation (calculation) on the selected token. At this time, each of the multiple blocks can be understood as being processed independently, and the selection of important tokens can be evaluated and selected based on at least one of the attention score (or score), activation magnitude, and projection (or projection) score.

[0260] That is, in the present invention, each sequence is divided into blocks and attention operations are performed only within the blocks, thereby increasing the efficiency of operations and memory.

[0261] In another embodiment, the large-scale language model (150) can process tokens included in an input text sequence based on their importance. When at least one text sequence included in the training data set (200) is input, the large-scale language model (150) can dynamically remove tokens with low utilization based on the attention score for the input text sequence, thereby removing tokens determined to have low importance from memory during a future prediction process. In addition, the large-scale language model (150) can identify important token features through attention weight analysis and selectively maintain only the features of the identified important tokens, thereby improving memory usage efficiency.

[0262] Meanwhile, a large-scale language model (150) can be trained using at least one of a low-precision training technique and a mixed precision training technique.

[0263] Low-precision learning may be a technique for performing learning on a large-scale language model (150) using a low-precision numeric representation (or format) during the learning process of the large-scale language model (150). More specifically, low-precision learning is a technique for performing parameters and operations of the large-scale language model (150) using a low-precision numeric representation. Generally, deep learning uses 32-bit floating point (float32), but low-precision learning uses 16-bit (float16), 8-bit (int8) or a lower bit number to learn the model. In the present invention, through low-precision learning on a large-scale language model (150), the learning and computational efficiency of the large-scale language model (150) can be improved, and memory and computational resources can be saved. At this time, low-precision learning allows the large-scale language model (150) to perform learning using less memory and at a faster speed while maintaining high performance.

[0264] Mixed-precision learning may be a technique for training a large-scale language model (150) by mixing various numerical precisions. More specifically, mixed-precision learning is a technique for training a model by mixing two data types (e.g., 32-bit and 16-bit), such that important calculations (e.g., weight updates, model parameter updates, loss calculations, etc.) can be performed with high precision (32-bit), and other calculations (e.g., activation functions, intermediate calculations, etc.) can be performed with low precision (16-bit). In other words, mixed-precision learning can significantly improve training speed and memory efficiency while maintaining model accuracy. In this way, the present invention can maximize GPU efficiency, reduce memory usage, and increase learning speed through mixed-precision learning for a large-scale language model (150).

[0265] Meanwhile, the large-scale language model (150) learned through the process discussed above may correspond to a teacher model configured to distill knowledge learned (or acquired) through learning on a learning data set (200) into at least one model corresponding to a student model.

[0266] Model distillation, also known as knowledge distillation, is a technique for transferring knowledge learned from a large model (e.g., a teacher model) to a smaller model (e.g., a student model) to improve computational efficiency while maintaining performance.

[0267] In other words, the present invention can grant the inference capabilities of a large model to a small model through model distillation, without retraining the small model using separate training data. This allows the present invention to be effectively utilized even in resource-constrained environments and significantly reduces computational costs during the model training process. Furthermore, by efficiently compressing the knowledge of the large model and learning from it, the small model can maintain the performance achieved by the large model.

[0268] Furthermore, the control unit (150) can obtain a large-scale language model (150) learned through the process described above.

[0269] However, the large-scale language model utilized in the unlearning process described below in the present invention is not limited to models trained through the process described above. For example, the large-scale language model may include a model previously trained through supervised learning using labeled training data. Alternatively, it may include a model trained through reinforcement learning.

[0270] Meanwhile, in the present invention, a process of specifying forgetting data and maintenance data for a previously trained large-scale language model (LLM) from a training data set stored in memory can be performed.

[0271] The control unit (140) can specify, among the pre-learned data for a large-scale language model (150), forget data (e.g., “forget set”) corresponding to data for which the learned result is to be removed, and retain data (e.g., “retain set”) corresponding to data for which the learned result is to be maintained.

[0272] As illustrated in FIG. 4, the control unit (140) can specify, in the learning data set (200) used in the learning process of the large-scale language model (150), learning data to be unlearned so that recognition is impossible through the large-scale language model (150) as forgetting data (210), and can specify learning data to be maintained in a state where recognition is possible through the large-scale language model (150) as maintenance data (220).

[0273] For example, the control unit (140) can specify text samples (or text data, text sequences, token sequences, language data, etc.) that are to be removed (i.e., targets for removal) to make them unrecognizable through the large-scale language model (150) from the learning data set (200) used for learning the large-scale language model (150) as forgotten data (210).

[0274] As another example, the control unit (140) can specify, as maintenance data (220), text samples that are to be maintained in a state that is recognizable through the large-scale language model (150) in the training data set (200) used for training the large-scale language model (150).

[0275] In this regard, various methods for specifying forgotten data and retained data in the present invention may exist. In the present invention, forgetting data and retained data may be specified based on user input (or request) or by the unlearning system (100) itself.

[0276] In one embodiment, after pre-training of a large-scale language model (150) is completed, if a user requests deletion of specific data from a training data set (200) used for training the large-scale language model (150), the control unit (140) may specify the specific data requested by the user (or received from the user terminal (10)) as forgotten data (210) and specify the remaining data excluding the specified forgotten data (210) as maintained data (220).

[0277] In another embodiment, the unlearning system (100) may analyze the training data set (200) used in the training process of the large-scale language model (150) based on preset criteria (or conditions). Here, the preset criteria may be criteria set in relation to the user's personal information (e.g., name, address, phone number, email, etc.) or copyright infringement elements. As a result of the analysis, if data related to the preset criteria is detected (or filtered) in the training data set (200) used in the training of the large-scale language model (150), the unlearning system (100) may specify the detected data as forgotten data (210) and specify the remaining data excluding the specified forgotten data (210) as maintained data (220).

[0278] In another embodiment, when the unlearning system (100) performs fine tuning (or fine tuning) on ​​a large-scale language model (150) trained based on large-scale text data so that only text data corresponding to a specific item (or type) can be recognized, the learning data related to the data corresponding to the specific item can be specified as maintenance data (220), and the learning data related to the data corresponding to an item different from the specific item can be specified as forget data (210).

[0279] At this time, the unlearning system (100) may be understood as performing unlearning on data corresponding to the above-mentioned other items, and here, the specific item or other items may be understood as meaning a category (or type) for data recognizable through a large-scale language model (150).

[0280] Additionally, the maintenance data (220) may include learning data related to items to be newly learned in addition to data previously learned for the large-scale language model (150).

[0281] However, in the present invention, the method (or method) for specifying the forgetting data (210) and the maintenance data (220) is not necessarily limited to the cases described above, and may be specified in various ways.

[0282] Furthermore, in the present invention, the learning data set (200) can be represented as in (a) of FIG. 7, the forgetting data (210) can be represented as in (b) of FIG. 7, and the maintenance data (220) can be represented as in (c) of FIG. 7.

[0283] Meanwhile, in the present invention, a process can be performed to compare forgotten data and maintained data to identify specific parameters with high importance for forgotten data among the parameters of a large-scale language model.

[0284] Here, comparing the forgotten data and the maintained data can also be understood as comparing and analyzing the relative importance of parameters for the forgotten data (210) and the maintained data (220).

[0285] Parameters can refer to learnable values, including the model's weights and biases. Parameters are adjustable values ​​that a model uses to learn data and make predictions (or inferences). For example, in an artificial neural network, the weights of each layer can be considered parameters. These parameters are optimized through the learning process, allowing the model to learn patterns from data and make predictions.

[0286] In this regard, parameter (or parameter, weight, etc.) changes due to adaptation of a large-scale language model (150) may inherently have a low-rank (or low-dimensional, low-rank, etc.) structure. More specifically, parameter changes of a large-scale language model (150) due to adaptation of the large-scale language model (150) may be approximated as low-rank matrices based on the assumption that they have a low rank.

[0287] Here, adaptation of a large-scale language model is a process of changing or adjusting a pre-trained model to suit a specific purpose (or task), and may include, for example, training, fine-tuning, and unlearning.

[0288] In addition, the fact that the parameter changes of the large-scale language model (150) have a low rank (or low-rank structure) may mean that when the parameters (e.g., the weight matrix) of the large-scale language model (150) change, the change occurs in a subspace with a relatively low dimensionality in the entire parameter space. This may mean that not all weights change independently, but rather the change occurs along a specific low-rank structure. In other words, it may mean that the main change occurs in a specific low-rank (e.g., low-rank) part rather than the entire weight matrix of the large-scale language model (150). For example, when the model learns data from a specific domain, it may also be understood that not all neurons are updated equally, but only some neurons play a major role.

[0289] As discussed above, assuming that the parameter change due to the adaptation of the large-scale language model (150) is low-rank, LoRA (Low-Rank Adaptation) models the parameter change of each linear weight (e.g., linear layer (or linear layer, Linear Layer) weight (or weight matrix) of the large-scale language model (150)) of the large-scale language model (150) as the product of multiple (two) low-rank matrices. Here, each linear weight can be represented as in (d) of FIG. 7, and the parameter change can be represented as in (e) of FIG. 7. In addition, multiple (two) low-rank matrices (A and B) can be represented as in (f) of FIG. 7, and the rank of the LoLA adapter can be represented as in (g) of FIG. That is, when input is given to a large-scale language model (150), the output of the adapted linear layer can be represented as in (h) of Fig. 7.

[0290] In this case, while fine-tuning the large-scale language model (150) is performed, the existing weights of the pre-trained large-scale language model (150) are fixed, and only the low-rank matrices (A and B) can be updated through gradient descent. To ensure that the initial attachment of the LoRA adapter does not change the output of the large-scale language model (150), LoRA defaults to initializing the first low-rank matrix (A) with a Kaiming-uniform distribution and setting the second low-rank matrix (B) to a zero matrix. After the fine-tuning of the large-scale language model (150) is completed, the LoRA adapter can be merged with the existing weights (see (i) of FIG. 7).

[0291] That is, LoRA can mean a method (or technique, method, etc.) that updates a model through low-rank changes instead of adjusting the entire weight by modeling the change in a weight matrix as a product of two low-rank matrices. With reference to this, the present invention can perform unlearning on a large-scale language model (150) in a way that reduces the amount of computation and increases efficiency based on the assumption that parameter changes of the large-scale language model (150) mainly occur in the low-rank region, without retraining the large-scale language model (150).

[0292] Meanwhile, the present invention can perform weighted low-rank decomposition by measuring Fisher information for each parameter (or weight) of a large-scale language model (150) and initializing adapter weights (A and B) using the measured results. In the present invention, this process can also be referred to as “Fisher-weighted LoRA Initialization (FLoRA).”

[0293] The FLoRA process according to the present invention aims to enable a large-scale language model (150) to quickly unlearn the forgotten data (210) and minimize performance degradation for the maintained data (220) by setting parameters that are more important for the forgotten data (210) to be adjusted first. To this end, the present invention quantifies the parameter importance for each of the forgotten data (210) and the maintained data (220) using a Fisher information matrix, and initializes the large-scale language model (150) based on this.

[0294] The control unit (140) can measure the parameter importance for each of the forgotten data (210) and the maintained data (220) using the Fisher Information Matrix. The control unit (140) can measure how important each weight (or specific weight) of the large-scale language model (150) is for each of the forgotten data (210) and the maintained data (220) using the Fisher Information Matrix.

[0295] Here, the Fisher information matrix may refer to the amount of information that the learning data set (200) provides to the parameters of the large-scale language model (150). The Fisher information matrix may be represented as in (a) of Fig. 8.

[0296] This Fisher information matrix can be computed as the second central moment of the first partial derivative of the log-likelihood (see the left side of (c) of FIG. 8). However, since it is computationally infeasible to integrate over the space of the training data set (200), the present invention can utilize the empirical Fisher information (or empirical Fisher information matrix). The empirical Fisher information can be represented as in (b) of FIG. 8. For large-scale language models, the empirical Fisher information can be computed as the root mean square of the propagated gradients for language modeling purposes (e.g., cross-entropy loss) (see (c) of FIG. 8). However, in this specification, the terms “Fisher information matrix (or Fisher information)” and “empirical Fisher information matrix (or empirical Fisher information)” may be used interchangeably.

[0297] A Fisher information matrix may refer to a value indicating how important a specific parameter of a model is for a given data. More specifically, a Fisher information matrix may refer to a measure (or value) indicating whether at least one parameter (or target parameter, specific parameter, target parameter, etc.) of a large-scale language model is important for a text sample (e.g., a sentence, document, token sequence, paragraph, etc.) included in the forgotten data (210) or the maintained data (220). For example, a Fisher information matrix may be a measure indicating whether a specific parameter of a large-scale language model (150) is important for a specific data, and the measure indicating the importance may be expressed as a slope.

[0298] In this case, parameters with a large absolute slope can be identified (or determined, judged, considered, etc.) as being important in the data. In other words, parameters with a large slope for specific data can be identified as playing a significant role in generating the data.

[0299] Accordingly, a parameter having a high absolute value of the slope for the forgotten data (210) can be specified as having a relatively high importance in the forgotten data (210), and a parameter having a high absolute value of the slope for the maintained data (220) can be specified as having a relatively high importance in the maintained data.

[0300] In this regard, the control unit (140) can measure the Fisher information matrix for each of the forget data (210) and the maintenance data (220).

[0301] Specifically, the control unit (140) can measure a Fisher information matrix for the forgotten data (210) and a Fisher information matrix for the maintained data (220) using the forgotten data (210) and the maintained data (220) for each parameter of the large-scale language model (150). The control unit (140) can obtain a Fisher information matrix for the forgotten data (210) measured using the forgotten data (210) and a Fisher information matrix for the maintained data (220) measured using the maintained data (220) for each parameter of the large-scale language model (150). Here, the Fisher information matrix measured for the forgotten data (210) can be represented as in (d) of FIG. 8, and the Fisher information matrix measured for the maintained data (220) can be represented as in (e) of FIG. 8.

[0302] In another embodiment, the control unit (140) may set (or select) at least one target parameter among the parameters of the large-scale language model, and measure a Fisher information matrix for the forgotten data (210) and a Fisher information matrix for the maintained data (220) using the forgotten data (210) and the maintained data (220) for the target parameter. The control unit (140) may obtain a Fisher information matrix for the forgotten data (210) measured using the forgotten data (210) and a Fisher information matrix for the maintained data (220) measured using the maintained data (220) for the target parameter. In this case, the target parameter may be randomly set (or selected) or may be set based on a preset criterion (e.g., a parameter with a high learning weight, a parameter with a high probability distribution of generating forgotten data, etc.).

[0303] In addition, the control unit (140) can measure (or analyze, quantify) the parameter importance for each of the forgotten data (210) and the maintained data (220) by using the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220) (S401, see FIG. 4).

[0304] Here, parameter importance can be measured using the relative Fisher information matrix between the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220). The control unit (140) can use the relative Fisher information matrix between the forgotten data (210) and the maintained data (220) as an importance index to preferentially specify (or select, identify, etc.) parameters that have high importance for the forgotten data (210) but low importance for the maintained data (220). The relative Fisher information matrix can be represented as shown in (f) of FIG. 8.

[0305] This relative Fisher information matrix can be calculated using the Fisher information matrix for the forgotten data (210) and the Fisher information matrix for the maintained data (220). In this case, calculating the relative Fisher information matrix can also be understood as calculating the relative importance of parameters for the forgotten data (210) and the maintained data (220).

[0306] The control unit (140) can calculate a relative Fisher information matrix between the Fisher information matrices measured for each of the forgotten data (210) and the maintained data (220), and measure (or analyze, quantify) the parameter importance for each parameter of the large-scale language model (150) based on the calculated result.

[0307] As a result of the parameter importance measurement, the control unit (140) can specify (or determine, select, etc.) at least one parameter among the parameters of the large-scale language model (150) that has a high Fisher information matrix measured for the forgotten data (210) as a specific parameter. The control unit (140) can specify a parameter with high parameter importance for the forgotten data (210) as a specific parameter (S402, see FIG. 4). In this case, the specific parameter can be understood as a parameter with a high absolute value of the slope for the forgotten data (210) and thus has relatively high importance in the forgotten data (210).

[0308] That is, in the unlearning process according to the present invention, a high Fisher information for the forgotten data (210) may indicate that the next token prediction loss in the forgotten data (210) induces a large absolute slope in the corresponding parameter. Therefore, in the present invention, such a parameter may be specified as a specific parameter important for generating a sequence of the forgotten data (210). In this specification, the specific parameter may also be referred to as a "specific weight," "important parameter," or "important weight."

[0309] In another embodiment, the control unit (140) may calculate a relative Fisher information matrix between the Fisher information matrix for the measured forgotten data (210) and the Fisher information matrix for the maintained data (220) for the target parameter, and based on the calculated result, measure (or analyze) which data the target parameter has a higher importance for. As a result of measuring the target parameter importance, if the target parameter has a high importance for the forgotten data (210), the control unit (140) may specify the target parameter as a specific parameter. Alternatively, as a result of measuring the target parameter importance, if the target parameter has a high importance for the maintained data (220), the control unit (140) may exclude the target parameter from the initialization target and perform the above process again to specify a parameter having a high importance for the forgotten data (210).

[0310] Accordingly, the control unit (140) may specify, as a specific parameter, a parameter among the parameters of the large-scale language model (150) in which the Fisher information for the forgotten data (210) is relatively higher than the Fisher information for the maintained data (220). For example, the control unit (140) may preferentially specify a specific parameter in which the Fisher information for the forgotten data (210) is high but the Fisher information for the maintained data (220) is low.

[0311] That is, the control unit (140) can compare the forgotten data (Fisher information for the forgotten data) and the maintained data (e.g., Fisher information for the maintained data) to specify a parameter that has a high importance (or importance) for the forgotten data (210) and a low importance (or importance) for the maintained data (220), as a specific parameter that should be set to be preferentially adjusted during the unlearning process.

[0312] Meanwhile, in the present invention, a process of initializing the weights of LoRA (Low-Rank Adaptation) based on specific parameters with high importance for forgotten data can be performed.

[0313] The control unit (140) can initialize the weight of the preset adapter based on the importance of a specific parameter.

[0314] Here, the importance of a specific parameter may be a value (or indicator) that numerically represents the contribution of the parameter when a large-scale language model (150) predicts tokens included in the forgotten data (210). In other words, it is a value that represents how much influence a specific parameter has on the forgotten data (210), and this can be quantified using the Fisher information and / or relative Fisher information discussed above.

[0315] Additionally, the preset adapter may be related to the LoRA adapter discussed above. For example, the control unit (140) may model the parameter changes of a large-scale language model (150) using a preset adapter configured to approximate the parameter changes of a large-scale language model (150) through a multiplication of multiple low-rank matrices based on specific parameters with high importance for the forgotten data (210), and initialize the weights of the adapter.

[0316] In the present invention, the initialization process may be a process of calculating the relative importance of parameters for forgotten data (210) and maintained data (220) and performing LoRA initialization based on the calculated results. Furthermore, the initialization process may be a process of initializing LoRA weights (A, B) such that parameters important to forgotten data (210) are greater.

[0317] Additionally, the initialization process may be a process in which the weights (A, B) of the LoRA adapter are initialized based on (or as a standard) a specific parameter with high importance for the forgotten data (210), so that the unlearning process is centered on the specific parameter, thereby allowing the weights to focus on removing knowledge of the forgotten data (210).

[0318] Additionally, the initialization process may be a process of applying LoRA to specific parameters with high importance for the forgotten data (210) (e.g., decomposing the specific parameters into LoRA adapter weights (A, B)) and initializing the LoRA weights (A, B) by reflecting information about the specific parameters (Fisher information, relative Fisher information, etc.). For example, the existing weights (W) of the pre-trained large-scale language model (150) are fixed so as not to change the output of the model when LoRA is applied.

[0319] Additionally, the initialization process may be a process of initializing the weight of LoRA (or the weight of the LoRA adapter) based on the importance (or importance information) of a specific parameter with high importance for the forgotten data (210), thereby effectively removing knowledge about the forgotten data (210).

[0320] In addition, the initialization process may be a process of initializing the weights (low-rank matrix) of LoRA so that the importance of specific parameters with high importance for the forgotten data (210) is reflected. In addition, the initialization process may be a process of initializing the weights of LoRA based on the relative importance of specific parameters with high importance for the forgotten data (210), thereby inducing unlearning to be performed quickly and efficiently centered on specific parameters through adjustment of the corresponding parameters.

[0321] Additionally, the initialization process may be a process of initializing the LoRA adapter (i.e., the low-rank matrix (A, B) of LoRA) in a direction that reflects the importance of specific parameters with high importance for the forgotten data (210), thereby enabling unlearning to be performed effectively (at this time, the weights of the pre-learned large-scale language model are fixed).

[0322] This initialization process can select (or specify) information that is important only to the forgotten data (210) among the forgotten data (210) and the maintained data (220) and reflect it in the initialization of LoRA. That is, the control unit (140) can initialize the weights of LoRA so that the unlearning of the large-scale language model is centered on specific parameters with high importance for the forgotten data (210) (S403, see FIG. 4).

[0323] In one embodiment, initializing the LoRA weights with parameters important for generating the forgotten data (210) can be advantageous during the unlearning process, as the gradient modifies only the parameters important for the forgotten data (210) and leaves the remaining parameters unchanged. Given the relative importance of each parameter of the large-scale language model (150), the solution to the Weighted Low-Rank Approximation (WLRA) problem can be expressed as the initialization of the LoRA adapter weights. This can be represented as shown in (a) of FIG. 9 .

[0324] At this time, the present invention assumes that the parameters of each row of the large-scale language model (150) have equal importance, and the weighted low-rank approximation problem can be redefined using the square root of the row-wise sum of the relative Fisher information matrix. This can be represented as in (b) of Fig. 9.

[0325] Here, a vector with all elements equal to 1 can be represented as in (c) of Fig. 9, and a function that converts a vector into a diagonal matrix and a matrix-vector product can be represented as in (d) of Fig. 9. In this way, the row-wise weighted low-rank approximation (or weighted low-dimensional approximation based on Fisher information) problem has a closed-form solution, which can be derived by applying the singular value decomposition (SVD) of the rank (r), as shown in (e) of Fig. 9. At this time, the optimal low-rank matrices (A and B) in the present invention can be calculated as shown in (f) of Fig. 9.

[0326] Here, the solution may include optimal LoRA weights obtained by low-rank approximation of existing model weights (W) containing important information in the forgotten data (210). For example, the solution may be a low-rank approximation of a weight matrix that has high importance for the forgotten data (210) but low importance for the maintained data (220).

[0327] After calculating the solution, the control unit (140) can use the calculated optimal low-dimensional matrix as the initial LoRA weight (see (g) of FIG. 9). After LoRA initialization, the control unit (140) can update the layers of the large-scale language model (150) to prevent the output of the large-scale language model (150) from being distorted. That is, the unlearning system (100) extracts specific parameters that are important for the forgotten data (210) but not for the maintained data (220), so that LoRA tuning can focus on removing knowledge about the forgotten data (210).

[0328] In this way, in the initialization process according to the present invention, among the forgotten data (210) and the maintained data (220), important information about the forgotten data (210) can be specified and reflected in the initialization process. As described above, in the present invention, a weighted low-rank approximation based on Fisher information can be performed so that only specific parameters important to the forgotten data (210) are reflected in the low-dimensional matrix of LoRA. The control unit (140) applies a weighted low-rank approximation based on Fisher information, selects parameters containing relatively important information from the forgotten data (210), and initializes the matrices (A and B) of LoRA, so that learning about information to be deleted can be performed faster and more precisely.

[0329] Meanwhile, in the present invention, a process of performing unlearning on a learned large-scale language model to which LoRA weights are applied can be performed.

[0330] The control unit (150) can perform unlearning on a large-scale language model (150) trained based on the results of LoRA weight initialization. That is, after the initialization process is completed, unlearning can be performed on a large-scale language model (150) to which the initialized LoRA weights are applied (or the weights of the LoRA adapter are applied).

[0331] In this case, while unlearning is performed on a large-scale language model (150), the parameters of the pre-trained large-scale language model (150) are fixed, and only the weights of LoRA can be updated. At this time, when unlearning on the forgotten data (210), a set of maintenance data (220) containing general knowledge can be used together.

[0332] The control unit (140) can perform unlearning on a large-scale language model (150) using a preset loss function to effectively remove data to be removed (forgotten data) and increase the efficiency of data to be retained (maintained data). For example, in the present invention, the preset loss function for unlearning the large-scale language model (150) may be a final loss function using Inverted Hinge Loss (IHL). This loss function may be represented as shown in (h) of FIG. 9 .

[0333] Unlearning of a large-scale language model (150) can proceed by sampling a text corpus (or data) from each data set and minimizing the final loss function through stochastic gradient descent via backpropagation.

[0334] Specifically, as illustrated in FIG. 6, the large-scale language model (150) can perform unlearning on the forgotten data (210) using a loss function preset for unlearning (S404, see FIG. 4). At this time, the control unit (140) can perform unlearning on the large-scale language model using the loss function so as to increase the prediction probability for the maintained data (220) of the large-scale language model (150) while decreasing the prediction probability for the forgotten data (210). That is, the control unit (140) can perform unlearning on the large-scale language model using the loss function so as to efficiently remove the data to be removed (forgotten data) and maintain the knowledge of the data to be maintained (maintained data) without affecting the inference and generation capabilities of the large-scale language model (150).

[0335] As discussed above, the present invention effectively performs unlearning by reducing the prediction scores for actual tokens (or correct tokens) while increasing the prediction scores for only a small number of other tokens. To this end, the Inverted Hinge Loss used in the preset loss function of the present invention can be represented as shown in (a) of Figure 10.

[0336] First, the control unit (140) can perform unlearning on the large-scale language model (150) in a direction of reducing the prediction probability (or prediction score) for the forgotten data (210) using a loss function so that the large-scale language model (150) eliminates (or does not generate (or recognize, infer, etc.)) knowledge about the forgotten data (210).

[0337] More specifically, the control unit (140) can adjust the probability distribution of the large-scale language model (150) in a direction that reduces the prediction probability of the large-scale language model (150) for the correct token (true token) included in the forgotten data (210) so that the prediction probability of the forgotten data (210) of the large-scale language model (150) is reduced (see FIG. 6). The probability of the correct token can be represented as in (b) of FIG. 10.

[0338] Additionally, the control unit (140) may maximize the log probability for the maintained data (220) using a loss function so that the large-scale language model (150) maintains knowledge of the maintained data (220) (or maintains the ability to generate (or recognize, infer, etc.) the maintained data (220). For example, the control unit (140) may maximize the log probability for the maintained data (220) to maintain (or increase) the ability of the large-scale language model (150) to generate the maintained data (220), thereby balancing the model so that it does not forget more than necessary.

[0339] More specifically, the control unit (140) can adjust the probability distribution of the large-scale language model (150) to increase the prediction probability of the large-scale language model for the replacement token with the highest probability among all tokens excluding the true token, so that the large-scale language model maintains the ability to generate maintenance data (220) (see FIG. 6). Here, the replacement token may correspond to any token with the highest probability of replacing the true token among all tokens excluding the true token. Such a replacement token may be represented as in (c) of FIG. 10.

[0340] Additionally, all tokens are tokens excluding correct tokens from the vocabulary set of a pre-trained large-scale language model (150). For example, the vocabulary set may refer to a set of words or subword units that the model can use. For another example, all tokens may include at least one token from the maintenance data set (220).

[0341] The minimum probability difference between the correct token and the replacement token is represented as in (d) of Fig. 10, and ensuring that the loss value is limited to 0 or more can be represented as in (e) of Fig. 10. The Inverted Hinge Loss used in the preset loss function according to the present invention (see (a) of Fig. 10) can converge the loss to 0 when the probability of the correct token becomes sufficiently smaller than that of the replacement token with the highest possibility of replacement (e.g., when unlearning is complete). In this case, the case when unlearning is complete can be represented as in (f) of Fig. 10, and the case when unlearning is not yet complete can be represented as in (g) of Fig. 10.

[0342] In one embodiment, considering the probability of the correct token defined through the softmax function, the Inverted Hinge Loss derivative for the logit value of the language model for a specific word v at time t (see (a) of FIG. 11) can be represented as in (b) of FIG. 11.

[0343] Additionally, during the gradient calculation process of Inverted Hinge Loss, if unlearning is in progress (see (c) of Figure 11), the probability of the correct token can be decreased and the probability of the replacement token can be increased. At this time, since the absolute value of the gradient is greater than or equal to that of the replacement token, the probability of the correct token can decrease more quickly. Although the adjustment is made toward increasing the probability of the replacement token, the increase can be slower than that of the correct token (because the absolute value is smaller than that of the correct token). In addition, the probability of other tokens can increase slowly in proportion to the difference in probability between the correct token and the replacement token.

[0344] Furthermore, in the process of calculating the gradient of the Inverted Hinge Loss, if unlearning is completed (see (d) of Fig. 11), the loss can converge to 0 if the value for “the difference between the probability of the correct token and the probability of the replacement token + 1” is less than 0.

[0345] That is, the present invention reduces the probability of correct tokens and increases the probability of replacement tokens. Once the unlearning of a large-scale language model (150) for forgotten data (210) is completed, the loss converges to zero, stably terminating the unlearning process. This allows for efficient gradient updates without affecting the inference and generation capabilities of the large-scale language model.

[0346] Meanwhile, the present invention discussed above can be applied and utilized effectively in a variety of industries and services. In one embodiment, the present invention can be applied and utilized effectively in at least one of the following: opt-out applications, natural language generation-related services, conversational AI and chatbots, text generation AI and content creation, personalized education and language learning, social media and online platforms, harmful content filtering, medical and healthcare, finance and law, gaming, and virtual environments.

[0347] In relation to the large-scale language model unlearning method and system described above, the present invention may include, in the inference stage, a process of receiving a user input requesting deletion of specific data from a training data set used for training a large-scale language model (LLM) (S1210), a process of specifying data corresponding to the specific data in the training data set as forgotten data based on the received user input and specifying the remaining data excluding the forgotten data as maintained data (S1220), a process of comparing the forgotten data and the maintained data and specifying specific parameters having high importance for the forgotten data among the parameters of the large-scale language model (S1230), a process of initializing the weights of LoRA (Low-Rank Adaptation) based on the specific parameters having high importance for the forgotten data (S1240), and a process of performing unlearning on the large-scale language model to which the weights of LoRA are applied (S1250).

[0348] As illustrated in FIG. 13, the control unit (140) can receive, from the user terminal (10), a user input (e.g., “Please delete the data I request...”) requesting deletion of specific data (1400) from the training data set (200) used for training the large-scale language model (150). This user input can be received in various ways. For example, it can be received in various ways, such as the user selecting specific data through a user interface, the user inputting a text query regarding specific data to be deleted, the user inputting a voice query regarding specific data to be deleted, etc., and is not necessarily limited to the cases described above.

[0349] The control unit (140) may analyze specific data corresponding to the received user input, and based on the analyzed result, may specify data corresponding to specific data (1400) among the learning data set (200) used for learning a large-scale language model (150) as forgotten data (210). In addition, the control unit (140) may specify the remaining data excluding the forgotten data (210) as maintained data (220).

[0350] Once the forgetting data (210) and the maintenance data (220) are specified, the control unit (140) can measure a Fisher information matrix for the forgetting data (210) and a Fisher information matrix for the maintenance data (220) for each parameter of the large-scale language model (150) using the forgetting data (210) and the maintenance data (220). Then, the control unit (140) can obtain a Fisher information matrix for the forgetting data (210) measured using the forgetting data (210) and a Fisher information matrix for the maintenance data (220) measured using the maintenance data (220) based on the measured results. Furthermore, the control unit (140) can measure (or analyze, quantify) parameter importance using the obtained Fisher information matrix for the forgetting data (210) and the Fisher information matrix for the maintenance data (220). As a result of the parameter importance measurement, the control unit (140) can specify a parameter having a high parameter importance for forgotten data and a low parameter importance for maintained data (220) as a specific parameter.

[0351] Finally, the control unit (140) can initialize the weights of LoRA (Low-Rank Adaptation) based on specific parameters with high importance for the forgotten data. The control unit (150) can perform unlearning on the large-scale language model (150) to which the LoRA weights are applied. More specifically, the control unit (140) can perform unlearning on the large-scale language model using a loss function so as to increase the prediction probability for the maintenance data (220) of the large-scale language model (150) while decreasing the prediction probability for the forgotten data (210). This can be understood as decreasing the prediction probability of the large-scale language model for the correct token included in the forgotten data so as to decrease the prediction probability of the large-scale language model for the true token, and increasing the prediction probability of the large-scale language model (150) for the replacement token with the highest probability among all tokens except the correct token, so as to decrease the prediction probability of the forgotten data (210).

[0352] Furthermore, when unlearning is completed for the forgotten data (210), it is possible to determine (or judge) whether the forgotten data (210) of the large-scale language model (150) has been forgotten. The determination of whether the forgotten data (210) has been forgotten can be determined based on various criteria. For example, whether the forgotten data (210) has been forgotten can be determined independently by the administrator or user of the unlearning system (100) or the artificial intelligence model (160) based on preset criteria (e.g., user history information, user personal information (e.g., name, address, phone number, email, etc.), criteria set in relation to copyright infringement elements, etc.).

[0353] In one embodiment, to determine whether forgotten data (210) has been forgotten, a query related to the forgotten data (210) may be input into a large-scale language model (150). When unlearning of knowledge about the forgotten data (210) is complete, the large-scale language model (150) may provide a response to the user indicating that unlearning of the forgotten data (210) has been completed.

[0354] In this way, the unlearning system (100) can delete knowledge about specific data (1400) that the user has requested to be deleted, while retaining knowledge about data that the user wishes to retain. In other words, the unlearning system (100) can delete knowledge about specific data (1400) corresponding to the user's deletion request from the learning results of the large-scale language model (150), while efficiently retaining knowledge about data (220) that must be retained from the learning results.

[0355] As discussed above, the large-scale language model optimization method and system according to the present invention, and the control method for the large-scale language model optimization system, enable training a large-scale language model on a training data set based on a preset attention mechanism. This allows the large-scale language model to efficiently process long sentences or text, thereby significantly reducing memory usage and improving computational efficiency.

[0356] Furthermore, according to the large-scale language model optimization method and system according to the present invention, and the control method of the large-scale language model optimization system, the large-scale language model can be trained to process text sequences using various methods (or techniques). This allows the present invention to save memory and computational resources when processing long sentences or texts, thereby constructing a memory-efficient large-scale language model. In this case, the constructed large-scale language model can exhibit high inference capabilities and performance in various inference tasks (e.g., coding, mathematics, science, logical reasoning, etc.).

[0357] Furthermore, as discussed above, the large-scale language model optimization method and system according to the present invention, and the control method of the large-scale language model optimization system, can measure Fisher information for each of the forgotten data and the maintained data, and perform unlearning of a large-scale language model based on the measured results. Therefore, the present invention can effectively remove knowledge about the forgotten data while maintaining knowledge about the maintained data by selecting and preferentially adjusting only the parameters (or weights) that are relatively important for the forgotten data. Through this, the present invention can provide the effect of maintaining the existing performance of the model by simultaneously unlearning the forgotten data more quickly and minimizing the impact on the maintained data.

[0358] Furthermore, the large-scale language model optimization method and system according to the present invention, and the control method of the large-scale language model optimization system, can analyze the relative importance of forgotten data and retained data, and perform an initialization process that selectively adjusts parameters important only to forgotten data based on the analyzed results. Through this, the present invention can reduce unnecessary computations during the unlearning process, lower computational costs, and efficiently perform unlearning in terms of time and resources without retraining the entire model.

[0359] Furthermore, according to the large-scale language model optimization method and system according to the present invention, and the control method of the large-scale language model optimization system, by focusing the gradient update only on the minimum viable replacements for the correct token, the data to be removed can be effectively removed while maintaining the language generation ability and inference performance of the existing model. Through this, the present invention provides an environment in which performance degradation that may occur during the information deletion process of the model can be minimized and privacy protection and copyright issues can be resolved. In other words, the present invention prevents unnecessary loss diffusion and performs effective unlearning by adjusting only the minimum viable replacement tokens, thereby maintaining the model's natural sentence generation ability even in situations where unlearning must be performed on specific data.

[0360] In this way, the present invention prioritizes adjusting important parameters in forgotten data and minimizes information loss in retained data. This allows for the removal of specific data without retraining the entire model, significantly reducing computational and operational costs. Furthermore, the present invention enables the rapid unlearning of data desired by the user, thereby contributing to data protection law compliance, reduced corporate operating costs, and enhanced AI system reliability. It can be utilized effectively in a variety of industries and services.

[0361] Meanwhile, the present invention discussed above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a quantum processor and quantum memory based on qubits, and may include software and hardware interfaces optimized for quantum computing.

[0362] Quantum processors in quantum computers utilize qubits (qubits) to efficiently perform complex calculations, such as parallel computation, quantum entanglement, and quantum superposition, that classical computers' binary bits cannot perform. Quantum processors process data using quantum gates, and can provide exponential speedups for certain problems.

[0363] Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a medium (or recording medium) that can be read by the computer.

[0364] Furthermore, the present invention discussed above can be implemented as computer-readable code or instructions on a program-recorded medium. In other words, the present invention can be provided in the form of a program.

[0365] Meanwhile, computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0366] Furthermore, the computer-readable medium may include a storage device and may be a server or cloud storage device accessible via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage device via wired or wireless communication.

[0367] A computer program can reach the system (100) via a variety of suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product tangibly embodying a computer program. The transmission mechanism may be a signal configured to reliably transmit a computer program over air or through an electrical connection. The system (100) may propagate or transmit a computer program as a computer data signal.

[0368] Furthermore, references to 'computer-readable storage media', 'computer program product', 'a computer program tangibly embodied', or the like, or to 'controller', 'computer', 'processor', etc., should be understood to include computers having various architectures, such as single / multiprocessor architectures and sequential (Von Neumann) / parallel architectures, as well as specialized circuits such as field-programmable gate arrays (FPGAs), application-specific circuits (ASICs), signal processing devices, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as the programmable contents of hardware devices, whether instructions for a processor or configuration settings for fixed-function devices, gate arrays, or programmable logic devices.

[0369] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, i.e., a CPU (Central Processing Unit), and there is no particular limitation on its type.

[0370] Meanwhile, the above detailed description should not be construed as limiting in any respect and should be considered illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A computerized method comprising: A step of specifying forgetting data and maintenance data for a previously trained large-scale language model (LLM) from a training data set stored in memory; A step of comparing the forgotten data and the maintained data to identify a specific parameter among the parameters of the large-scale language model that has high importance for the forgotten data; A step of initializing the weights of LoRA (Low-Rank Adaptation) based on the specific parameters having high importance for the above-mentioned forgotten data; and A method for unlearning a large-scale language model, characterized by comprising a step of performing unlearning on the large-scale language model to which the weights of the LoRA are applied.

2. In paragraph 1, A method for unlearning a large-scale language model, further comprising a step of measuring parameter importance for each of the forgotten data and the maintained data using a Fisher Information Matrix.

3. In paragraph 2, The above Fisher information matrix is, An unlearning method for a large-scale language model, characterized in that at least one parameter of the large-scale language model is a measure indicating whether the parameter is important for a text sample included in the forgotten data or the maintained data.

4. In paragraph 3, A parameter with a high absolute value of the slope for the above forgetting data is determined to have a relatively high importance in the above forgetting data, An unlearning method for a large-scale language model, characterized in that a parameter having a high absolute value of a slope for the above-mentioned maintenance data is specified to have a relatively high importance in the above-mentioned maintenance data.

5. In paragraph 2, In the above measuring step, For each parameter of the large-scale language model, using the forget data and the maintain data, a Fisher information matrix for the forget data and a Fisher information matrix for the maintain data are measured, A large-scale language model unlearning method characterized in that the parameter importance is measured using the Fisher information matrix for the forgotten data and the Fisher information matrix for the maintained data.

6. In paragraph 5, The importance of the above parameters is, A method for unlearning a large-scale language model, characterized in that the relative Fisher information matrix between the Fisher information matrix for the above-mentioned forgotten data and the Fisher information matrix for the above-mentioned maintained data is measured.

7. In paragraph 4, The above relative Fisher information matrix is, An unlearning method for a large-scale language model, characterized in that it is calculated using a Fisher information matrix for the above-mentioned forgotten data and a Fisher information matrix for the above-mentioned maintained data.

8. In paragraph 7, A method for unlearning a large-scale language model, characterized in that a parameter having high parameter importance for the forgotten data is specified as the specific parameter based on the relative Fisher information matrix.

9. In paragraph 8, The specificity of the above specific parameters is, A method for unlearning a large-scale language model, characterized in that a parameter having a high importance for the above-mentioned forgotten data and a low importance for the above-mentioned maintained data is specified as the specific parameter.

10. In paragraph 1, The above initialization is, A large-scale language model unlearning method characterized in that it is performed by calculating the relative importance of parameters for each of the above-mentioned forgotten data and the above-mentioned maintained data.

11. In paragraph 10, In the step of performing the above initialization, An unlearning method for a large-scale language model, characterized in that the weights of the LoRA are initialized with a focus on the specific parameters having high importance in the forgotten data.

12. In paragraph 10, A large-scale language model unlearning method characterized in that, among the above-mentioned forgotten data and the above-mentioned maintained data, important information about the above-mentioned forgotten data is specified and reflected in the above-mentioned initialization.

13. In paragraph 1, In the step of performing the above unlearning, A method for unlearning a large-scale language model, characterized in that unlearning is performed on the forgotten data using a loss function set for unlearning in the large-scale language model.

14. In paragraph 13, In the step of performing the above unlearning, A method for unlearning a large-scale language model, characterized in that unlearning is performed on the large-scale language model using the loss function so that the prediction probability for the maintained data of the large-scale language model is increased while the prediction probability for the forgotten data is decreased.

15. In paragraph 13, In the step of performing the above unlearning, To reduce the prediction probability for the forgotten data, the prediction probability of the large-scale language model for the true token included in the forgotten data is reduced, An unlearning method for a large-scale language model, characterized in that it increases the prediction probability of the large-scale language model for the replacement token having the highest probability among all tokens except the correct token.

16. In paragraph 15, The above replacement token is, An unlearning method for a large-scale language model, characterized in that, among all tokens excluding the correct answer token, a token corresponds to one token that is most likely to replace the correct answer token.

17. In paragraph 13, A method for unlearning a large-scale language model, characterized in that while unlearning is performed on the large-scale language model, the parameters of the large-scale language model are fixed and only the weights of the LoRA are updated.

18. A step of receiving a user input requesting deletion of specific data from a training data set used for training a large-scale language model (LLM); A step of specifying data corresponding to the specific data in the learning data set as forgotten data based on the received user input, and specifying the remaining data excluding the forgotten data as maintained data; A step of comparing the forgotten data and the maintained data to identify a specific parameter among the parameters of the large-scale language model that has high importance for the forgotten data; A step of initializing the weights of LoRA (Low-Rank Adaptation) based on the specific parameters having high importance for the above-mentioned forgotten data; and A control method for an unlearning system of a large-scale language model, characterized in that it includes a step of performing unlearning on the large-scale language model to which the weights of the LoRA are applied.

19. A system comprising a memory configured to store executable instructions, and one or more processors configured to perform operations by executing one or more instructions, The above system, In the training data set stored in the above memory, specify forgetting data and maintenance data for the previously trained large-scale language model (LLM), By comparing the above-mentioned forgotten data and the above-mentioned maintained data, a specific parameter having high importance for the above-mentioned forgotten data among the parameters of the large-scale language model is identified, Initialize the weights of LoRA (Low-Rank Adaptation) based on the specific parameters with high importance for the above-mentioned forgotten data, A large-scale language model unlearning system characterized by performing unlearning on the large-scale language model to which the LoRA weights are applied.

20. A program that is executed by one or more processes in an electronic device and stored in a computer-readable recording medium, The above program is, In a training data set, a step of specifying forget data and retention data for a pre-trained large-scale language model (LLM); A step of comparing the forgotten data and the maintained data to identify a specific parameter among the parameters of the large-scale language model that has high importance for the forgotten data; A step of initializing the weights of LoRA (Low-Rank Adaptation) based on the specific parameters having high importance for the above-mentioned forgotten data; and A program stored on a computer-readable recording medium, characterized in that it includes commands for performing a step of performing unlearning on the large-scale language model to which the weights of the LoRA are applied.

Images