Large language model forgetting method fusing reward mechanism and module sparse pruning

Abstract

The invention discloses a big language model forgetting method fusing a reward mechanism and module sparse pruning, and the method comprises the steps: firstly analyzing a forgetting target of a big language model, and constructing a model forgetting pruning theory with a module as a basic unit; secondly, importance evaluation of all modules of the model under the forgotten target is calculated, structured pruning is carried out on the model according to a module positioning result, and efficient deletion of target data is achieved; and finally, finishing model fine tuning on a conventional data set. According to the method, a model pruning forgetting scheme fused with a reward mechanism is put forward for the first time, training characteristics and reasoning characteristics of a large language model are fully combined, modules highly related to forgetting targets are accurately recognized, and after pruning, two-step core operation of fine tuning of the lexical reward mechanism is introduced. The time cost and the calculation overhead of forgetting processing are reduced, the performance loss is made up, meanwhile, user data privacy is guaranteed, and high-performance stable operation of the model after forgetting is achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of large language model security alignment, and particularly relates to a large language model forgetting method fusing a reward mechanism and module sparsification pruning. BACKGROUND

[0002] In the artificial intelligence technology system, large language models have developed into core technologies with transformative significance. Through the learning of massive text data, the core capabilities in the field of natural language processing have been significantly improved, and outstanding performance has been shown in various downstream scenarios, becoming an important driving force for the application of artificial intelligence. However, the performance of large language models depends on the large-scale training corpus, which covers various types of data such as Internet text and literature. Due to the complexity of data sources and the vast amount of data, it is almost impossible to completely filter the training corpus from a technical perspective, resulting in the inevitable inclusion of potentially dangerous information, harmful content, or biased expressions in the corpus. In this context, while the model learns effective knowledge and language rules, it may also learn and generate texts containing toxic speech, illegal information, discriminatory content, or invasion of personal privacy. As the parameter size of large language models expands and their generation capabilities increase, such risks are further exacerbated. This not only weakens the reliability and security of large language models in practical applications, but also may lead to data privacy breaches and social ethical conflicts. To address the data privacy and security risks posed by large language models, a targeted legal and regulatory system has been gradually established worldwide, clearly defining users' control over personal data, one of the core rights being the right to be forgotten. According to this right, users have the right to request data processors (including large language model providers) to delete personal data related to them, and even to cover model parameters and algorithm logic generated based on these personal data, in order to achieve source protection of personal information. From a technical practice perspective, the standard solution to meet the right to be forgotten is to retrain the large language model after removing the target deletion data. However, this solution has significant limitations: the parameter size of large language models usually reaches tens of billions or even hundreds of billions, and retraining consumes extremely high computing resources, with training periods lasting for weeks or even months. The huge computing cost and time cost make it difficult for model providers to bear, especially in scenarios that require frequent deletion requests.

[0003] Recent studies have proposed model pruning forgetting techniques, the core idea of which is to accurately identify the key parameters that play a decisive role in the forgetting process, limit the model pruning to this small range of parameters, and thus achieve controllability and precision optimization of the forgetting process. The pruning technique quantifies the importance of the forgetting-related parameters of the model by designing a differentiated evaluation strategy, and then realizes the pruning effect of the parameters. The advantage of this technical path is that it can significantly reduce the computational resource overhead and improve the processing efficiency of the forgetting process while ensuring the forgetting effect. However, the existing technology relies heavily on heuristic rules or empirical strategies when identifying key parameters related to forgetting, and lacks systematic modeling of the correlation between the model and the target forgetting data. In addition, the evaluation mode of globally selecting important parameters has extremely high time and computational complexity when facing large language models with a large number of parameters, resulting in a significant decrease in the efficiency of the parameter selection process. More importantly, the performance crisis caused by pruning results in a sharp drop in the performance baseline of general tasks and a loss of practical value. Model pruning forgetting techniques achieve efficiency improvement by removing parameters, but the original general task processing capability will also be affected by pruning, making it difficult to meet the basic requirements of performance in actual application scenarios. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provide a large language model forgetting method that fuses reward mechanisms and module sparsification pruning. This method optimizes the forgetting target by utilizing the structural characteristics of large language models, and provides a more efficient and accurate solution for large language model forgetting by fusing a post-pruning reward fine-tuning scheme.

[0005] The purpose of the present application is achieved by the following technical solutions: The first aspect of the present application provides a large language model forgetting method that fuses reward mechanisms and module sparsification pruning, comprising the following steps: (1) Maximize the loss value of the model on the forgetting data set as the forgetting target, determine the attention head and filter in the large language model as the basic forgetting module unit, and construct the corresponding model forgetting pruning theory for the forgetting target; (2) Technically analyze the model forgetting pruning theory, and under the constraint of the forgetting target, calculate the gradient norm and Hessian curvature matrix of each attention head and filter module to be evaluated in the large language model, and based on the calculation results of the gradient norm and Hessian curvature matrix, construct a module importance evaluation function to quantitatively represent the correlation degree of each module with the forgetting target; (3) Calculate the importance score based on the importance evaluation function constructed in step (2) to obtain the score result, identify the key modules highly related to the forgetting target according to the score result, and perform a structured pruning operation on the key modules; (4) After completing the structured pruning, a token reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on the regular dataset, thereby improving the performance of the model in the regular task scenario, and completing the coordination of precise forgetting and performance preservation.

[0006] Specifically, the step (1) constructs a corresponding model forgetting pruning theory for the forgetting target; the specific formula of the model forgetting pruning theory is: wherein, represents a module mask, represents an empirical risk on a dataset, is an optimal model trained on an original dataset, is a forgetting dataset.

[0007] Further, the optimal model trained on the original dataset using the model forgetting pruning theory is subjected to a second-order Taylor expansion, and the specific process is: wherein, represents a full 1 vector of the module mask, is a gradient of the model with respect to the module mask, represents a Hessian curvature matrix with respect to the module mask . In order to maximize the forgetting pruning theory, the constant term is removed. Secondly, since the optimal model is considered to have reached a local minimum on the original dataset, the negative gradient on the forgetting dataset is converted into a positive gradient on the regular dataset from which the forgetting data is removed, ensuring non-negativity; further analysis is as follows: ; wherein, represents the i-th module mask, 1 indicates that the module needs to be pruned, and 0 indicates that the module needs to be retained; is a regular dataset from which the forgetting data is removed, represents a diagonalization term of the Hessian curvature matrix with respect to the module mask . In actual calculation process, the diagonalized Fisher information matrix is used to approximate it; the above function forms a comprehensive evaluation index of module importance by fusing the gradient norm term and the curvature matrix term, providing a quantitative basis for precise positioning of key forgetting modules.

[0008] Further, the step (3) specifically includes the following sub-steps: (3.1) Construct a forgetting dataset according to the user's data deletion request, and construct a corresponding regular dataset. Using the obtained importance evaluation function, calculate the gradient norm of each module on the retained dataset and the Hessian curvature matrix on the forgetting dataset, and take them as the importance scores of the modules.​​​ (3.2) Based on the module importance score obtained in step (3.1), the top 20% of modules ranked by score are subjected to structured pruning, and the parameter weights corresponding to these modules are reset to 0, thereby removing the influence of forgotten data association.

[0009] Furthermore, in step (4), a word reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on a regular dataset. Specifically, after completing the structured pruning according to step (3), a fine-tuning method incorporating a word reward mechanism is adopted to improve the model's performance in regular task scenarios. This is achieved through the following loss function: ;in, These represent the input question and the output answer, respectively, forming the basic data units for fine-tuning the large language model. The length of the data sequence. This indicates that the gradient operation operator is stopped. This represents the probability calculation function when the model parameter is θ. Given the input data, a stopping gradient operation is performed on the probability calculation result of the next word to form a reward signal for that word. This enables the model to strengthen its learning focus on word features in the regular dataset, thereby improving its performance in regular task scenarios.

[0010] A second aspect of the present invention also provides a forgetting device for large language models that integrates a reward mechanism and module sparsity pruning, the device specifically comprising the following modules: Constructing a forgetting unit module: With the goal of maximizing the model's loss value on the forgetting dataset, the attention head and filter in the large language model are determined as the basic forgetting module units. For the forgetting goal, the corresponding model forgetting pruning theory is constructed. Importance assessment module: The forgetting pruning theory of the model is technically analyzed, and under the constraint of the forgetting target, the gradient norm and Hessian curvature matrix of each attention head and filter module to be evaluated in the large language model are calculated respectively. Based on the calculation results of the gradient norm and Hessian curvature matrix, the module importance assessment function is constructed to quantitatively characterize the degree of correlation between each module and the forgetting target. Structured pruning module: The importance assessment function constructed by the importance assessment module is used to calculate the importance score and obtain the score result. Based on the score result, key modules that are highly related to the forgetting target are identified; and structured pruning operations are performed on the key modules. Fine-tuning performance recovery module: After completing structured pruning, a word reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on a regular dataset to improve the model's performance in regular task scenarios, achieving a synergy between accurate forgetting and performance preservation.

[0011] A third aspect of the present invention: an electronic device comprising a memory and a processor, wherein the memory is coupled to the processor; the memory is used to store program data, and the processor is used to execute the program data to implement the large language model forgetting method that combines the fusion reward mechanism and module sparsity pruning.

[0012] A fourth aspect of the present invention: a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the large language model forgetting method that combines the fusion reward mechanism and module sparsity pruning.

[0013] The beneficial effects of this invention are as follows: By integrating a reward mechanism and module sparsity pruning, a modular forgetting method for large language models is proposed. This method can accurately locate key model modules related to the forgetting task, thereby significantly improving the efficiency and effectiveness of large language models in forgetting tasks. The method locates key modules through module importance evaluation, fully combining the training and inference characteristics of the large model and considering internal structural relationships to ensure accurate identification of modules related to the data to be forgotten. Based on this, structured pruning techniques are used to remove the correlation influence of the target data, and a lexical reward mechanism is simultaneously incorporated into regular training for fine-tuning. This reduces the time cost and computational overhead of forgetting processing, compensates for performance losses, protects user data privacy, and achieves high-performance and stable operation of the model after forgetting. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall process of the large language model forgetting method that integrates the reward mechanism and modular pruning of the present invention; Figure 2 This is a schematic diagram illustrating the implementation process of the fine-tuning scheme for the fusion word reward mechanism after model pruning involved in this invention. Detailed Implementation

[0015] This invention aims to provide an efficient and effective forgetting method for large language models. Its core technology relies on the structural characteristics of large model modules, deeply analyzing and deriving the important forgetting theory of maximizing loss on forgotten data, and mastering the module importance evaluation function. By calculating the importance of modules through forgotten data and regular data, structural pruning techniques are used to achieve efficient, accurate, and controllable forgetting. Simultaneously, a fusion reward mechanism is used to fine-tune the pruned model, preserving its high performance in various downstream natural language processing tasks.

[0016] The present invention will now be described clearly and completely with reference to the accompanying drawings, so that those skilled in the art can understand the technical content of the present invention.

[0017] like Figure 1As shown, this invention constructs a comprehensive modular sparse pruning framework for large language models. This framework derives an importance evaluation function through deep analysis of large model forgetting theory, and implements structured pruning accordingly. Finally, it achieves an efficient, accurate, and balanced model forgetting mechanism through a fine-tuning scheme based on a lexical reward mechanism. Specifically, it includes the following steps: Step 1: The forgetting direction of large language models can be summarized as maximizing the model's loss value on the forgotten dataset to ensure the model's effectiveness in deleting forgotten data. Using the attention head and filter of the large language model as the basic forgetting module unit, corresponding model forgetting pruning mechanisms are constructed for the two different forgetting objectives mentioned above.

[0018] In this embodiment, the goal of large language model data forgetting is to maximize the empirical loss on the forgotten dataset. The optimal model trained on the original full dataset... Starting from this point, the model parameters are optimized based on the forgetting objective to obtain the forgetting model. : ;in, This represents the empirical risk within a specified dataset. For the forgotten dataset, These represent the input question and the output answer, respectively, forming the basic data units for the fine-tuning process of the large language model. The loss function of the model. The model is updated after forgetting. Maximizing the empirical loss on forgotten data leads to a significant performance drop on forgotten data. However, optimizing the model using the above methods requires a full data update, consuming substantial resources and negatively impacting the model's performance on general tasks. Therefore, this paper focuses on accurately locating key modules during the forgetting process, using attention heads and filters from large language models as basic forgetting module units. Thus, the forgetting objective is transformed into a model forgetting theory centered on these basic forgetting module units. The specific process is as follows: ; in, This indicates the mask of the module that needs optimization. The optimal model obtained by training on the original full dataset. This is a forgetting dataset. It should be understood that the core goal of this forgetting theory is to accurately locate the key modules related to forgetting. Based on these key modules, the model can efficiently implement pruning and forgetting, and further reduce the performance degradation caused by full model updates.

[0019] Step 2: Analyze the model forgetting pruning theory proposed in Step 1, and construct an importance evaluation function for the model modules from a theoretical derivation perspective. This evaluation function is jointly quantified by two modules: the gradient norm and the Hessian curvature matrix. In this embodiment, analyzing the model forgetting theory involves techniques such as Taylor expansion and convex optimization. The specific process is as follows: ;in, Indicates the first A module mask, where 1 indicates that the module needs to be pruned, and 0 indicates that the module needs to be retained. A vector of all ones representing the module mask. To remove forgotten data from a regular dataset, The gradient of the model with respect to the module mask. Represents relative to the module mask The Hessian curvature matrix. It should be understood that the analytical process... It is a binary mask, taking values ​​of 0 or 1. The optimal module mask is based on two modules: the gradient norm part related to the target forgotten data and the Hessian matrix part related to the normal data. The interaction of these two parts quantifies the module importance assessment.

[0020] Step 3: As Figure 2 As shown, based on the importance evaluation function constructed in step 2, importance scores are first calculated for each module, and key modules highly correlated with the forgetting target are accurately located according to the score results. Subsequently, structured pruning is performed on these key modules to remove the associated influence of the target data remaining in the model. Finally, a lexical reward mechanism is introduced to enhance the model's ability to distinguish features of regular lexical units, and the pruned model is fine-tuned on a regular dataset to achieve efficient data forgetting.

[0021] (3.1) Construct a forgotten dataset based on the user's data deletion request, and construct a corresponding regular dataset. Using the importance evaluation function obtained in step 2, calculate the Hessian matrix of the forgotten data of each module and the gradient norm on the regular dataset to obtain the importance score of each module.

[0022] It should be understood that the construction of the regular dataset needs to ensure orthogonality with the forgotten dataset, and the number of modules in the regular dataset should be the same as that in the forgotten dataset. Furthermore, the importance score focuses on the interaction relationships within the model structure; the Hessian matrix obtained from the regular dataset can be further optimized to obtain accurate module importance.

[0023] (3.2) Based on the module importance score obtained in step (3.1), the top 20% of modules ranked by score are subjected to structured pruning, and the parameter weights corresponding to these modules are reset to 0, thereby removing the influence of forgotten data association.

[0024] It should be understood that the number of pruned modules directly affects the model's forgetting effect and post-forgetting model performance. By pruning only the top 20% of modules by importance score, a balance between forgetting effectiveness and model utility is achieved, while minimizing impact on model performance. Structured pruning directly affects the model's architecture and can lead to anomalies such as dimensionality mismatch. By setting the parameters of the modules to zero, the normalized inference paradigm of the model is maintained.

[0025] (3.3) Based on the pruned model obtained in step (3.2) above, the conventional training method can be expressed as: ;in, Indicates the length of the data sequence. This represents the probability solution function when the model parameters are θ. These represent the input question and the output answer, respectively, forming the basic data units for fine-tuning the large language model. However, conventional training logic does not consider the importance of lexical units in typical datasets. Therefore, a fine-tuning method incorporating a lexical reward mechanism is adopted to improve the model's performance in typical task scenarios. This is specifically achieved through the following loss function: ;in, This indicates the cessation of gradient operation operators. It should be understood that, to ensure the model updates focus more on important terms in the regular dataset, the reward signal for important terms is strengthened, focusing model fine-tuning on key terms. By solving for the probability of each term in model inference, the cessation of gradient operation calculation is assisted. This only numerically strengthens the model fine-tuning algorithm without changing the direction of model updates, providing sufficient and reasonable reward signals.

[0026] This invention also provides a forgetting device for large language models that integrates reward mechanisms and module sparsity pruning, the device specifically comprising the following modules: Constructing a forgetting unit module: With the goal of maximizing the model's loss value on the forgetting dataset, the attention head and filter in the large language model are determined as the basic forgetting module units. For the forgetting goal, the corresponding model forgetting pruning theory is constructed. Importance assessment module: The forgetting pruning theory of the model is technically analyzed, and under the constraint of the forgetting target, the gradient norm and Hessian curvature matrix of each attention head and filter module to be evaluated in the large language model are calculated respectively. Based on the calculation results of the gradient norm and Hessian curvature matrix, the module importance assessment function is constructed to quantitatively characterize the degree of correlation between each module and the forgetting target. Structured pruning module: The importance assessment function constructed by the importance assessment module is used to calculate the importance score and obtain the score result. Based on the score result, key modules that are highly related to the forgetting target are identified; and structured pruning operations are performed on the key modules. Fine-tuning performance recovery module: After completing structured pruning, a word reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on a regular dataset to improve the model's performance in regular task scenarios, achieving a synergy between accurate forgetting and performance preservation.

[0027] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0028] This invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements the large language model forgetting method that combines the fusion reward mechanism and module sparsity pruning as described in the above embodiments.

[0029] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0030] Example 1: To verify the effectiveness of the proposed method, this invention constructed different forgetting request datasets and a regular dataset based on publicly available forgetting learning benchmark datasets on the internet. The benchmark dataset contains biographical information on 200 different individuals, with 20 question-answer pairs corresponding to each author. During construction, the forgetting data primarily originated from forgetting requests actively submitted by the authors, encompassing their entire biographical information; the regular dataset integrated information from Wiki datasets, other authors' biographies, and general world knowledge. In the experimental setup, the forgetting data was divided into three levels based on forgetting difficulty: containing biographical information for 2, 10, and 20 individuals, respectively. To ensure experimental balance, the forgetting data at each difficulty level maintained a similar scale to the regular dataset. In the method implementation phase, the gradient norm and Hessian matrix were first calculated for different datasets to evaluate the importance of each module in the model. Subsequently, the top 20% of modules by importance were pruned, and the biographical information of the remaining authors in the regular dataset was used as the fine-tuning dataset for incorporating lexical rewards. In terms of evaluation, this invention employs a dual-indicator approach to comprehensively assess the forgetting effect: the forgetting effect is evaluated by calculating the generation probability and Rouge score of author biographies in the forgotten data; simultaneously, the model performance retention is evaluated by calculating the generation probability and Rouge score of world knowledge and other author biographies. The effectiveness of the forgetting method is ultimately judged based on the combined performance of these two aspects, as detailed in Table 1 below.

[0031] Table 1 ; Table 1 shows the effects of various methods after forgetting the biographical information of 10 publicly available individuals. Retraining specifically refers to fully retraining the model after removing the data to be forgotten; this method is the benchmark for evaluating the performance of data forgetting methods in this field. However, due to limitations in training costs and computational resources, retraining is usually not feasible in practical applications. Therefore, the ideal data forgetting method in this field should be as close as possible to the performance of retraining in all performance dimensions. Currently, gradient ascent, gradient discretization, and negative preference optimization are the mainstream lightweight data forgetting methods in this field. While these lightweight methods can avoid the high resource consumption problem of retraining, in practical applications, they often struggle to simultaneously achieve both effective data forgetting and maintain the original model performance. Gradient ascent and negative preference optimization are prone to over-forgetting, while gradient discretization, although similar to the solution of this invention in terms of forgetting effect, performs poorly in terms of model performance retention. Therefore, this invention focuses on balancing these two aspects, significantly weakening the memory traces of the target data while minimizing performance degradation.

[0032] This invention is not limited to the embodiments described above. All other embodiments obtained by those skilled in the art without creative effort, using the same or similar methods as the embodiments described above, are within the protection scope of this invention.

Claims

1. A forgetting method for large language models that integrates reward mechanisms and module sparsity pruning, characterized in that, The method specifically includes the following steps: (1) Taking the maximization of the model's loss value on the forgotten dataset as the forgetting objective, the attention head and filter in the large language model are determined as the basic forgetting module units, and the corresponding model forgetting pruning theory is constructed for the forgetting objective; (2) Perform technical analysis on the forgetting pruning theory of the model, and under the constraint of the forgetting target, calculate the gradient norm and Hessian curvature matrix of each attention head and filter module to be evaluated in the large language model. Based on the calculation results of the gradient norm and Hessian curvature matrix, construct a module importance evaluation function to quantitatively characterize the degree of association between each module and the forgetting target. (3) Calculate the importance score by combining the importance assessment function constructed in step (2), identify the key modules that are highly related to the forgetting target based on the score results, and perform structured pruning operation on the key modules; (4) After completing the structured pruning, a word reward mechanism is introduced, and the pruned model is fine-tuned on a regular dataset to improve the model’s performance in regular task scenarios and achieve the synergy between accurate forgetting and performance preservation.

2. The forgetting method for large language models that integrates reward mechanisms and module sparse pruning as described in claim 1, characterized in that, In step (1), a corresponding model forgetting pruning theory is constructed for the forgetting target; the specific formula of the model forgetting pruning theory is expressed as: ; in, Indicates the module mask. This represents the empirical risk on the dataset. The optimal model obtained by training on the original dataset. This is a forgotten dataset.

3. The forgetting method for large language models that integrates reward mechanisms and module sparse pruning as described in claim 2, characterized in that, The optimal model obtained by training the model on the original dataset using the forgetting pruning theory. The second-order Taylor expansion is performed as follows: ; in, A vector of all ones representing the module mask. The gradient of the model with respect to the module mask. Represents relative to the module mask The Hessian curvature matrix; to maximize the forgetting pruning theory, the constant term is removed; secondly, since the optimal model is considered to have reached a local minimum on the original dataset, the negative gradient on the forgotten dataset is transformed into a positive gradient on the regular dataset after removing the forgotten data, ensuring non-negativity; further analysis is as follows: ;in, This represents the mask of the i-th module; 1 indicates that the module needs to be pruned, while 0 indicates that the module needs to be retained. To remove forgotten data from a regular dataset, Represents relative to the module mask In actual calculations, the diagonalized term of the Hessian curvature matrix is ​​approximated by a diagonalized Fisher information matrix. The above function integrates the gradient norm term and the curvature matrix term to form a comprehensive evaluation index of module importance, providing a quantitative basis for the accurate location of key forgetting modules.

4. The forgetting method for large language models that integrates reward mechanisms and module sparsity pruning as described in claim 1, characterized in that, Step (3) specifically includes the following sub-steps: (3.1) Construct a forgotten dataset based on the user's data deletion request, and construct a corresponding regular dataset. Using the obtained importance evaluation function, calculate the gradient norm of each module on the retained dataset and the Hessian curvature matrix on the forgotten dataset, and use them as the importance score of each module. (3.2) Based on the module importance score obtained in step (3.1), the top 20% of modules ranked by score are subjected to structured pruning, and the parameter weights corresponding to these modules are reset to 0, thereby removing the influence of forgotten data association.

5. The forgetting method for large language models that integrates reward mechanisms and module sparse pruning as described in claim 1, characterized in that, In step (4), a word reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on a regular dataset. Specifically, after completing the structured pruning in step (3), a fine-tuning method incorporating the word reward mechanism is adopted to improve the model's performance in regular task scenarios. This is achieved through the following loss function: ;in, These represent the input question and the output answer, respectively, forming the basic data units for fine-tuning the large language model. The length of the data sequence. This indicates that the gradient operation operator is stopped. This represents the probability calculation function when the model parameter is θ. Given the input data, a stopping gradient operation is performed on the probability calculation result of the next word to form a reward signal for that word. This enables the model to strengthen its learning focus on word features in the regular dataset, thereby improving its performance in regular task scenarios.

6. An apparatus for a forgetting method for large language models that integrates a reward mechanism and modular sparse pruning according to any one of claims 1-5, characterized in that, The device specifically includes the following modules: Constructing a forgetting unit module: With the goal of maximizing the model's loss value on the forgetting dataset, the attention head and filter in the large language model are determined as the basic forgetting module units. For the forgetting goal, the corresponding model forgetting pruning theory is constructed. Importance assessment module: The forgetting pruning theory of the model is technically analyzed, and under the constraint of the forgetting target, the gradient norm and Hessian curvature matrix of each attention head and filter module to be evaluated in the large language model are calculated respectively. Based on the calculation results of the gradient norm and Hessian curvature matrix, the module importance assessment function is constructed to quantitatively characterize the degree of correlation between each module and the forgetting target. Structured pruning module: The importance assessment function constructed by the importance assessment module is used to calculate the importance score and obtain the score result. Based on the score result, key modules that are highly related to the forgetting target are identified; and structured pruning operations are performed on the key modules. Fine-tuning performance recovery module: After completing structured pruning, a word reward mechanism is introduced, and fine-tuning optimization is performed on the pruned model on a regular dataset to improve the model's performance in regular task scenarios, achieving a synergy between accurate forgetting and performance preservation.

7. An electronic device comprising a memory and a processor, wherein, The memory is coupled to the processor; characterized in that the memory is used to store program data, and the processor is used to execute the program data to implement the large language model forgetting method that combines the fusion reward mechanism and module sparsity pruning as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the large language model forgetting method that combines the fusion reward mechanism and module sparsity pruning as described in any one of claims 1-5.

Images