Channel sensitivity driven large language model privacy memory forgetting method
By using a channel-sensitivity-driven approach to identify and remove privacy-sensitive parameters in large language models, the problem of privacy memory removal is solved, achieving efficient and low-cost privacy forgetting while maintaining model performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-10
AI Technical Summary
Existing large language models struggle to effectively remove ingrained privacy memories during training, leading to privacy leaks and compliance risks. Furthermore, existing privacy forgetting techniques are costly, inefficient, and unstable.
By using a channel sensitivity-driven approach, highly sensitive parameters in large language models are identified and pruned, privacy-sensitive analysis units are constructed, the importance of parameters is quantified using the Fisher information matrix, and weight pruning and reconstruction are performed using the OBS update method to form a complete model without retraining.
It achieves efficient and controllable removal of privacy memories, maintains stable model performance, reduces computational overhead, and ensures the thoroughness of privacy forgetting and the practical deployment feasibility of the model.
Smart Images
Figure CN121637564A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning model compression and privacy protection, specifically to a channel-sensitive driven method and system for forgetting privacy memories of large language models. It is applicable to the deployment of large-scale pre-trained language models in privacy-compliant scenarios, especially for application scenarios that require "forgetting" sensitive user data. Background Technology
[0002] With the widespread deployment of Large Language Models (LLMs), these models may memorize user-uploaded personal data, conversation logs, or text samples containing sensitive information during training. When these privacy memories embedded in the model parameters are triggered, it could lead to privacy leaks, the generation of unauthorized user data, and even compliance risks.
[0003] Current mainstream privacy-preserving forgetting techniques include: data deletion, which only removes samples from the training set and cannot remove fixed memories from the trained parameters; retraining, which completely retrains the model, is extremely costly and unsuitable for large models; fine-tuning overlay, which covers privacy memories with counterfactual data, but is difficult to guarantee thoroughly and verifiably; knowledge distillation and reverse training, which require auxiliary models or complex reverse reasoning, are cumbersome and unstable; and gradient backscrambling, which requires complete recording of the training process and is difficult to implement in real-world applications. All of these methods have shortcomings in terms of verifiability, cost, efficiency, and practical deployment capabilities.
[0004] On the other hand, sparsity pruning, as an important technique for model compression, can remove parameters that "contribute little to the task." However, traditional pruning aims to preserve performance or accelerate inference, but it does not address privacy-forgotten scenarios or consider highly sensitive parameters for specific samples.
[0005] Therefore, designing a privacy forgetting method that can directly affect the model parameter level, efficiently identify privacy-related parameters, does not require retraining the entire model, is verifiable, low-cost, and controllable is of great technical significance. Summary of the Invention
[0006] The purpose of this invention is to solve the problem of the difficulty in eliminating privacy memories in existing large language models. It proposes a channel-sensitive driven method for forgetting privacy memories in large language models. By identifying and removing parameters that are highly sensitive to privacy samples at the parameter level, the model no longer responds to or reproduces the privacy content, while maintaining the overall stability of the model's capabilities.
[0007] To achieve the above objectives, the present invention provides the following technical solution: a channel-sensitive driven method for privacy-preserving memory forgetting in large language models, comprising the following steps:
[0008] S1. Data preprocessing: Based on the data source, privacy attributes and relevance to downstream tasks, the raw data is divided into two categories: the forgotten set and the retained set. The split datasets are cleaned to ensure that each dataset is functionally independent and covers the entire process requirements of model pruning and forgetting verification.
[0009] S2. Construction of privacy sample representation: Through the forward propagation process of the large language model, the deep representation of privacy samples is extracted, and the loss statistics of forgotten samples are constructed to provide a basis for the subsequent construction of "privacy-sensitive analysis units".
[0010] S3. Construction of Privacy-Sensitive Analysis Units: Based on the layered structure characteristics of large language models, the model is split into layers and "privacy-sensitive analysis units" are constructed along the input channel direction to achieve parameter filtering at the input channel level, laying the foundation for subsequent precise pruning;
[0011] S4. Privacy-Sensitive Unit Pruning and Reconstruction: For each privacy-sensitive analysis unit, the importance of parameters to privacy samples is quantified through the Fisher information matrix. The weight pruning and reconstruction are then achieved by combining the OBS (Optimal Brain Surgeon) update method. This removes privacy-related parameters while ensuring the general performance of the model.
[0012] S5. Pruned Model Assembly: The layer weights corresponding to all the pruned and reconstructed privacy-sensitive analysis units are assembled according to the original model structure to form a complete large language model. No fine-tuning or retraining is required throughout the process, ensuring pruning efficiency and model stability.
[0013] S6. Forgetting Validation and Performance Calibration: The pruned model is jointly evaluated using three metrics: privacy forgetting effect, general model performance, and pruning efficiency. If the preset target is not met, the pruning parameters are adjusted and re-optimized to finally obtain a forgetting model that meets the requirements.
[0014] Preferably, the data preprocessing in step S1 mainly includes dataset partitioning, data cleaning, and subset partitioning. Dataset partitioning includes reading the original data and dividing it proportionally into a forget set and a retain set using random sampling, for subsequent sensitivity modeling and forgetting validation. Data cleaning includes preprocessing the partitioned datasets, cleaning the titles, removing irrelevant title prefixes, special symbols, detecting outliers, removing samples with token lengths less than 10 or greater than 2048, samples containing more than 10% garbled characters / special symbols, and formatting, uniformly converting to a text format compatible with large language models, supplementing samples not exceeding 2048-token with model-specific padding characters, and truncating excessively long samples according to semantic integrity. Subset partitioning includes filtering valid data in each dataset and dividing it into training, validation, and test sets.
[0015] Preferably, step S2, privacy sample representation construction, mainly includes:
[0016] S2-1. Input the training set samples of the forgotten dataset into the large language model and extract the output feature matrix of each Transformer layer. ( For layer index, For the sample size, (for layer output dimensions)
[0017] S2-2. Calculate the "privacy sensitivity score" for each sample across all Transformer layers, as follows:
[0018] ,
[0019] in For the sample With preserved dataset samples in layers cosine similarity of the feature matrices This represents the number of Transformer layers.
[0020] Preferably, step S3 specifically includes:
[0021] S3-1, Model Layer Splitting: The Transformer layers of the large language model (including attention Q / K / V projection layers and fully connected layers FC1 / FC2) are split into layers, with each layer treated as an independent pruning unit, and the weight matrix of each layer is extracted. ( Input the number of channels. (Number of output channels).
[0022] S3-2, Channel-level unit partitioning: Along the input channel direction of each layer, divide the weight matrix according to the input channel index. ( Divide into privacy-sensitive analysis units Each cell contains a weight column for that input channel. The characteristic mean of forgotten samples in this channel Channel privacy sensitivity score ;
[0023] S3-3, Calculation of Channel Privacy Sensitivity Score:
[0024] ,
[0025] in (Reference value) ), The L2 norm of the weighted column. To delete the channel Subsequent hierarchical loss increment;
[0026] S3-4, Unit Priority Sorting: According to Sort in descending order. As a highly sensitive unit, As a medium-sensitive unit, For low-sensitivity units, determine the pruning priority.
[0027] Preferably, step S4 specifically includes:
[0028] S4-1, Fisher Information Matrix Calculation: Randomly sampled samples from the forgetting dataset training set, and obtained units through forward and backward propagation. Weighted column gradient Calculate the regularized Fisher matrix:
[0029] ,
[0030] The number of samples collected. This is a damping term to avoid matrix singularities;
[0031] S4-2, Parameter Importance Quantification: Calculating Weight Columns Each parameter Importance score:
[0032] , ,
[0033] Here, the diagonal elements of the Fisher matrix are represented. The lower the score, the smaller the impact of the parameters on the loss of privacy samples, and the higher the pruning priority.
[0034] S4-3. Dynamic pruning ratio setting: Prune 40%-50% for high-sensitivity units, 20%-30% for medium-sensitivity units, and 5%-10% for low-sensitivity units. Filter parameters to be pruned in ascending order of parameter importance score.
[0035] S4-4, OBS parameter reconstruction: For each parameter to be pruned Calculate OBS update volume:
[0036] ,
[0037] The update amount is added to the remaining parameters to obtain the reconstructed weight column. .
[0038] Preferably, step S5 mainly includes weight matrix integration, inter-layer compatibility verification, and forward propagation verification. Specifically, firstly, the reconstructed weight columns are integrated according to the original input channel order. Concatenate to form the complete weight matrix of layer l Preserve the original bias term Next, check the consistency of input and output dimensions between adjacent layers. If the dimensions do not match, restore the original weights of the corresponding layers and prune them again. Finally, perform forward propagation verification. Input 10-20 retained dataset validation set samples into the concatenated model to verify whether the complete forward propagation has been completed. Calculate the accuracy of the model on this sample set. If the accuracy drops by less than 10% compared to the original model, the concatenation is valid.
[0039] Preferably, step S5 mainly includes a privacy forgetting effect index, a general model performance index, and a pruning efficiency index. The parameter adjustment rules are as follows: if the privacy effect is not up to standard, increase the pruning ratio of privacy-sensitive units; if the performance loss is too great, decrease the pruning ratio of medium and low-sensitivity units; if the efficiency is not up to standard, reduce the number of samples sampled by the Fisher matrix or the scale of calibration data.
[0040] Compared with the prior art, the present invention has the following significant advantages:
[0041] 1) This invention focuses on the structurally concentrated features of privacy information in the input dimension of LLMs, aiming to overcome the limitation of traditional output-side pruning in failing to cut off sensitive paths at the source. By constructing "sensitive analysis units" at the input channel level, it accurately locates parameter regions carrying user behavior features, achieving deep privacy removal while maintaining controllable model performance. This invention solves the core challenges of LLMs, such as numerous layers, complex structures, and performance maintenance under high sparsity, and is an effective means to improve the thoroughness of forgetting.
[0042] 2) Addressing the conflict between privacy leakage and model performance in the parameter space, this invention focuses on the problem of "how to forget privacy without weakening model capabilities." The core idea is to dual-evaluate the privacy contribution and performance value of parameters, and based on this, design flexible strategies such as selective weakening and gradual suppression to avoid inadvertently damaging critical functional paths through harsh pruning, thus achieving a dynamic balance between privacy protection and model performance. This is a key technical aspect in solving the problem of insufficient robustness to forgetting parameters.
[0043] 3) To address the challenges of extremely high computational costs and scalability of second-order derivatives (such as Hessian) in LLMs, this invention utilizes the property of "first-order gradient approximating second-order curvature" of the Fisher information matrix to construct a lightweight sensitivity accumulation mechanism that can be iteratively updated during the pruning process. This method significantly reduces computational overhead while maintaining the accuracy of parameter importance assessment, making privacy forgetting in LLMs with a large number of parameters scalable and practically deployable, and is a core technology for improving forgetting efficiency. Attached Figure Description
[0044] Figure 1This is a schematic diagram illustrating the steps of the channel-sensitivity driven large language model privacy memory forgetting method of the present invention. Detailed Implementation
[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] Please see Figure 1 This invention provides a technical solution: a channel-sensitive driven method for privacy-preserving memory forgetting in large language models, such as... Figure 1 As shown, it includes the following steps:
[0047] 1) Data Preprocessing: The ml-100k dataset contains 100,000 rating records, 943 users, and 1682 movies. Data cleaning is required to remove noise, including invalid content, duplicate content, and insufficient interaction. The cleaned data is then subjected to length serialization. The specific process is as follows:
[0048] 1-1) We perform data cleaning on the ml-100k dataset, including column renaming, outlier detection, data deduplication, and missing data handling; the noise includes spelling errors, information asymmetry, and content duplication.
[0049] 1-2) Length serialization processing: The cleaned data is processed to a fixed length to conform to the model input data.
[0050] 2) Construction of privacy-preserving sample representations: After data preprocessing, data conforming to the model is obtained. Through the forward propagation process of the large language model, the training set samples of the forgotten dataset are input into the large language model to extract the output feature matrix of each Transformer layer. ( For layer index, For the sample size, (where the layer output dimension is used), calculate the "privacy-sensitive score" for each sample across all Transformer layers, as shown below:
[0051] ,
[0052] in For the sample With preserved dataset samples in layers cosine similarity of the feature matrices The number of Transformer layers;
[0053] 3) Construction of privacy-sensitive analysis units: Based on the layer structure characteristics of large language models, the model is split into layers, and every 128 columns along the input channel direction constitute a "privacy-sensitive analysis unit" to achieve parameter filtering at the input channel level, laying the foundation for subsequent precise pruning;
[0054] 4) Privacy-Sensitive Unit Pruning and Reconstruction: For each privacy-sensitive analysis unit, the importance of parameters to privacy samples is quantified using the Fisher information matrix. Weight pruning and reconstruction are then implemented using the OBS (Optimal Brain Surgeon) update method. This removes privacy-related parameters while ensuring the model's general performance. The specific process is as follows:
[0055] 4-1) Fisher Information Matrix Calculation: Randomly sample samples from the training set of the forgotten dataset, and obtain units through forward and backward propagation. Weighted column gradient Calculate the regularized Fisher matrix:
[0056] ,
[0057] The number of samples collected. This is a damping term to avoid matrix singularities;
[0058] 4-2) Parameter Importance Quantification: Calculate the weight column Each parameter Importance score:
[0059] , ,
[0060] Here, the diagonal elements of the Fisher matrix are represented. The lower the score, the smaller the impact of the parameters on the loss of privacy samples, and the higher the pruning priority.
[0061] 4-3) Dynamic pruning ratio setting: prune 40%-50% for high-sensitivity units, 20%-30% for medium-sensitivity units, and 5%-10% for low-sensitivity units. Filter the parameters to be pruned in ascending order of parameter importance score.
[0062] 4-4) OBS parameter reconstruction: For each parameter to be pruned Calculate OBS update volume:
[0063] ,
[0064] The update amount is added to the remaining parameters to obtain the reconstructed weight column. .
[0065] 5) Pruned Model Assembly: The layer weights corresponding to all pruned and reconstructed privacy-sensitive analysis units are assembled according to the original model structure to form a complete large language model. No fine-tuning or retraining is required throughout the process, ensuring pruning efficiency and model stability. Details are as follows:
[0066] 5-1) Weight Matrix Integration: Following the original input channel order, the reconstructed weight columns are... Concatenate to form the complete weight matrix of layer l Preserve the original bias term ;
[0067] 5-2) Inter-layer compatibility check: Check the consistency of input and output dimensions between adjacent layers, specifically as follows:
[0068] ,
[0069] in, For the output dimension of the previous layer, This is the input dimension for the next layer. If the dimension does not match, the original weights of the corresponding layer are restored and the layer is pruned again.
[0070] 5-3) Forward propagation validation: Input 10-20 validation set samples from the reserved dataset into the concatenated model to verify whether the complete forward propagation has been completed. Calculate the accuracy of the model on this sample set. If the accuracy is less than 10% of that of the original model, the concatenation is valid.
[0071] 6) Forgetting Validation and Performance Calibration: The pruned model is jointly evaluated using three metrics: privacy forgetting effect, general model performance, and pruning efficiency. If the preset goals are not met, the pruning parameters are adjusted and re-optimized to finally obtain a forgetting model that meets the requirements. The specific process is as follows:
[0072] 6-1) Privacy forgetting effect metrics: Member reasoning attack success rate less than 35% (attack model built based on Shadow Model), and perplexity increment of forgotten dataset test set samples greater than 15% (compared to the original model);
[0073] 6-2) General performance metrics of the model: The accuracy of downstream tasks on the retained dataset test set exceeds 90% of the original model. Downstream tasks include Lambada continuation, PIQA commonsense reasoning, and ARC-Easy question answering. The perplexity of the retained dataset test set is ≤ 1.1 times that of the original model.
[0074] 6-3) Pruning efficiency metrics: Total pruning time per GPU (NVIDIA A100 80GB) is less than 8 hours, and the model compression rate is at least 40%.
Claims
1. A method for forgetting private memories of a large language model driven by channel sensitivity, characterized in that, Comprising the following steps: S1, data preprocessing: according to the data source, privacy attribute and downstream task correlation, the original data is divided into two types of forget set and retain set, the divided data set is cleaned to ensure the function independence of each data set and cover the whole process requirements of model pruning and forget verification; S2, privacy sample representation construction: through the forward propagation process of the large language model, the deep representation of the privacy sample is extracted, and the loss statistical index of the forget sample is constructed to provide the basis for the construction of the subsequent "privacy sensitive analysis unit"; S3, privacy sensitive analysis unit construction: based on the layer structure characteristics of the large language model, the model is split by layer, and the "privacy sensitive analysis unit" is constructed along the input channel direction to realize the parameter screening of the input channel level and lay the foundation for the subsequent precise pruning; S4, privacy sensitive unit pruning and reconstruction: for each privacy sensitive analysis unit, the importance of the parameter to the privacy sample is quantified through the Fisher information matrix, and the weight pruning and reconstruction are realized by combining the OBS (Optimal Brain Surgeon) update method, which removes the privacy related parameters while ensuring the general performance of the model; S5, pruning model splicing: the layer weights of all privacy sensitive analysis units after pruning and reconstruction are spliced according to the original model structure to form a complete large language model, which does not need fine tuning or retraining throughout the process to ensure the pruning efficiency and model stability; S6, forget verification and performance calibration: the pruned model is jointly evaluated through three indicators of privacy forget effect, model general performance and pruning efficiency, if the preset target is not reached, the pruning parameters are adjusted for re-optimization, and finally the forget model meeting the requirements is obtained.
2. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: The step S1 data preprocessing comprises the following steps: S1-1, data set division: read the original data, and divide the data set into forget set and retain set by random sampling for subsequent sensitivity modeling and forget verification; S1-2, data cleaning: pre-process the divided data set to clean up the title, detect abnormal values and format; S1-2, subset division: screen the effective data in each data set and divide it into training set, validation set and test set.
3. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: In step S2, the deep representation extraction object of the privacy sample is the Transformer layer of the large language model (excluding the embedding layer and output head), which specifically includes: S2-1, for the training set samples of the forgetting data set, input the large language model to extract the output feature matrix of each Transformer layer wherein is a layer index, is a sample number, is a layer output dimension; S2-2, calculate the "privacy sensitive score" of each sample in all Transformer layers: ， wherein is a sample with the reserved dataset sample at layer cosine similarity of the feature matrix, is the number of Transformer layers.
4. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: The construction process of the privacy sensitive analysis unit in step S3 comprises: S3-1, model layer splitting: split the Transformer layer (including attention Q / K / V projection layer, full connection layer FC1 / FC2) of the large language model by layer, each layer as an independent pruning unit, extract the weight matrix of each layer is the input channel number, is the output channel number); S3-2, Channel-wise unit division: along each layer input channel direction, divide the weight matrix by input channel index ( ) dividing privacy-sensitive analysis unit , each unit contains the weight column of the input channel , the feature mean of the forgetting sample in the channel , channel privacy-sensitive score ; S3-3, channel privacy sensitive score calculation: ， wherein the reference value ), is the weighted column L2 norm, is the deletion channel loss increment after the level S3-4, cell prioritization: prioritize in descending order as high sensitive cells, as medium sensitive cells, as low sensitive cells, determine pruning priority. 5. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: The privacy sensitive unit pruning and reconstruction in step S4 comprises the following steps: S4-1, Fisher information matrix calculation: randomly sample samples from the training set of the forgetting data set, obtain the unit through forward propagation and back propagation weight column gradient , calculate the regularization Fisher matrix: ， for the number of samples, for the damping term to avoid matrix singularity; S4-2, Parameter importance quantification: Compute weight column Importance score for each parameter in the middle ， ， Where is the diagonal element of the Fisher matrix, the lower the score, the smaller the influence of the parameter on the privacy sample loss, and the higher the pruning priority; S4-3, dynamic pruning ratio setting: high-sensitive unit pruning 40%-50%, medium-sensitive unit pruning 20%-30%, low-sensitive unit pruning 5%-10%, screening the parameters to be pruned in ascending order of parameter importance score; S4-4, OBS parameter reconstruction: for each parameter to be pruned , calculate the OBS update amount: ， adding the update quantity to the remaining parameters to obtain a reconstructed weight column .
6. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: The step S5 includes: S5-1, Weight matrix integration: The reconstructed weight columns are ordered by the original input channel order to form the complete weight matrix for layer l , preserving the original bias terms ; S5-2, inter-layer compatibility verification: checking the consistency of the input and output dimensions of adjacent layers, that is ， wherein, is the output dimension of the previous layer, is the input dimension of the next layer. If the dimensions do not match, the original weights of the corresponding layer are restored and pruned again; S5-3, forward propagation verification: inputting 10-20 reserved dataset verification set samples into the spliced model to verify whether complete forward propagation is completed, calculating the accuracy of the model on the sample set, and if the accuracy decreases by less than 10% compared with the original model, the splicing is effective.
7. The passage-sensitive, driven, large language model privacy memory forgetting method of claim 1, wherein: The three types of evaluation indexes and the preset target in the step S6 include: S6-1, privacy forgetting effect index: member reasoning attack success rate ≤ 35% (based on Shadow Model to build an attack model), forgetting data set test set sample perplexity increment ≥ 15% (compared with the original model); S6-2, model general performance index: downstream task accuracy of the reserved data set test set ≥ 90% of the original model (downstream tasks include Lambada continuation, PIQA common sense reasoning, and ARC-Easy question answering), and the perplexity of the reserved data set test set ≤ 1.1 times of the original model; S6-3, pruning efficiency index: single GPU (NVIDIA A100 80GB) pruning total time is less than 8 hours, and model compression rate is at least 40%.