A large language model machine forgetting application method based on distribution guidance

Abstract

The application relates to a large language model machine forgetting application method based on distribution guidance, which comprises the following steps: obtaining a natural language query from a user terminal or an application system, wherein the natural language query is a text composed of a variable-length sequence; taking the natural language query as an initial prefix sequence, and iteratively executing the following steps according to time steps: splicing the initial prefix sequence and a reply token updated in the last step into a current conditional context, inputting the current conditional context into a trained target model to calculate a conditional probability distribution on a word table, selecting a next token according to the conditional probability distribution by adopting a preset sampling strategy, and appending the next token to the end of a sequence of the reply token updated in the last step to serve as a reply token updated in the current step; repeating the above steps until a termination condition is met, and obtaining a reply text; and outputting the reply text to the user terminal or the application system as a response to the natural language query. The method can realize the coordination and unification of directional forgetting and model utility without relying on real reserved data.

Description

Technical Field

[0001] This invention belongs to the field of security and compliance technology of large language models, specifically involving a machine forgetting application method for large language models based on distributed guidance. Background Technology

[0002] Large Language Models (LLMs) typically acquire general language understanding and generation capabilities through pre-training and alignment training on massive amounts of internet text, user-generated content, and multi-source heterogeneous corpora. However, due to the complexity and sheer scale of the training data sources, it inevitably contains personal privacy information, copyrighted content, and sensitive factual knowledge. As these models are widely deployed in scenarios such as search, question answering, writing, customer service, and decision support, data compliance risks and security vulnerabilities increase significantly. To meet regulatory compliance requirements and reduce the risk of models remembering and reproducing sensitive content, machine forgetting techniques have been proposed. These techniques aim to significantly reduce the model's ability to remember and generate specific data to be forgotten or target knowledge without complete retraining, while maintaining the model's general capabilities and generation quality across non-target data distributions as much as possible.

[0003] Existing machine forgetting methods mainly fall into two categories: one relies on retained data to constrain the model's update direction, while the other attempts to forget data provided only. In practical applications, the original training corpus is often unavailable due to privacy, copyright, or storage limitations; therefore, forgetting with unretained data is more practical. For scenarios with unretained data, researchers have proposed forgetting training through loss adjustment, self-distillation, or negative preference optimization. Furthermore, gradient ascent and its variants (such as weighted gradient ascent) are widely used as baseline methods.

[0004] However, the aforementioned existing technologies all face a common problem when dealing with forgetting data without retention: because they rely solely on the data to be forgotten to drive model parameter updates and lack retained data as a reference, the model is prone to severe drift in the overall output distribution during forgetting training. Furthermore, it struggles to distinguish between target knowledge that should be deleted and general language knowledge that should be retained. This leads to a situation where, as the scale of forgetting increases, existing methods either suffer from insufficient constraints, allowing the knowledge to be forgotten to be recovered (incomplete forgetting), or excessive forgetting destroys a large amount of non-target knowledge, causing a sharp decline in the model's generality. This contradiction has not yet been effectively resolved in existing technologies, particularly lacking a technical solution that can simultaneously suppress distribution drift and precisely control the scope of forgetting without using any retained data. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides a distributed-guided method for applying machine forgetting to large language models. This invention provides a method for applying machine forgetting based on a distributed guided large language model, including: S1: Obtain a natural language query from a user terminal or application system, wherein the natural language query is text composed of a variable-length sequence; S2: Using the natural language query as the initial prefix sequence, iterate step by step: concatenate the initial prefix sequence with the updated response tokens from the previous step to form the current conditional context, input it into the trained target model to calculate the conditional probability distribution on the vocabulary, wherein the target model is obtained through a distribution-guided machine forgetting training method under no-retention data conditions. Select the next token according to the conditional probability distribution using a preset sampling strategy, and append the token to the end of the sequence of the updated response tokens from the previous step as the updated response tokens for the current step; repeat until the termination condition is met to obtain the response text; S3: Output the reply text to the user terminal or the application system as a response to the natural language query.

[0006] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the problem that existing machine forgetting methods struggle to balance the degree of forgetting and the model's generality when dealing with forgetting unretained data, this invention provides a distributed guided machine forgetting application method for large language models. By deploying the target model obtained through distributed guided machine forgetting training under unretained data conditions in real-world application scenarios, during the interaction between user queries and model responses, the model effectively avoids outputting specific factual details corresponding to the knowledge to be forgotten when outputting response text, thereby reducing privacy leaks and content compliance risks. At the same time, it maintains the ability to provide natural and fluent responses to general knowledge to ensure the quality of model services. Without relying on real retained data, this invention achieves a synergistic unity between targeted forgetting and model utility, significantly improving the practicality and reliability of machine forgetting methods in privacy-compliant scenarios. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating the machine forgetting application method based on distributed guidance for large language models provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the training process of the target model provided in an embodiment of the present invention; Figure 3 This is an example diagram of the training process of the target model provided in an embodiment of the present invention; Figure 4This invention provides a model for forgetting scenarios using retained data, as provided in this embodiment. During the forgetting process, its distribution on the data to be forgotten and the retained data relative to the reference model. Simulation plot of KL divergence variation; Figure 5 This invention provides an embodiment of the method for training on the data to be forgotten and the data that should have been retained, without using retained data in a forgetting scenario, and relying solely on the data to be forgotten, as shown by the distribution of the model relative to the reference model. Simulation plot of KL divergence variation; Figure 6 This is an example of the experimental results provided by this invention, showing the results of various mainstream forgetting methods on the TOFU dataset for three forgetting ratio tasks: forget 1, forget 5, and forget 10, without using retained data. Figure 7 This is a comparative analysis of different forgetting methods on the TOFU dataset without using retained data, provided by an embodiment of the present invention. It focuses on evaluating the performance of GA, DPO, NPO, FLAT, and Undial+NORT methods under different forgetting ratios. Figure 8 This is a comparative analysis of different forgetting methods on the TOFU dataset without using retained data, provided by an embodiment of the present invention. It focuses on evaluating the performance of GA, DPO, NPO, FLAT, and Undial+NORT methods under different forgetting ratios. Detailed Implementation

[0008] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0009] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0010] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0011] The present invention will now be described in detail with reference to the accompanying drawings, a method for applying machine forgetting based on a distributed guided large language model.

[0012] Figure 1 This is a flowchart illustrating the machine forgetting application method based on a distributed guided large language model provided in an embodiment of the present invention. Figure 1 As shown, the method includes: S1: Obtain natural language queries from user terminals or application systems. Natural language queries are text consisting of variable-length sequences.

[0013] For example, in one scenario, a user directly enters a text question in a dialog box on a webpage, mobile app, or smart device, such as "How do I reset my account password?" or "Please explain the principle of the CPO method." This text is captured by the front-end interface and transmitted to the back-end service system via API calls, thus being "acquired" as a natural language query to be processed. Alternatively, the user speaks a command through a smart speaker, in-vehicle voice assistant, etc., such as "Remind me to attend a meeting at 8 AM tomorrow." The terminal device first converts the speech signal into text using Automatic Speech Recognition (ASR) technology. This converted text ("Remind me to attend a meeting at 8 AM tomorrow") is then acquired as a "natural language query" and fed into the model.

[0014] In another scenario, the application system automatically generates a query text based on rules. For example, a data analysis platform might automatically convert a user's selection of a chart into a query text such as "Please analyze the reasons for the decline in sales of product A in North China in the third quarter," and then call the model service.

[0015] Here, a sequence refers to text being broken down into a series of discrete basic units (often called "lexicons" or "sub-words" in natural language processing) and arranged sequentially. For example, the sentence "Hello world" might be broken down into a sequence of lexicons ["you", "hello", "world"]. Variable length means that the number of lexicons contained in the sequence is not fixed. Different natural language queries correspond to different lexicon sequence lengths.

[0016] S2: Using the natural language query as the initial prefix sequence, iterate step by step: concatenate the initial prefix sequence with the updated response tokens from the previous step to form the current conditional context, and input it into the trained target model to calculate the conditional probability distribution on the vocabulary. The target model is obtained through iterative training using a distribution-guided machine forgetting training method under no-retention data conditions. Select the next token according to the conditional probability distribution using a preset sampling strategy, and append the token to the end of the sequence of updated response tokens from the previous step as the updated response token for the current step; repeat until the termination condition is met to obtain the response text.

[0017] It should be noted that the training process of the target model is not described here, only its use in the actual inference process.

[0018] Here, the target model specifically refers to a large language model that has undergone iterative training using a distribution-guided machine forgetting training method under conditions of no retained data, and has achieved machine forgetting. Essentially, the large language model is a word predictor. Given a context (i.e., the "current conditional context"), it calculates a probability distribution representing the most likely next word. In the initial iteration, the response words are empty, and the current conditional context only includes the initial prefix sequence. After inputting the current conditional context into the target model, the model outputs a vector with a length equal to the vocabulary size (e.g., 50,000 words), where each position represents the probability that the next word is that word. All probabilities sum to 1. The next word is selected from this vector using a pre-defined sampling strategy (e.g., temperature sampling, Top-K, Top-P, or greedy search), and this word is added to the current conditional context. More specifically, the current conditional context is updated at the end of the updated response word sequence from the previous step, and this updated current conditional context is input into the target model to select the next word, until a termination condition is met. Here, the termination condition includes generating an end-of-word character or the current conditional context sequence reaching its maximum length.

[0019] S3: Output the response text to the user terminal or application system as a response to the natural language query.

[0020] It should be understood that the response text will avoid outputting specific, sensitive, or privacy-related information related to the dataset to be forgotten, while maintaining the ability to respond to general knowledge.

[0021] For example, in S1, if a user asks "How do I reset my account password?", the response text could be: "To ensure your account security, the specific process for resetting your password may vary depending on the platform. We suggest you: 1. Visit the login page of the relevant website or app and find the 'Forgot Password' link; 2. Follow the instructions to complete identity verification by binding your email address or mobile phone number; 3. Set a new, strong password. If you encounter any problems during the process, the most direct and effective way is to contact the platform's official customer service for dedicated assistance." In other words, the response text does not provide any specific system's URL, menu path, or internal instructions; it only provides a general, secure operational framework.

[0022] Furthermore, for queries from data analysis platforms such as "Please analyze the reasons for the decline in sales of Product A in North China in the third quarter," the response text could be: Analyzing a product's sales decline in a specific region and period typically involves attribution analysis from the following dimensions: 1. Market and environmental factors: Examine whether new competing products emerged in the region during the quarter, whether negative public opinion events occurred, or whether overall market demand experienced a cyclical decline; 2. Sales and channel factors: Assess whether distribution channels are functioning smoothly, whether promotional activities are effective, or whether key sales partners have been lost; 3. Product and operational factors: Examine whether product quality is stable, whether there are problems in the supply chain, or whether pricing strategies have been adjusted inappropriately. It is recommended that you collect sales details, market competitor data, and channel feedback reports for that period to conduct more in-depth quantitative and qualitative analysis. In other words, the response text does not contain any specific sales data, internal report content, or confidential business information; it only outputs a structured, general analytical framework and methodology.

[0023] The training process of the target model in this invention will now be described in detail.

[0024] Figure 2 This is a schematic diagram of the training process of the target model provided in an embodiment of the present invention. Figure 3 This is an example diagram illustrating the training process of the target model provided in an embodiment of the present invention. Figure 2-3 As shown, the target model was obtained through training in the following manner: Repeat the following operations until the preset convergence condition is met, and use the model to be forgotten after the last iteration as the target model: S110: Obtain the model to be forgotten obtained after the last model training as the current model. During the initial model training, the model parameters of the model to be forgotten are the same as those of the reference model. The reference model is a pre-trained large language model, and the model to be forgotten is a large language model to be trained.

[0025] It should be noted that the reference model's parameters remain fixed during model training. During initial model training, the model parameters of the model to be forgotten are made identical to those of the reference model. Then, the target model is trained (i.e., fine-tuned) using the data to be forgotten, allowing it to "forget" the specified knowledge while maintaining consistency with the reference model in other knowledge as much as possible.

[0026] S120: Obtain the original samples of the current batch from the dataset to be forgotten, and input them into the current model and the reference model respectively, and output the first predicted probability distribution and the reference probability distribution; at the same time, obtain the pseudo samples of the current batch from the pseudo sample set generated based on the dataset to be forgotten, and input them into the current model and the reference model respectively, and output the second predicted probability distribution and the pseudo-retained reference probability distribution.

[0027] Here, the original samples of the current batch are input into the current model to obtain the first predicted probability distribution, and the original samples of the current batch are input into the reference model to obtain the reference probability distribution.

[0028] The dataset to be forgotten includes data from user interaction logs, privacy compliance review databases, or web crawler text sets. Each dataset contains at least one input-output pair, where the input is natural language text consisting of a variable-length sequence, and the output is the corresponding natural language answer. Furthermore, the pseudo-sample set is obtained as follows: each input-output pair is mapped to the embedding space of a reference model to obtain a corresponding continuous semantic vector; all continuous semantic vectors obtained from mapping all the data samples to be forgotten are treated as a set, and a Gaussian mixture model is used to fit the overall distribution of this set to obtain a distribution model for the data to be forgotten; sampling is performed within the distribution range covered by the distribution model to generate multiple pseudo-samples that do not contain factual information; all pseudo-samples are then aggregated to form a pseudo-sample set.

[0029] Here, the embedding space of the reference model can be understood as the space spanned by the representation layer inside the reference model, which maps discrete tokens to continuous numerical vectors (i.e., embedding vectors).

[0030] It should be noted that, in one case, after fitting the distribution model of the data to be forgotten (or the embedding vector distribution of the data to be forgotten) using a Gaussian mixture model, sampling is directly performed from this distribution model. Since all continuous semantic vectors are "inside" or "near" the original input-output pair in the semantic space, but are newly synthesized (generated) by the current model and are not exactly the same as any real, specific data points (sentences / lexicals) to be forgotten, it can be ensured that the specific information of the original data is not carried at the vector representation level.

[0031] In machine forgetting tasks without retained data, model updates are driven solely by the data to be forgotten. This easily leads to two core problems: first, the overall output distribution of the model deviates significantly from the original distribution, i.e., the "distribution drift" problem, which severely impairs the model's general language ability and reasoning stability; second, in the absence of clear references, the model struggles to accurately determine knowledge boundaries, i.e., the "distribution discrimination" problem, thus failing to distinguish between specific knowledge to be forgotten and general knowledge to be retained. Relying solely on a single forgetting objective aimed at reducing the retention of the target knowledge (such as negative preference optimization) can drive forgetting, but it not only exacerbates distribution drift but also fails to solve the problem of accurate discrimination, ultimately resulting in impaired model utility. Therefore, to effectively forget while firmly "anchoring" the model's general knowledge distribution and guiding the model to learn to distinguish knowledge boundaries, it is necessary to introduce additional loss mechanisms with different constraint orientations. This paper designs and jointly optimizes multiple complementary loss functions to constrain the model training process from different perspectives, collectively forming a mechanism to suppress distribution drift, solve the distribution discrimination problem, and achieve accurate and controllable forgetting.

[0032] S130: Based on the first predicted probability distribution and the reference probability distribution, calculate the degree of preference of the current model in generating the output sequence to be forgotten on the dataset to be forgotten, and use it as the first loss value.

[0033] Specifically, the first loss value is the negative preference optimization loss, whose core function is to explicitly reduce the current model at the sequence level (i.e., complete input-output pairs). Generate answers to be forgotten The preference. The expression for calculating the first loss value corresponding to each model training is: ; In the formula, These are the model parameters of the current model. It is the temperature coefficient. It is about dealing with forgotten datasets Calculate the mathematical expectation of all inputs and outputs. This represents the Sigmoid function. It is a dataset to be forgotten One of the input / output pairs, ) is the first predicted probability distribution of the current model on the dataset to be forgotten. This represents the reference probability distribution of the reference model on the dataset to be forgotten.

[0034] S140: Calculate the distribution difference between the second predicted probability distribution and the pseudo-preserved reference probability distribution as the second loss value.

[0035] Specifically, the second loss is the negative distillation loss, which does not directly act on the data to be forgotten. Instead, it constrains the target model's behavior to avoid deviating from the original model (reference model) on "pseudo-samples" derived from the distribution of the data to be forgotten, which are semantically similar but content-safe. This aims to prevent the model from experiencing a degradation in its general capabilities (i.e., distribution drift) during the forgetting process. The negative distillation loss complements the proactively driven negative preference optimization loss: one encourages the model to "forget" specific content, while the other helps the model "remember" general patterns. Simultaneously, together with the truncated divergence loss operating on the keyword meta-subspace, it constrains the distribution at different granularities (overall behavior vs. key facts), addressing the "distribution discrimination problem."

[0036] In one possible implementation, the expression for calculating the second loss value corresponding to each model training iteration is: ; In the formula, This refers to the batch size, which represents the number of pseudo-samples processed simultaneously during each model training parameter update. It represents the sequence length, indicating the number of tokens contained in each pseudo-sample. Represents cross-entropy loss, The reference model is in the first... The sample, the first The pseudo-preserved reference probability distribution at each location Indicates the current model at the th The sample, the first The second predicted probability distribution at each location.

[0037] S150: Calculate the divergence between the lexical probability distributions of the current model and the reference model on the keyword lexical set, as a third loss value.

[0038] Here, the third loss value is the truncated divergence loss. The truncated divergence loss only constrains the output probability distribution of the target model on the subset of keyword meta-information containing specific factual information, so that it does not deviate excessively from the distribution of the reference model, thereby achieving fine-grained suppression of distribution drift and guiding the model to learn to distinguish: which are specific factual distributions that need to be "forgotten" and changed, and which are general language distributions that need to be kept stable.

[0039] In one possible implementation, the expression for calculating the third loss value corresponding to each model training iteration is: ; In the formula, It is a set of keyword meta-elements. The reference model is in a given prefix Term probability distribution under certain conditions Is the current model in a given prefix The probability distribution of lexical terms under given conditions.

[0040] S160: The first loss value, the second loss value, and the third loss value are weighted and summed to obtain the current total loss value.

[0041] In one possible implementation, the expression for calculating the current total loss value for each model training iteration is: ; In the formula, These are all weighting coefficients used to balance forgetting intensity, model utility preservation, and distribution stability.

[0042] S170: Update the parameters of the current model based on the current total loss value to obtain the model to be forgotten after the current training cycle.

[0043] After obtaining the model to be forgotten after the current model training, determine whether it meets the preset convergence condition. If not, continue to the next model training. If so, end the training and obtain the target model that completes machine forgetting.

[0044] Simulation software is now used for performance verification. Figure 4 This invention provides a model for forgetting scenarios using retained data, as provided in this embodiment. During the forgetting process, its distribution on the data to be forgotten and the retained data relative to the reference model. Simulation diagram of KL divergence variation. Figure 4 In the example (a), the task of forgetting approximately 1% of the data (forget 1) is shown. It can be seen that both the distribution of the data to be forgotten and the distribution of the retained data show relatively small deviations relative to the reference model, with the overall distribution remaining largely stable. Furthermore, a certain interval still exists between the two, indicating that the model has experienced almost no significant distribution drift. In this case, the main role of the retained data is to maintain the distinguishability between the two types of data. Figure 4 In example (b), the task corresponds to forgetting approximately 5% of the data (forget 5). At this point, the KL shift between the two distributions increases, but remains within a relatively controllable range, and a clear interval still exists between the retained data and the data to be forgotten. This indicates that as the scale of forgetting increases, the retained data not only helps maintain the discriminative power of the distribution but also begins to play a role in constraining the overall distribution shift. Figure 4In the example (c), the task involves forgetting approximately 10% of the data (forget 10). At this point, the shifts of both distributions relative to the reference model increase further, but overall, there is still no drastic drift, and the distribution gap between the two remains. This demonstrates that even under large-scale forgetting conditions, introducing retained data can effectively suppress excessive shifts in the model distribution away from the reference model and maintain the ability to distinguish between the forgotten target and retained knowledge.

[0045] In summary, when retained data is involved, the model can achieve a balance between "effective forgetting" and "maintaining original capabilities," that is, maintaining the ability to distinguish distributions while controlling the overall distribution drift.

[0046] Figure 5 This invention provides an embodiment of the method for training on the data to be forgotten and the data that should have been retained, without using retained data in a forgetting scenario, and relying solely on the data to be forgotten, as shown by the distribution of the model relative to the reference model. Simulation diagram of KL divergence variation. Figure 5 In the example (a), the task of forgetting approximately 1% of the data (forget 1) is shown. In this case, the overall shift between the two distributions is small, not significantly different from the situation when using retained data; only the interval between the two changes slightly. This indicates that when the amount of forgotten data is small, even without retained data, the model distribution remains relatively stable, although its discriminative power has slightly decreased. Figure 5 In example (b), the task corresponds to forgetting approximately 5% of the data (forget 5). At this point, the shifts between the distribution of the data to be forgotten and the distribution of the retained data relative to the reference model increase significantly, while the distance between them begins to decrease. This indicates that the overall model distribution has drifted more noticeably and is gradually losing its ability to distinguish between forgotten and retained data. Figure 5 In the example (c), the task involves forgetting approximately 10% of the data (forget 10). In this case, the two distributions not only shift significantly relative to the reference model, but also tend to overlap further, with the distribution intervals decreasing dramatically and becoming indistinguishable. This demonstrates that in the absence of constraints on retained data, large-scale forgetting leads to a severe deviation of the model distribution from the original distribution, and significantly degrades the ability to distinguish between the forgotten target and the retained knowledge.

[0047] In summary, two core problems exist under the condition of no retained data: first, the overall distribution of the model undergoes an uncontrollable shift; second, the distributions of the data to be forgotten and the retained data gradually become mixed. These two problems become the key challenges that subsequent methods need to address.

[0048] Figure 6This is an example of the experimental results provided by this invention, showing the performance of various mainstream forgetting methods on the TOFU dataset for three forgetting ratios: forget 1, forget 5, and forget 10, without using retained data. Figure 6 As shown, "↑" indicates that a larger value means better performance, including forget quality and model utility, while "↓" indicates that a smaller value means better performance, corresponding to the forget truth ratio. The comparison methods include representative algorithms such as Gradient Ascent (GA), Direct Preference Optimization (DPO), Negative Preference Optimization (NPO), Representation Misdirection for Unlearning (RMU), and Weighted Gradient Ascent (WGA).

[0049] In the forget 1 (1% forgetting) task, most methods can maintain model utility to some extent, but the forgetting quality is generally low, indicating that the model still struggles to achieve accurate forgetting. In contrast, the method of this invention achieves the highest forgetting quality (0.405) while maintaining relatively stable model utility (0.50), achieving a good balance. In the forget 5 (5% forgetting) task, other methods generally show significant degradation. For example, the model utility of GA, RMU, and WGA is close to 0, indicating a severe loss of model capability. While DPO and NPO partially retain performance, their forgetting effect is significantly reduced. In contrast, the method of this paper achieves a better trade-off between forgetting quality (0.54) and model utility (0.19), significantly outperforming the baseline methods. In the forget 10 (10% forgetting) task, the model utility of all methods is close to 0, indicating that large-scale forgetting has a significant impact on model capability. However, the method of this paper still maintains high forgetting quality (0.468), indicating that it still has good capabilities in strong forgetting scenarios.

[0050] Figure 7 This is a comparative analysis of different forgetting methods on the TOFU dataset without using retained data, provided by an embodiment of the present invention. It focuses on evaluating the performance of GA, DPO, NPO, FLAT, and Undial+NORT methods under different forgetting ratios.

[0051] from Figure 7As can be seen in (a) of the paper, in the scenario of small-scale forgetting (1%), most methods can maintain a certain level of model performance, but there is a general phenomenon of sacrificing forgetting quality to maintain model capability, resulting in low overall forgetting quality. Figure 7 As shown in (b), in a medium-scale forgetting (5%) task, the baseline methods exhibit significant divergence: one type of method (such as GA, DPO, and NPO) leads to a severe decline in model utility during forgetting, while another type of method (such as Undial+NORT) tends to maintain the original model's capabilities but has insufficient forgetting effect, resulting in a low level of forgetting quality. In contrast, the method of this invention significantly improves forgetting quality in this task while maintaining a certain level of model utility, demonstrating a superior balance. Further from... Figure 8 It can be seen that under the scenario of large-scale forgetting (10%), the forgetting quality of each method decreases overall, but the method of this invention still maintains a relatively high level, indicating that it still has good robustness under stronger forgetting pressure.

[0052] In summary, under the challenging setting of unretained data, existing methods generally struggle to simultaneously achieve both forgetting effectiveness and model performance. However, the method of this invention can achieve superior forgetting quality performance under different forgetting scales, thus verifying its effectiveness and stability in unretained data scenarios.

[0053] To address the problem that existing machine forgetting methods struggle to balance the degree of forgetting with the model's generalizability when dealing with forgetting unretained data, this invention provides a machine forgetting application method based on distributed guided large language models, which offers the following technical advantages: 1. Effectively suppresses model output distribution drift and maintains the model's generality. This invention introduces a pseudo-sample-based negative distillation loss to provide the model with pseudo-retention constraints without relying on real retained data. This allows the target model to maintain behavior similar to the reference model in the general language knowledge region, thereby anchoring the model's generality during forgetting training. This avoids the problem of the overall output distribution deviating significantly from the original distribution due to relying solely on the data to be forgotten, and significantly improves the model's stability in language understanding and generation after forgetting.

[0054] 2. Enhance the ability to distinguish between forgotten targets and retained knowledge, achieving precise forgetting. This invention constructs a keyword meta-screening mechanism, using a Gaussian mixture model to identify the set of keyword meta-elements carrying factual information at the meta-element level. Furthermore, it introduces truncated divergence loss to constrain the model distribution only in the keyword meta-element subspace. This mechanism enables the model to accurately distinguish between the distribution of target knowledge that should be deleted and the distribution of general language knowledge that should be retained, preventing the forgetting process from spreading to non-target knowledge regions. This effectively suppresses excessive forgetting while achieving complete forgetting.

[0055] 3. Joint optimization of multiple losses to achieve a synergistic balance between forgetting quality and model utility. This invention employs a weighted joint optimization of negative preference optimization loss, negative distillation loss, and truncation divergence loss. These three losses impose constraints on the model training process from three different dimensions: sequence-level forgetting drive, overall distribution anchoring, and fine-grained control of keyword meta-elements. This joint optimization mechanism enables the model to achieve a dynamic balance between forgetting intensity, generality preservation, and distribution stability, overcoming the shortcomings of single-objective or weakly constrained methods that experience a sharp performance decline as the forgetting scale increases.

[0056] 4. Reduced data dependence and improved practicality. This invention is implemented entirely without retained data, requiring no access to the original training data or any retained data; forgetting training can be completed solely based on the data to be forgotten. Compared to existing technologies that rely on retained data constraints, this invention significantly reduces the storage and retrieval costs of the original training data, making it more suitable for real-world applications where large language model training data cannot be traced or reused due to privacy, copyright, or storage limitations.

[0057] In summary, this invention, without using any retained data, achieves targeted forgetting while significantly suppressing model distribution drift and enhancing the model's ability to discriminate knowledge boundaries through a distribution-guided joint optimization mechanism. It can still achieve stable and reliable machine forgetting effects even with a large forgetting scale, providing effective technical support for the practical application of large language models in privacy compliance and security deployment scenarios.

[0058] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A machine forgetting application method based on a distributed guided large language model, characterized in that, include: S1: Obtain a natural language query from a user terminal or application system, wherein the natural language query is text composed of a variable-length sequence; S2: Using the natural language query as the initial prefix sequence, iteratively execute the following steps: concatenate the initial prefix sequence with the previously updated response tokens to form the current conditional context, input it into the trained target model to calculate the conditional probability distribution on the vocabulary, wherein the target model is obtained through iterative training using a distribution-guided machine forgetting training method under no-retention data conditions. Based on the conditional probability distribution, a preset sampling strategy is used to select the next token, and this token is appended to the end of the sequence of previously updated response tokens as the current updated response token; repeat until the termination condition is met to obtain the response text; S3: Output the reply text to the user terminal or the application system as a response to the natural language query.

2. The machine forgetting application method based on distributed guided large language model according to claim 1, characterized in that, The target model was obtained through training in the following manner: Repeat the following operations until the preset convergence condition is met, and use the model to be forgotten after the last iteration as the target model: S110: Obtain the model to be forgotten obtained after the last model training as the current model. During the initial model training, the model parameters of the model to be forgotten are the same as the model parameters of the reference model. The reference model is a pre-trained large language model, and the model to be forgotten is a large language model to be trained. S120: Obtain the original samples of the current batch from the dataset to be forgotten, and input them into the current model and the reference model respectively, and output the first predicted probability distribution and the reference probability distribution; at the same time, obtain the pseudo samples of the current batch from the pseudo sample set generated based on the dataset to be forgotten, and input them into the current model and the reference model respectively, and output the second predicted probability distribution and the pseudo-retained reference probability distribution. S130: Based on the first predicted probability distribution and the reference probability distribution, calculate the degree of preference of the current model in generating the output sequence to be forgotten on the dataset to be forgotten, and use it as the first loss value; S140: Calculate the distribution difference between the second predicted probability distribution and the pseudo-preserved reference probability distribution as the second loss value; S150: Calculate the divergence between the lexical probability distributions of the current model and the reference model on the keyword lexical set, as a third loss value; S160: The first loss value, the second loss value, and the third loss value are weighted and summed to obtain the current total loss value; S170: Update the parameters of the current model based on the current total loss value to obtain the model to be forgotten after the current model training is completed.

3. The machine forgetting application method based on distributed guided large language model according to claim 2, characterized in that, The dataset to be forgotten includes data to be forgotten from user interaction logs, privacy compliance review databases, or web crawler text sets. The dataset to be forgotten contains at least one input-output pair, where the input in each input-output pair is natural language text consisting of a variable-length sequence, and the output is the corresponding natural language answer. The set of pseudo-samples was obtained in the following way: Each input-output pair is mapped to the embedding space of the reference model to obtain the corresponding continuous semantic vector; The continuous semantic vectors obtained by mapping all the data samples to be forgotten are taken as a set, and the overall distribution of the set is fitted using a Gaussian mixture model to obtain the distribution model of the data to be forgotten. Sampling is performed within the distribution range covered by the data to be forgotten distribution model to generate multiple pseudo-samples that do not contain factual information; All pseudo-samples are aggregated to form the pseudo-sample set.

4. The machine forgetting application method based on distributed guided large language model according to claim 3, characterized in that, The keyword meta set was obtained through the following method: Each input-output pair in the dataset to be forgotten is segmented into lexical units to obtain multiple lexical sequences. Each word sequence is mapped to the embedding space of the reference model to obtain the corresponding word semantic vector; By treating all semantic vectors of words as a set, and fitting the overall distribution of this set using a Gaussian mixture model, we obtain the distribution model of words to be forgotten. Based on the statistical characteristics of each Gaussian distribution component in the lexical distribution model to be forgotten, Gaussian distribution components that carry factual information and have a posterior probability greater than a preset threshold are identified, and lexical components belonging to the Gaussian distribution components carrying factual information are selected to construct the keyword lexical set.

5. The machine forgetting application method based on distributed guided large language model according to claim 3, characterized in that, The first loss value is the negative preference optimization loss, and the expression for calculating the first loss value for each model training iteration is as follows: ； In the formula, These are the model parameters of the current model. It is the temperature coefficient. The dataset to be forgotten Calculate the mathematical expectation of all inputs and outputs. This represents the Sigmoid function. The dataset to be forgotten One of the input / output pairs, ) is the first predicted probability distribution of the current model on the dataset to be forgotten. This represents the reference probability distribution of the reference model on the dataset to be forgotten.

6. The machine forgetting application method based on distributed guided large language model according to claim 2, characterized in that, The second loss value is the negative distillation loss, and the calculation expression for the second loss value corresponding to each model training is as follows: ； In the formula, This refers to the batch size, which represents the number of pseudo-samples processed simultaneously during each model training parameter update. It represents the sequence length, indicating the number of tokens contained in each pseudo-sample. Represents cross-entropy loss, This indicates that the reference model is in the first... The sample, the first The pseudo-preserved reference probability distribution at each location Indicates that the current model is in the th The first sample, the first The second predicted probability distribution at each location.

7. The machine forgetting application method based on distributed guided large language model according to claim 4, characterized in that, The third loss value is the truncated divergence loss, and the calculation expression for the third loss value corresponding to each model training is as follows: ； In the formula, It is the set of keyword elements, The reference model is in a given prefix Term probability distribution under certain conditions The current model is given the prefix. The probability distribution of lexical terms under given conditions.

8. The machine forgetting application method based on distributed guided large language model according to claim 1, characterized in that, The preset sampling strategies include temperature sampling, Top-K, Top-P, or greedy search.

9. The machine forgetting application method based on distributed guided large language model according to claim 1, characterized in that, The termination conditions include generating an end symbol or the sequence length of the current condition context reaching its maximum length.

Images