Sandbox dynamic machine forgetting and utility compensation method, device, equipment and medium

Abstract

The application discloses a sandbox dynamic machine forgetting and utility compensation method, device, equipment and medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: when performing large model training in a sandbox, a data influence graph is constructed; when a data sovereignty revocation instruction initiated by a data provider is captured and the revocation instruction is verified, based on the data influence graph, a machine forgetting engine is used to perform a forgetting operation on a target parameter block corresponding to the revocation instruction; after the forgetting operation is completed, the forgetting completion degree of the large model is verified to obtain a verification result; and if the verification result is verified, the large model after forgetting is compensated for utility. Through the application, the technical problem that the model memory stripping of revoked data cannot be legally and compliantly completed without destroying the model and without full retraining in the prior art is solved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and specifically relates to a method, device, equipment and medium for sandbox dynamic machine forgetting and utility compensation. Background Technology

[0002] With the rapid iteration and large-scale deployment of AI technologies such as large language models and generative AI, AI model training heavily relies on externally authorized data from multiple sources and in multiple batches. Compliant data authorization, dynamic access control, and data sovereignty traceability have become core requirements for the compliant development of the AI ​​industry. In actual business scenarios, data authorization is dynamic; data providers can initiate data authorization revocation requests at any time based on compliance requirements, business changes, or adjustments to rights, demanding the complete termination of any impact of authorized data on the effectiveness of AI models.

[0003] However, existing data sandbox and trusted execution environment technologies have fundamental technical flaws. Existing technologies can only achieve physical access to the original data and terminate subsequent calculations and calls based on the data, but cannot eliminate the logical memory and feature influence of the target data solidified in the model parameters during the model iterative training process.

[0004] Therefore, there is currently a lack of a lightweight and verifiable data forgetting technology solution that is adapted to the limited sandbox computing environment, compatible with the distributed architecture of large models, can be accurately traced, can be quantitatively verified, and can be automatically compensated. It is impossible to legally and compliantly complete the model memory stripping of revoked data without destroying the model or retraining the entire dataset, which makes it difficult to meet the core needs of dynamic data authorization governance, data sovereignty protection, and compliant model operation in the AI ​​era. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method, apparatus, device, and medium for sandbox dynamic machine forgetting and utility compensation.

[0006] To achieve the above objectives, this application provides the following technical solution: A sandbox dynamic machine forgetting and utility compensation method, the method comprising: When training large models within a sandbox, construct a data influence graph; When a data sovereignty revocation command initiated by the data provider is captured and the revocation command passes the verification, based on the data influence graph, a machine forgetting engine is used to perform a forgetting operation on the target parameter block corresponding to the revocation command. After the forgetting operation is completed, the forgetting completion rate of the large model is verified to obtain the verification results; If the verification result is successful, then utility compensation is performed on the forgotten large model, and the reasoning service of the forgotten large model is restored.

[0007] Based on the same inventive concept, this application also provides a sandbox dynamic machine forgetting and utility compensation device, the device comprising: The graph building module is configured to build data influence graphs when performing large model training within a sandbox. The forgetting operation execution module is configured to perform a forgetting operation on the target parameter block corresponding to the revocation instruction based on the data influence graph and through the machine forgetting engine when a data sovereignty revocation instruction initiated by the data provider is captured and the revocation instruction passes the verification. The forgetting completion verification module is configured to verify the forgetting completion of the large model after the forgetting operation is completed, and obtain the verification result. The utility compensation module is configured to perform utility compensation on the forgotten large model and restore the inference service of the forgotten large model if the verification result is a successful verification.

[0008] Based on the same inventive concept, this application also provides an electronic device, including: a memory and a processor; the processor is used to read and execute a computer program stored in the memory to implement the steps of the aforementioned sandbox dynamic machine forgetting and utility compensation method.

[0009] Based on the same inventive concept, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement the steps of the aforementioned sandbox dynamic machine forgetting and utility compensation method.

[0010] Compared with the prior art, this application has the following advantages: By using the constructed data influence graph, the parameter blocks affected by the revoked data can be accurately located without retraining the entire model, and a rapid response can be made to users' data sovereignty revocation requests. Once the forgetting completion of the large model is verified, utility compensation is performed on the forgotten large model. Without compromising the forgetting effect, the model's performance on general tasks is quickly restored. This ensures that the model does not relearn the features of the revoked data while restoring performance, thus guaranteeing the stability of the forgetting effect.

[0011] This application solves the technical problem in the prior art that it is impossible to legally and compliantly strip away model memory from data without destroying the model or retraining the entire dataset.

[0012] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating the method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the overall process of an embodiment of this application; Figure 3 This is a schematic diagram of the functional modules of an embodiment of the sandbox dynamic machine forgetting and utility compensation device of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To address the shortcomings of existing technologies, refer to Figure 1 This application discloses a sandbox dynamic machine forgetting and utility compensation method, the method comprising: Step S10: Construct a data influence graph while performing large model training in the sandbox; In some specific embodiments, step S10 includes: Step S101: When performing large model training in the sandbox, a block-incremental approximate Hessian matrix calculation method is used to calculate the influence score of each batch of training data on the model parameters of the large model. Step S102: Based on the influence score of each training data batch on the model parameters of the large model, the parameter block identifier corresponding to each training data batch, and the hash fingerprint of the gradient vector, multiple influence records are obtained and written into a tamper-proof trusted storage. Step S103: Using the identifier of the data batch as the retrieval index, all influence records are grouped and aggregated to construct a data influence map that reflects the relationship between each training data batch and the model parameters of the large model.

[0017] In this embodiment, when performing large model training within the sandbox, the system embeds an influence tracking module to record the parameter contribution of each training data batch in real time. Considering the limited available memory in the sandbox (especially the TEE environment), this application adopts a block-based incremental approximation Hessian matrix calculation method, rather than calculating the complete Hessian matrix all at once. Specifically, the model parameters of the large model are divided into K parameter blocks by layer or module, and the local Hessian approximation matrix of each parameter block is calculated using a first preset formula; that is, the local Hessian matrix is ​​calculated using the Kronecker factorization approximation (K-FAC) method, decomposing the hierarchical Hessian matrix into the Kronecker product of the input covariance matrix and the gradient covariance matrix.

[0018] The first preset formula is as follows:

[0019] In the formula, This represents the Hessian approximation matrix of the k-th parameter block. Let be the covariance matrix of the k-th parameter block. This represents the covariance matrix of the k-th parameter block. This represents the Kronecker product operation.

[0020] and During training, updates are performed online using an exponential moving average, requiring no additional storage or computational overhead. Compared to directly calculating the complete Hessian matrix (memory requirement O(P²), where P is the number of parameters), K-FAC decomposition reduces the memory requirement to O(Σ(n)). k ² + m k ²), where n k Let m be the input dimension of the k-th layer. k Let the output dimension of the k-th layer be feasible in the limited memory of the TEE, O( The space / memory complexity is represented by Σ(n). By using K-FAC block decomposition to approximate the calculation of the complete Hessian matrix, the memory requirement for influence tracking is reduced from O(P²) to O(Σ(n)). k ²+m k ²)) makes it feasible under the condition of limited memory in TEE, and solves the problem that existing influence function methods cannot be applied in constrained computing environments.

[0021] Furthermore, it should be noted that, in addition to the K-FAC decomposition approximation, alternatives include: lightweight influence estimation methods based on first-order gradient information (such as tracking methods based on gradient cosine similarity, which calculate the cosine similarity between the gradient of each data batch and the final gradient of the model as an influence approximation); and higher-precision Hessian approximation methods based on EK-FAC (Eigenvalue-corrected K-FAC).

[0022] For the i-th batch of training data Based on the i-th batch of training data Using the loss value, the inverse of the Hessian matrix, and the gradient of the loss function under the current model parameters W of the large model, the influence score of the i-th batch of training data on the model parameters of the large model is calculated through the second preset formula. By analogy, the influence score of each batch of training data on the model parameters of the large model is calculated. The second preset formula is as follows:

[0023] In the formula, This represents the i-th batch of training data. The influence score on the model parameters of large models. This represents the inverse of the Hessian approximation matrix of the k-th parameter block. , representing the covariance matrix of the k-th parameter block. The inverse matrix represents the covariance matrix of the k-th parameter block. The inverse matrix, This represents the k-th parameter block. Represents the gradient operator. This indicates that the loss function L is applied to the k-th parameter block. The gradient, L(Di;W) represents the gradient of the i-th batch of training data. The loss value under the current model parameters W of the large model, where L represents the loss function. Indicates the first Each batch of training data Represents the model parameters of a large model, || || represents norm operations.

[0024] Based on the influence score of each training data batch on the model parameters of the large model, the parameter block identifier corresponding to each training data batch, and the hash fingerprint of the gradient vector, multiple influence records are obtained and written into a tamper-proof trusted storage to ensure that they cannot be modified later, providing an irrefutable basis for data forgetting audit.

[0025] Using the data batch identifier as the retrieval index, all influence records are grouped and aggregated to construct a data influence graph reflecting the relationship between each training data batch and the model parameters of the large model. In the graph, each training data batch is a node, and each parameter block is also a node. The edge weight between the two is the influence score of that batch on the corresponding parameter block.

[0026] When it is necessary to perform forgetting on a batch of data later, the main parameter blocks that are affected by the batch can be quickly located through the data influence map, and the forgetting operation can be performed only on these target parameter blocks.

[0027] Step S20: When a data sovereignty revocation instruction initiated by the data provider is captured and the revocation instruction passes the verification, based on the data influence graph, a forgetting operation is performed on the target parameter block corresponding to the revocation instruction through the machine forgetting engine. Further, in one embodiment, after the data sovereignty revocation instruction initiated by the data provider is captured, the process includes: The legality of data sovereignty revocation orders is verified, including: whether the initiator of the order is the original data provider, whether the data batch identifier specified in the order exists in the influence graph of the current training task, and whether the revocation order is within the revocation window period agreed upon in the digital contract. After successful verification, the data sovereignty revocation command will be marked as pending execution.

[0028] In this embodiment, when a data provider initiates a data sovereignty revocation command through the Trusted Data Space Service Platform (e.g., authorization expiration, unilateral termination, compliance requirements, etc.), the sovereignty audit module within the sandbox captures the command in real time. The sovereignty audit module first verifies the legality of the revocation command, including: whether the initiator of the command is the original data provider, whether the data batch identifier specified in the command exists in the influence graph of the current training task, and whether the revocation command is within the allowed revocation window period stipulated in the digital contract. After successful verification, the revocation command is marked as "pending execution."

[0029] Furthermore, in one embodiment, after the data sovereignty revocation instruction initiated by the data provider is captured, the method further includes: When multiple data sovereignty revocation commands are received concurrently from data providers within the same time window, they are queued according to the arrival time of the data sovereignty revocation commands and the forgetting operation is performed in sequence. For concurrent data sovereignty revocation requests with the same parameter blocks, their combined influence is calculated and then a forgetting operation is performed together.

[0030] In this embodiment, if multiple concurrent revocation commands are received from data providers within the same time window, the system queues them according to their arrival time and performs the forgetting operation sequentially. For concurrent revocation requests involving the same parameter block, the system calculates their combined influence and performs a single forgetting operation to avoid parameter oscillations caused by repeated rollbacks.

[0031] For example, suppose a large text model training platform receives concurrent data sovereignty revocation instructions from three data providers within the same time window, namely: Request A: Cancel the batch-A data provided by user A; Request B: Cancel the batch-B data provided by user B; Request C: Revoke the batch-C data provided by user C; 1. Execute in a queue according to arrival time (different parameter block scenarios).

[0032] First, the platform sorts the requests according to the order in which they arrive: Request A (10:00:01), Request B (10:00:02), Request C (10:00:03).

[0033] According to the pre-constructed data influence map query: Request A primarily affects Block 1 (input layer) of the model; Request B primarily affects Block 2 (the middle layer) of the model; Request C primarily affects Block 3 (output layer) of the model; Since the three target parameter blocks do not overlap, the platform performs the forgetting operation sequentially according to the arrival order: First, process request A: perform targeted forgetting on Block 1, and simultaneously perform subsequent utility compensation and verification; After request A is processed, request B will be processed: perform forgetting and compensation on Block 2; Final processing request C: Perform forgetting and compensation on Block 3.

[0034] This queuing mechanism avoids conflicts caused by multiple requests concurrently modifying the same parameter block, ensuring the atomicity and correctness of each forgotten operation.

[0035] 2. Combine and calculate joint influence (for scenarios with the same parameter blocks).

[0036] Suppose that within the same time window, the platform receives two other concurrent requests: Request D: Revoke the batch-D data provided by user D, which mainly affects Block 2 of the model; Request E: Revoke the batch-E data provided by user E, which mainly affects Block 2 of the model; Both requests have a target parameter block of Block 2. Instead of processing them sequentially, the platform performs a merge optimization: First, query the data influence graph to obtain the influence scores of Batch-D and Batch-E for Block 2; The combined influence of both factors on Block 2 is calculated to obtain a comprehensive influence vector. Based on this combined influence, a batch forgetting operation is performed on Block 2 at once, while a unified utility compensation and verification is conducted.

[0037] This approach avoids repeated fine-tuning and verification of the same parameter block, significantly improving processing efficiency in concurrent scenarios and reducing the risk of performance fluctuations caused by repeated model modifications.

[0038] Through the above embodiments, the platform not only ensures the orderly processing of concurrent data sovereignty revocation requests, but also achieves efficient merging of requests with the same parameter blocks, thus balancing the correctness of data forgetting with the system's operational efficiency.

[0039] In some specific embodiments, the step of performing a forgetting operation on the target parameter block corresponding to the revocation instruction based on the data influence graph and through a machine forgetting engine includes: In the data influence graph, retrieve the target influence score of the target training data batch corresponding to the revocation instruction; Determine the target parameter block from the parameter block corresponding to each target influence score, wherein the target influence score corresponding to the target parameter block is greater than the influence threshold; In a distributed training architecture, the forgetting coordinator included in the machine forgetting engine determines the corresponding target computing node based on the target parameter block. The new weight of the k'th target parameter block after the forgetting operation is calculated using the third preset formula. The machine forgetting engine uses a forgetting coordinator to send forgetting instructions carrying the new weights to the target computing node. Based on the new weights, in a local trusted security environment, after independently performing reverse parameter updates on the target parameter blocks corresponding to each target computing node, the update results of the reverse parameter updates and the parameter hash values ​​after the reverse parameter updates are returned to the forgetting coordinator. If the update results of each target computing node are all successful, and the consistency check of the parameter hash value after the reverse parameter update passes, then the forget transaction is submitted uniformly. If the update result of at least one of the target computing nodes fails, or the consistency check of the parameter hash value after the reverse parameter update fails, the entire forgetting operation will be rolled back to the checkpoint before the update. The third preset formula is as follows:

[0040] In the formula, Indicates the first The new weights after the forgetting operation is performed on each target parameter block. Indicates the first The current weights of each objective parameter block, where η represents the adaptive forgetting step size. This represents the inverse of the covariance matrix of the input activation of the k-th objective parameter block. The inverse matrix of the covariance matrix of the output gradient of the k-th objective parameter block. This represents the Kronecker product operation. No. Parameter block, Represents the gradient operator. No. Batch of training data for each target Model parameters of the large model before performing the forgetting operation. The loss value is given below, where L represents the loss value. This represents the model parameters of the large model before the forgetting operation is performed. Indicates the first Each batch of target training data.

[0041] In this embodiment, after the revocation command passes verification, the sandbox control plane immediately performs the following safety operations: freezes the inference service exit of the current model version containing the revoked data batch (returning a "model under maintenance" status code), suspends the ongoing training task and saves the training checkpoint, and transfers control to the machine forgetting engine. The machine forgetting engine retrieves the target influence score of the target training data batch corresponding to the revocation command from the data influence graph.

[0042] Target parameter blocks are determined from the parameter blocks corresponding to each target influence score, where the target influence score of the target parameter block is greater than the influence threshold. Further, the affected parameter blocks are sorted according to the target influence score, with priority given to processing the parameter blocks with the highest influence. For target influence scores lower than the influence threshold θ... skip The parameter block (default is 1% of the maximum influence) skips the forgetting operation to save computational resources.

[0043] For the first one that needs to be forgotten For each target parameter block, the system calculates the new weight of the target data batch Di after performing a forgetting operation on that parameter block using a third preset formula, and then performs a reverse parameter update: The third preset formula is as follows:

[0044] In the formula, Indicates the first The new weights after the forgetting operation is performed on each target parameter block. Indicates the first The current weights of each objective parameter block, where η represents the adaptive forgetting step size. This represents the inverse of the covariance matrix of the input activation of the k-th objective parameter block. The inverse matrix of the covariance matrix of the output gradient of the k-th objective parameter block. This represents the Kronecker product operation. No. Parameter block, Represents the gradient operator. No. Batch of training data for each target Model parameters of the large model before performing the forgetting operation. The loss value is given below, where L represents the loss value. This represents the model parameters of the large model before the forgetting operation is performed. Indicates the first Each batch of target training data.

[0045] The kth target parameter block corresponds to the kth network layer.

[0046] The value of the forgetting step size η needs to balance the thoroughness of forgetting with parameter stability: if η is too large, it may lead to parameter oscillations and the incidental forgetting of other data; if η is too small, forgetting will be incomplete. This embodiment uses a forgetting step size that is adaptively adjusted based on influence scores: η(Di) = η_base × Influence(Di) / max(Influence), where η_base is the base forgetting step size (default value is 0.1), Influence(Di) is the influence score of the target data batch, and max(Influence) is the maximum influence score of all data batches in the current influence map. This adaptive mechanism ensures that high-influence data receives a greater forgetting force, while low-influence data has a smaller forgetting step size to reduce collateral effects on other data.

[0047] When large models employ a distributed training architecture with data parallelism or model parallelism, model parameters are distributed across multiple computing nodes. The forgetting operation needs to be executed synchronously on all nodes holding copies of the affected parameter block. Therefore, in a distributed training architecture, the forgetting coordinator included in the machine forgetting engine is required to determine the target computing node corresponding to the target parameter block. The forgetting coordinator determines the list of affected parameter blocks based on the influence graph, issues forgetting instructions with new weights to each computing node holding these parameter blocks, and after each node independently performs parameter updates in its local TEE, it returns the update results of the reverse parameter update and the updated parameter hash value to the coordinator.

[0048] For example, suppose a large model uses a distributed training architecture, deployed across three computing nodes (Node 1, Node 2, and Node 3), with each node responsible for a different parameter block of the model. Upon receiving a batch data revocation command from a user, the distributed forgetting operation is performed as follows: 1. Search for target influence score: In the data influence graph, retrieve the target influence score of the target training data batch corresponding to the revocation instruction. The query results show that the influence of this batch of data on Block 1 (Node 1) of the model is 0.7, on Block 2 (Node 2) is 0.6, and on Block 3 (Node 3) is 0.2.

[0049] 2. Determine the target parameter block: The preset influence threshold is 0.5, and parameter blocks with scores greater than this threshold are selected. Therefore, the target parameter blocks are Block 1 and Block 2, and the corresponding target computation nodes are Node 1 and Node 2.

[0050] 3. Calculate the new weights after forgetting: Using the third preset formula, calculate the new weights of Block 1 and Block 2 after the forgetting operation: New weights for Block 1: ; New weights for Block 2: .

[0051] 4. Issue distributed forgetting instructions: The forgetting coordinator in the machine forgetting engine distributes new weights to Node 1 and Node 2 respectively. and The forget instruction.

[0052] 5. Perform reverse update locally on the node: Each target computing node independently performs reverse parameter updates within its local trusted security environment (such as a TEE): Node 1 updates the parameters of Block 1 to And calculate the updated parameter hash value. ; Node 2 updates the parameters of Block 2 to And calculate the updated parameter hash value. ; After the update is complete, both nodes return the update result and hash value to the forget coordinator.

[0053] 6. Consistency verification and transaction commit or rollback: If the reverse parameter updates of Node 1 and Node 2 both result in "success", and the coordinator updates the parameter hash value... and If the consistency check passes, the forgetting operation is deemed valid, the forgetting transaction is submitted uniformly, and the model is officially switched to the updated state.

[0054] If any node returns an update result of "update failed" for the reverse parameter update, or the updated parameter hash value... and If the verification is inconsistent, a global rollback is triggered, restoring the parameters of Node 1 and Node 2 to the checkpoint state before the update. This forgotten operation is invalidated to ensure that the model state is not destroyed.

[0055] This embodiment fully reproduces the precise forgetting based on influence graphs, independent node execution, and two-phase commit consistency guarantee process under a distributed architecture, demonstrating the atomicity and security of the patented technology. Furthermore, building upon traditional sandbox physical isolation, it adds parameter-level forgetting capabilities based on inverse gradient updates, completely removing the influence of revoked data at the model weight level. The adaptive forgetting step size mechanism ensures thorough forgetting while reducing collateral impacts on other data.

[0056] Furthermore, it should be noted that, in addition to the method based on inverse gradient update, alternatives include: the SISA framework method, which trains the training data into an ensemble of multiple sub-models, and only retrains the sub-model slice containing the target data when the training is cancelled; and the gradient ascent-based forgetting method, which performs several gradient ascent operations on the cancelled data to actively increase its loss value, forcing the model to "forget" the data.

[0057] Step S30: After the forgetting operation is completed, the forgetting completion rate of the large model is verified to obtain the verification result; In some specific embodiments, the verification of the forgetting completion rate of the large model to obtain the verification results includes: The samples in the target data batch corresponding to the target parameter block are used as member samples; Data that are distributed in the same manner as the target data batch but were not used in training are used as non-member samples; A binary classification attack model is trained based on the member samples and the non-member samples to obtain the trained binary classification attack model. The trained binary classification attack model is used to classify member samples and non-member samples in the target data batch. The classification accuracy of the binary classification attack model after statistical training was calculated. Based on the classification accuracy, the forgetting completion score is calculated using the fourth preset formula. If the forgetting completion score is greater than or equal to the forgetting threshold, the verification result of the forgetting completion is "verification passed". If the forgetting completion score is less than the forgetting threshold, the verification result for forgetting completion is that the verification failed. The fourth preset formula is as follows:

[0058] in, This indicates that the trained binary classification attack model is effective against the first... The classification accuracy of classifying samples in a batch of target data; Indicates the large model for the first Forgotten completion score for each batch of target data.

[0059] In this embodiment, after the forgetting operation is completed, the system automatically performs a Membership Inference Attack (MIA) test to verify whether the revoked data can still be identified from the model. The test method is as follows: using samples from the revoked data batch Di as "member samples" and using data with the same distribution as Di but not involved in training as "non-member samples", a binary classification attack model is trained based on the member samples and non-member samples to obtain the trained binary classification attack model; the member samples and non-member samples in the target data batch are classified based on the trained binary classification attack model.

[0060] The classification accuracy of the binary classification attack model after statistical training is calculated; based on the classification accuracy, the forgetting completion score is calculated using the fourth preset formula. The fourth preset formula is as follows:

[0061] in, This indicates that the trained binary classification attack model is effective against the first... The classification accuracy of classifying samples in a batch of target data; Indicates the large model for the first Forgotten completion score for each batch of target data.

[0062] If the forgetting completion score is greater than or equal to the forgetting threshold, it indicates that the model can no longer distinguish between member samples and non-member samples, and the forgetting effect meets expectations. In this case, the verification result of forgetting completion is considered to be successful. If the forgetting completion score is less than the forgetting threshold, it indicates that forgetting is incomplete, and the verification result for forgetting completion is that the verification fails.

[0063] By employing member inference attack testing and the ForgetScore for forgetting completion, verifiable evidence of forgetting completion is provided to data providers, addressing the trust gap in existing solutions where "forgetting was performed but cannot be proven to be complete." The iterative forgetting mechanism ensures that the set forgetting completion threshold is reached.

[0064] Furthermore, it should be noted that, in addition to verification based on member inference attacks, alternative methods include: verification based on differences in the output distribution of the large model, comparing the changes in the predicted probability distribution of the large model for the revoked data before and after forgetting (such as KL divergence); and verification based on adversarial example detection, constructing adversarial examples targeting the features of the revoked data to verify whether the large model remains sensitive to these adversarial examples after forgetting.

[0065] Further, in one embodiment, after verifying the forgetting completion rate of the large model and obtaining the verification result, the process includes: When the verification result is that the verification fails, the forgetting step size is increased, and the new forgetting step size is used as the forgetting step size. The process is then returned to the step of performing the forgetting operation on the target parameter block corresponding to the cancellation instruction based on the data influence map and through the machine forgetting engine, until the verification result is that the verification passes or the maximum number of iterations is reached. If the verification result still fails when the maximum number of iterations is reached, an alarm will be generated and the parameter blocks other than the target parameter block in the large model will be frozen. The target parameter block will be retrained only based on the remaining training data after removing the revoked data.

[0066] In this embodiment, if the forgetting completion score ForgetScore(Di) is less than the forgetting threshold θforget, it indicates that a single forgetting operation has failed to completely eliminate the influence of the target data. The system automatically performs iterative forgetting: increases the forgetting step size η (multiplied by the amplification factor 1.5), uses the new forgetting step size as the forgetting step size, and returns to the execution of the steps based on the data influence map, through the machine forgetting engine, to perform the forgetting operation on the target parameter block corresponding to the revocation instruction, until the verification result is verified as passed or the maximum number of iterations (default 5 times) is reached.

[0067] If the maximum number of iterations is reached and the verification still fails, an alarm will be generated and the parameter blocks other than the target parameter block in the large model will be frozen. The target parameter block will be retrained only based on the remaining training data after removing the revoked data.

[0068] Step S40: If the verification result is successful, then utility compensation is performed on the forgotten large model, and the reasoning service of the forgotten large model is restored.

[0069] In some specific embodiments, step S40 includes: Obtain the first and second classification accuracies of the large model on the benchmark set before and after forgetting; Based on the first classification accuracy and the second classification accuracy, the utility loss rate is calculated using the fifth preset formula; The fifth preset formula is as follows: UtilityLoss = (ACC_before ACC_after) / ACC_before In the formula, UtilityLoss represents the utility loss rate; ACC_before represents the first classification accuracy of the large model on the benchmark set before forgetting; and ACC_after represents the second classification accuracy of the large model on the benchmark set after forgetting. If the utility loss rate is less than or equal to the utility threshold, then the reasoning service of the large model after forgetting is restored; If the utility loss rate is greater than the utility threshold, then synthetic data is used to compensate for the utility loss in the large model after forgetting. The optimization objective of the utility compensation is determined by a sixth preset formula, which is as follows: W_comp = argmin E(x,y)~Dsynth [L(f(x; W_after), y)] + λ||W W_after||² In the formula, W_comp represents the model parameters of the large model after utility compensation, and argmin( ) represents the parameter W_comp, E(x,y)~Dsynth[ that minimizes the entire expression. The expression ] represents the average loss calculated for samples (x, y) in the synthetic data Dsynth; L represents the loss function, f(x; W_after) represents the forward propagation function of the model, W_after represents the model parameters of the large model after forgetting, x represents the input sample, y represents the label of the synthetic data, Dsynth represents the synthetic data, λ represents the regularization coefficient, and ||W W_after||² represents the regularization term, and W represents the current model parameters of the large model.

[0070] In this embodiment, after the forgetting verification is passed, the sovereign audit module automatically generates a structured "Forgetting Completion Certificate." The certificate includes: the identifier and initiator information of the revocation instruction, the identifier and data volume of the revoked data batch, the execution time of the forgetting operation and the list of involved parameter blocks, the difference hash values ​​of the model parameters before and after forgetting, the ForgetScore score of the member inference attack test, and the accuracy of the model on the benchmark set after forgetting (to prove the model is still usable). The Forgetting Completion Certificate is signed by the trusted signature key of the sandbox environment, submitted to the Trusted Data Space Service Platform for record-keeping, and the data provider is notified. The data provider can use this to verify that its data has indeed been removed from the model.

[0071] After the forgetting operation and validation are completed, the system uses a pre-set benchmark set to evaluate the utility of the model after forgetting. Specifically, it obtains the first classification accuracy and the second classification accuracy of the large model before and after forgetting on the benchmark set; based on the first classification accuracy and the second classification accuracy, it calculates the utility loss rate using a fifth pre-set formula.

[0072] The fifth preset formula is as follows: UtilityLoss = (ACC_before ACC_after) / ACC_before In the formula, UtilityLoss represents the utility loss rate; ACC_before represents the first classification accuracy of the large model on the benchmark set before forgetting; and ACC_after represents the second classification accuracy of the large model on the benchmark set after forgetting.

[0073] If the utility loss rate is less than or equal to the utility threshold, then the inference service of the large model after forgetting is restored, that is, all influence records of the revoked data batch Di are deleted from the influence graph; the model version number is updated and the forgetting event is recorded; the model inference interface is reopened to restore external services; and the data consumer is notified that the model has been updated (including the reason for forgetting and the utility evaluation results, but excluding the specific content of the revoked data).

[0074] If the utility loss rate exceeds the utility threshold, the system utilizes pre-preserved public, copyright-free data, or generates a batch of synthetic data (Dsynth) in real time using a generative model that satisfies differential privacy guarantees. The synthetic data must meet two conditions: first, it must have similar global statistical distribution characteristics (such as feature mean, variance, and correlation structure) to the revoked data to repair the feature space damaged by the forgetting operation; second, it must not contain any privacy information that identifies a specific data subject. The synthetic data is then used to compensate for the utility loss in the large model after the forgetting process.

[0075] The optimization objective of utility compensation is determined by the sixth pre-set formula, which is as follows: W_comp = argmin E(x,y)~Dsynth [L(f(x; W_after), y)] + λ||W W_after||² In the formula, W_comp represents the model parameters of the large model after utility compensation, and argmin( ) represents the parameter W_comp, E(x,y)~Dsynth[ that minimizes the entire expression. [] represents the average loss calculated for samples (x, y) in the synthetic data Dsynth; L represents the loss function, f(x; W_after) represents the forward propagation function of the model, W_after represents the model parameters of the large model after forgetting, x represents the input sample, y represents the label of the synthetic data, Dsynth represents the synthetic data, λ represents the regularization coefficient, W represents the current model parameters of the large model, ||W W_after||² represents the regularization term, which constrains the compensation fine-tuning to avoid deviating too far from the forgotten parameters, and prevents accidental "relearning" of the features of the revoked data during the compensation process.

[0076] Lightweight fine-tuning based on synthetic data, combined with parameter regularization constraints, restores model accuracy while preventing the relearning of features from revoked data. A utility loss assessment mechanism provides a quantitative assessment of compensation eligibility, avoiding unnecessary computational overhead.

[0077] Furthermore, it should be noted that, in addition to fine-tuning based on synthetic data, alternatives include: a compensation method based on knowledge distillation, which uses the large model before forgetting as the teacher model and the large model after forgetting as the student model, and performs knowledge distillation on public data that does not involve the revoked data to restore utility; and a parameter protection method based on elastic weight consolidation (EWC), which marks important parameters before forgetting and applies protective constraints to them during forgetting and compensation.

[0078] Further, in one embodiment, after the utility compensation is performed on the forgotten large model, the process includes: Return to the step of validating the forgetting completion rate of the large model and obtaining the validation results; If the verification result is that the verification fails, then strengthen the regularization constraint on utility compensation, or reduce the learning rate of utility compensation, or reduce the number of iterations of utility compensation, and then perform utility compensation again, and re-verify the forgetting completion of the large model to obtain the verification result. If the verification result is still that the verification failed, then the parameter blocks other than the target parameter block in the large model are frozen, and the target parameter block is retrained only based on the remaining training data after removing the revoked data.

[0079] In this embodiment, it is assumed that a large model has completed the forgetting operation on the user-revoked data batch Di, and then performs utility compensation fine-tuning to restore model performance. After compensation, closed-loop verification and iterative correction are performed according to the following process: 1. Return to perform the forgetting completion verification.

[0080] First, the forgotten model is validated using Member Inference Attack (MIA). Samples from the target data batch Di are used as member samples, and samples that were not trained and are in the same distribution are used as non-member samples. A binary classification attack model is trained and the classification accuracy is calculated to obtain the forgetting completion score.

[0081] The initial validation results showed that the forgetting completion score was 0.6, which was lower than the forgetting threshold of 0.8, and the validation was deemed unsuccessful. This indicates that the utility compensation fine-tuning caused the model's memory of Di to regress.

[0082] First round of corrections: Adjusting the utility compensation hyperparameter To restore performance while preventing rollback due to forgetfulness, the system performs the following hyperparameter adjustments: Increase the regularization constraint strength of utility compensation, or reduce the learning rate of utility compensation, or reduce the number of iterations of utility compensation.

[0083] Strengthen the regularization constraint: Increase the regularization coefficient λ in the formula from 0.01 to 0.1 to increase the restriction on parameter updates; Reduce the learning rate: decrease the utility-compensated learning rate from 1e-4 to 1e-5; Reduce the number of iteration rounds: reduce the number of fine-tuning iteration rounds from 10 rounds to 3 rounds.

[0084] After the adjustment is completed, utility compensation is re-executed based on the synthetic data, and the forgetting completion rate of the model is verified again.

[0085] If the second verification result shows that the forgetting completion score is still 0.7, which is still below the forgetting threshold of 0.8, the verification fails. In this case, the system performs a fallback forgetting operation: Freeze all parameter blocks in the large model except for the target parameter block (such as Block 1 and Block 3), and only open the target parameter block Block 2 affected by Di; Using the remaining training data after removing the revoked data Di, only Block 2 is retrained to reconstruct the parameters affected by the compensated fine-tuning interference, and to eliminate the residual features of the revoked data Di since the training data no longer contains it. After fine-tuning, the forgetting completion rate is verified again.

[0086] If the third validation result shows a forgetting completion score of 0.92, which is greater than the forgetting threshold of 0.8, the validation is considered successful. At this point, the model has both restored its performance for general tasks and stably achieved the forgetting effect on Di, and the process ends.

[0087] Furthermore, the verification results are as follows: after verification, all influence records of the revoked data batch Di are deleted from the influence graph; the model version number is updated and the forgetting event is recorded; the model inference interface is reopened to resume external service; and the data consumer is notified that the model has been updated (including the reasons for forgetting and the utility evaluation results, but excluding the specific content of the revoked data).

[0088] Specifically, 1. Delete the records of data that have been revoked in the influence graph.

[0089] The system removes all influence records related to the revoked data batch Di from the data influence graph, including the batch's influence scores on each parameter block, parameter block identifiers, and gradient vector hash fingerprints. The updated graph no longer contains any entries related to Di, ensuring that subsequent data operations cannot trace back to this batch of data.

[0090] 2. Update the model version number and record forgotten events.

[0091] Update the model version number from v1.2.0 to v1.2.1, and record this forgotten event in the system audit log, including the event ID, the batch identifier Di that was revoked, the time of the forgotten operation, the execution result, and other information, to form an unalterable operation audit record.

[0092] 3. Restore the model's external inference service.

[0093] After completing the above updates, the system reopens the model's inference interface and resumes external services. At this time, when users call the model API for inference, they will use the new version of the model v1.2.1, which has been forgotten Di and has undergone utility compensation. This version no longer contains features of the revoked data and has restored general task performance.

[0094] 4. Notify data consumers of model updates.

[0095] The system sends a model update notification to all data consumers, informing them that their models have completed data omission and version upgrades. The notification includes the reason for omission (such as revocation of user data sovereignty), the model version number change, and the updated utility evaluation results (such as general task accuracy, response latency, etc.), but does not include any specific content of the revoked data, thus ensuring data privacy and security.

[0096] Through the above steps, the system completes a closed loop of the entire process from data forgetting and utility compensation to service recovery, ensuring that the model service can continue to provide services to the public safely and stably under the premise of compliance.

[0097] In this embodiment, a data influence graph is constructed during large-scale model training within the sandbox. When a data sovereignty revocation command initiated by the data provider is captured and the revocation command passes verification, a forgetting operation is performed on the target parameter block corresponding to the revocation command based on the data influence graph and through a machine forgetting engine. After the forgetting operation is completed, the forgetting completion rate of the large model is verified to obtain the verification result. If the verification result is successful, utility compensation is performed on the forgotten large model, and the inference service of the forgotten large model is restored. Through this embodiment, the constructed data influence graph is used to accurately locate the parameter block affected by the revoked data, without retraining the entire model, enabling a rapid response to user data sovereignty revocation requests. After the forgetting completion rate of the large model is verified, utility compensation is performed on the forgotten large model, quickly restoring the model's performance on general tasks without compromising the forgetting effect. This ensures that the model does not relearn the features of the revoked data while restoring performance, guaranteeing the stability of the forgetting effect. This solves the technical problem in the prior art of legally and compliantly completing the model memory stripping of revoked data without destroying the model or retraining the entire model.

[0098] Reference Figure 2 , Figure 2 This is a schematic diagram of the overall process of an embodiment of this application, as shown below. Figure 2 As shown, when the data provider (user) initiates a data sovereignty revocation command, the system can safely and compliantly "forget" the target data from the trained large model without retraining the entire model. This is achieved through a closed-loop process of precise positioning, reverse parameter updating, forgetting effect verification, utility evaluation, and compensation optimization, while ensuring the overall availability and security of the model.

[0099] The specific process is broken down step by step as follows: 1. Process initialization and preliminary preparations: Start → When performing large model training in the sandbox, construct a data influence graph (i.e., the relationship between the influence scores of data batches and parameter blocks). Essentially, this records the influence weight of each batch of training data on each parameter block of the model, which is the basis for locating subsequent forgetting operations.

[0100] 2. Command validity verification: The system captures data sovereignty revocation orders initiated by data providers and then verifies the legality of these orders, specifically checking whether their source, permissions, and format are compliant.

[0101] If the verification fails, the instruction is rejected and an alarm is issued, and the process is terminated.

[0102] If the verification passes, proceed to the subsequent forgetting process.

[0103] 3. Target parameter block location: Based on the influence graph, the target parameter block is located by using a machine forgetting engine (filtering parameter blocks with influence ≥ threshold), that is, only the part of the model parameters that have been affected by the target data are found, avoiding a full model update.

[0104] 4. Reverse parameter update: The new weights are calculated using the third preset formula, and a reverse parameter update is performed. After multi-node synchronization and consistency verification of the distributed forgetting coordinator, the forgetting transaction is committed. This step directly modifies the model parameters to offset the impact of the target data.

[0105] 5. Forgotten Completion Verification: Perform forgetting completion verification on the large model after forgetting (i.e., perform member inference attack test and calculate the forgetting completion score ForgetScore (the forgetting completion score measures whether the target data can still be recognized by the model)).

[0106] If the ForgetScore is greater than or equal to the forgetting threshold, it means that the target data has been effectively forgotten, and the next step of utility evaluation can be performed.

[0107] If the ForgetScore is less than the Forgetting Threshold, it means that the forgetting is incomplete, and the iterative optimization branch is entered: increase the forgetting step size η and perform the reverse update again; If the maximum number of iterations is not reached, a downgrade process is applied: all parameters except the target parameter block are frozen, and the target parameter block is retrained using only the remaining training data after removing the revoked data.

[0108] 6. Model utility evaluation: UtilityLoss (a measure of the impact of forgetting operations on the overall performance of the model) is calculated using the fifth preset formula.

[0109] If the utility loss rate ≤ the utility threshold, it means that the model performance has not been significantly affected, and the process can proceed to the finalization stage.

[0110] If UtilityLoss > Utility threshold, it indicates that the model performance has degraded excessively and utility compensation needs to be performed: use synthetic data to compensate for the forgotten model and use regularization constraints to prevent the model from relearning the revoked data.

[0111] After compensation, re-verify ForgetScore: If it meets the target: return to the main process to finish; if it does not meet the target: adjust the strategy (enhance regularization constraints, reduce the learning rate, reduce the number of iterations) and then perform compensation again.

[0112] 7. Finishing and Updates: Remove the records of revoked data from the influence graph, update the model version number, and record this forgotten event; Resume the model's inference service and notify the data consumer that the model has been updated; The process is complete.

[0113] Based on the same inventive concept, embodiments of this application also provide a sandbox dynamic machine forgetting and utility compensation device.

[0114] In one embodiment, reference is made to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of the sandbox dynamic machine forgetting and utility compensation device of this application. Figure 3 As shown, the sandbox dynamic machine forgetting and utility compensation device includes: Graph construction module 10 is configured to construct a data influence graph when performing large model training within a sandbox; The forgetting operation execution module 20 is configured to perform a forgetting operation on the target parameter block corresponding to the revocation instruction based on the data influence graph and through the machine forgetting engine when a data sovereignty revocation instruction initiated by the data provider is captured and the revocation instruction passes the verification. The forgetting completion verification module 30 is configured to verify the forgetting completion of the large model after the forgetting operation is completed, and obtain the verification result. The utility compensation module 40 is configured to perform utility compensation on the forgotten large model and restore the reasoning service of the forgotten large model if the verification result is a successful verification.

[0115] Optionally, in one embodiment, the map construction module 10 is configured to: When training a large model in the sandbox, a block-incremental approximate Hessian matrix calculation method is used to calculate the influence score of each batch of training data on the model parameters of the large model. Based on the influence score of each training data batch on the model parameters of the large model, the parameter block identifier corresponding to each training data batch, and the hash fingerprint of the gradient vector, multiple influence records are obtained and written into a tamper-proof trusted storage. Using the identifier of the data batch as the retrieval index, all influence records are grouped and aggregated to construct a data influence map that reflects the relationship between each training data batch and the model parameters of the large model.

[0116] Optionally, in one embodiment, the map construction module 10 is configured to: The model parameters of the large model are divided into K parameter blocks by layer or module, and the local Hessian approximation matrix of each parameter block is calculated by the first preset formula. For the i-th batch of training data, based on the loss value of the i-th batch of training data under the model parameters W of the current large model, the inverse of the Hessian matrix, and the gradient of the loss function, the influence score of the i-th batch of training data on the model parameters of the large model is calculated using the second preset formula. By analogy, the influence score of each batch of training data on the model parameters of the large model is calculated. The first preset formula is as follows:

[0117] In the formula, This represents the Hessian approximation matrix of the k-th parameter block. Let be the covariance matrix of the k-th parameter block. This represents the covariance matrix of the k-th parameter block. This represents the Kronecker product operation; The second preset formula is as follows:

[0118] In the formula, This represents the i-th batch of training data. The influence score on the model parameters of large models. This represents the inverse of the Hessian approximation matrix of the k-th parameter block. This represents the k-th parameter block. Represents the gradient operator. This indicates that the loss function L is applied to the k-th parameter block. The gradient, L( W) represents the i-th batch of training data. The loss value under the current model parameters W of the large model, where L represents the loss function. Indicates the first Each batch of training data The parameters represent the model parameters of the large model, and || represents the norm operation.

[0119] Optionally, in one embodiment, the sandbox dynamic machine forgetting and utility compensation device further includes a verification module configured to: The legality of data sovereignty revocation orders is verified, including: whether the initiator of the order is the original data provider, whether the data batch identifier specified in the order exists in the influence graph of the current training task, and whether the revocation order is within the revocation window period agreed upon in the digital contract. After successful verification, the data sovereignty revocation command will be marked as pending execution.

[0120] Optionally, in one embodiment, the forgetting operation execution module 20 is configured to: In the data influence graph, retrieve the target influence score of the target training data batch corresponding to the revocation instruction; Determine the target parameter block from the parameter block corresponding to each target influence score, wherein the target influence score corresponding to the target parameter block is greater than the influence threshold; In a distributed training architecture, the forgetting coordinator included in the machine forgetting engine determines the corresponding target computing node based on the target parameter block. The new weight of the k'th target parameter block after the forgetting operation is calculated using the third preset formula. The machine forgetting engine uses a forgetting coordinator to send forgetting instructions carrying the new weights to the target computing node. Based on the new weights, in a local trusted security environment, after independently performing reverse parameter updates on the target parameter blocks corresponding to each target computing node, the update results of the reverse parameter updates and the parameter hash values ​​after the reverse parameter updates are returned to the forgetting coordinator. If the update results of each target computing node are all successful, and the consistency check of the parameter hash value after the reverse parameter update passes, then the forget transaction is submitted uniformly. If the update result of at least one of the target computing nodes fails, or the consistency check of the parameter hash value after the reverse parameter update fails, the entire forgetting operation will be rolled back to the checkpoint before the update. The third preset formula is as follows:

[0121] In the formula, Indicates the first The new weights after the forgetting operation is performed on each target parameter block. Indicates the first The current weights of each objective parameter block, where η represents the adaptive forgetting step size. This represents the inverse of the covariance matrix of the input activation of the k-th objective parameter block. The inverse matrix of the covariance matrix of the output gradient of the k-th objective parameter block. This represents the Kronecker product operation. No. Parameter block, Represents the gradient operator. No. Batch of training data for each target Model parameters of the large model before performing the forgetting operation. The loss value is given below, where L represents the loss value. This represents the model parameters of the large model before the forgetting operation is performed. Indicates the first Each batch of target training data.

[0122] Optionally, in one embodiment, the forgetting completion verification module 30 is configured to: The samples in the target data batch corresponding to the target parameter block are used as member samples; Data that are distributed in the same manner as the target data batch but were not used in training are used as non-member samples; A binary classification attack model is trained based on the member samples and the non-member samples to obtain the trained binary classification attack model. The trained binary classification attack model is used to classify member samples and non-member samples in the target data batch. The classification accuracy of the binary classification attack model after statistical training was calculated. Based on the classification accuracy, the forgetting completion score is calculated using the fourth preset formula. If the forgetting completion score is greater than or equal to the forgetting threshold, the verification result of the forgetting completion is "verification passed". If the forgetting completion score is less than the forgetting threshold, the verification result for forgetting completion is that the verification failed. The fourth preset formula is as follows:

[0123] in, This indicates that the trained binary classification attack model is effective against the first... The classification accuracy of classifying samples in a batch of target data; Indicates the large model for the first Forgotten completion score for each batch of target data.

[0124] Optionally, in one embodiment, the forgetting completion verification module 30 is further configured to: When the verification result is that the verification fails, the forgetting step size is increased, and the new forgetting step size is used as the forgetting step size. The process is then returned to the step of performing the forgetting operation on the target parameter block corresponding to the cancellation instruction based on the data influence map and through the machine forgetting engine, until the verification result is that the verification passes or the maximum number of iterations is reached. If the verification result still fails when the maximum number of iterations is reached, an alarm will be generated and the parameter blocks other than the target parameter block in the large model will be frozen. The target parameter block will be retrained only based on the remaining training data after removing the revoked data.

[0125] Optionally, in one embodiment, the utility compensation module 40 is configured to: Obtain the first and second classification accuracies of the large model on the benchmark set before and after forgetting; Based on the first classification accuracy and the second classification accuracy, the utility loss rate is calculated using the fifth preset formula; The fifth preset formula is as follows: UtilityLoss = (ACC_before ACC_after) / ACC_before In the formula, UtilityLoss represents the utility loss rate; ACC_before represents the first classification accuracy of the large model on the benchmark set before forgetting; and ACC_after represents the second classification accuracy of the large model on the benchmark set after forgetting. If the utility loss rate is less than or equal to the utility threshold, then the reasoning service of the large model after forgetting is restored; If the utility loss rate is greater than the utility threshold, then synthetic data is used to compensate for the utility loss in the large model after forgetting. The optimization objective of the utility compensation is determined by a sixth preset formula, which is as follows: W_comp = argmin E(x,y)~Dsynth [L(f(x; W_after), y)] + λ||W W_after||² In the formula, W_comp represents the model parameters of the large model after utility compensation, and argmin( ) represents the parameter W_comp, E(x,y)~Dsynth[ that minimizes the entire expression. The expression ] represents the average loss calculated for samples (x, y) in the synthetic data Dsynth; L represents the loss function, f(x; W_after) represents the forward propagation function of the model, W_after represents the model parameters of the large model after forgetting, x represents the input sample, y represents the label of the synthetic data, Dsynth represents the synthetic data, λ represents the regularization coefficient, and ||W W_after||² represents the regularization term, and W represents the current model parameters of the large model.

[0126] Optionally, in one embodiment, the forgetting completion verification module 30 is configured to: Return to the step of validating the forgetting completion rate of the large model and obtaining the validation results; If the verification result is that the verification fails, then strengthen the regularization constraint on utility compensation, or reduce the learning rate of utility compensation, or reduce the number of iterations of utility compensation, and then perform utility compensation again, and re-verify the forgetting completion of the large model to obtain the verification result. If the verification result is still that the verification failed, then the parameter blocks other than the target parameter block in the large model are frozen, and the target parameter block is retrained only based on the remaining training data after removing the revoked data.

[0127] Optionally, in one embodiment, the forgetting operation execution module 20 is configured to: When multiple data sovereignty revocation commands are received concurrently from data providers within the same time window, they are queued according to the arrival time of the data sovereignty revocation commands and the forgetting operation is performed in sequence. For concurrent data sovereignty revocation requests with the same parameter blocks, their combined influence is calculated and then a forgetting operation is performed together.

[0128] The functions of each module in the aforementioned sandbox dynamic machine forgetting and utility compensation device correspond to the steps in the aforementioned sandbox dynamic machine forgetting and utility compensation method embodiment, and their functions and implementation processes will not be described in detail here.

[0129] Based on the same inventive concept, embodiments of this application also provide an electronic device, the structure of which is as follows: Figure 4 As shown, it includes: a memory and a processor, wherein the processor is used to read and execute the computer program stored in the memory to implement the aforementioned sandbox dynamic machine forgetting and utility compensation method.

[0130] Based on the same inventive concept, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned sandbox dynamic machine forgetting and utility compensation method.

[0131] Finally, it should be noted that while some processes described in the embodiments of this application include multiple operations or steps that appear in a specific order, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0132] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A sandbox dynamic machine forgetting and utility compensation method, characterized in that, The method includes: When training large models within a sandbox, construct a data influence graph; When a data sovereignty revocation command initiated by the data provider is captured and the revocation command passes the verification, based on the data influence graph, a machine forgetting engine is used to perform a forgetting operation on the target parameter block corresponding to the revocation command. After the forgetting operation is completed, the forgetting completion rate of the large model is verified to obtain the verification results; If the verification result is successful, then utility compensation is performed on the forgotten large model, and the reasoning service of the forgotten large model is restored.

2. The method according to claim 1, characterized in that, When performing large model training within a sandbox, the construction of a data influence graph includes: When training a large model in the sandbox, a block-incremental approximate Hessian matrix calculation method is used to calculate the influence score of each batch of training data on the model parameters of the large model. Based on the influence score of each training data batch on the model parameters of the large model, the parameter block identifier corresponding to each training data batch, and the hash fingerprint of the gradient vector, multiple influence records are obtained and written into a tamper-proof trusted storage. Using the identifier of the data batch as the retrieval index, all influence records are grouped and aggregated to construct a data influence map that reflects the relationship between each training data batch and the model parameters of the large model.

3. The method according to claim 2, characterized in that, The method employing a block-based incremental approximation Hessian matrix calculation calculates the influence score of each training data batch on the model parameters of the large model, including: The model parameters of the large model are divided into K parameter blocks by layer or module, and the local Hessian approximation matrix of each parameter block is calculated by the first preset formula. For the i-th batch of training data, based on the loss value of the i-th batch of training data under the model parameters W of the current large model, the inverse of the Hessian matrix, and the gradient of the loss function, the influence score of the i-th batch of training data on the model parameters of the large model is calculated using the second preset formula. By analogy, the influence score of each batch of training data on the model parameters of the large model is calculated. The first preset formula is as follows: In the formula, This represents the Hessian approximation matrix of the k-th parameter block. Let be the covariance matrix of the k-th parameter block. This represents the covariance matrix of the k-th parameter block. This represents the Kronecker product operation; The second preset formula is as follows: In the formula, This represents the i-th batch of training data. The influence score of the model parameters of a large model. This represents the inverse of the Hessian approximation matrix of the k-th parameter block. This represents the k-th parameter block. Represents the gradient operator, This indicates that the loss function L is applied to the k-th parameter block. The gradient, L( W) represents the i-th batch of training data. The loss value under the current model parameters W of the large model, where L represents the loss function. Indicates the first Each batch of training data Represents the model parameters of a large model, || || represents norm operations.

4. The method according to claim 1, characterized in that, After a data sovereignty revocation instruction initiated by the data provider is captured, the following is included: The legality of data sovereignty revocation orders is verified, including: whether the initiator of the order is the original data provider, whether the data batch identifier specified in the order exists in the influence graph of the current training task, and whether the revocation order is within the revocation window period agreed upon in the digital contract. After successful verification, the data sovereignty revocation instruction will be marked as pending execution.

5. The method according to claim 1, characterized in that, The step of performing a forgetting operation on the target parameter block corresponding to the revocation instruction based on the data influence graph and through a machine forgetting engine includes: In the data influence graph, retrieve the target influence score of the target training data batch corresponding to the revocation instruction; Determine the target parameter block from the parameter block corresponding to each target influence score, wherein the target influence score corresponding to the target parameter block is greater than the influence threshold; In a distributed training architecture, the forgetting coordinator included in the machine forgetting engine determines the corresponding target computing node based on the target parameter block. The new weight of the k'th target parameter block after the forgetting operation is calculated using the third preset formula. The machine forgetting engine uses a forgetting coordinator to send forgetting instructions carrying the new weights to the target computing node. Based on the new weights, in a local trusted security environment, after independently performing reverse parameter updates on the target parameter blocks corresponding to each target computing node, the update results of the reverse parameter updates and the parameter hash values ​​after the reverse parameter updates are returned to the forgetting coordinator. If the update results of each target computing node are all successful, and the consistency check of the parameter hash value after the reverse parameter update passes, then the forget transaction is submitted uniformly. If the update result of at least one of the target computing nodes fails, or the consistency check of the parameter hash value after the reverse parameter update fails, the entire forgetting operation will be rolled back to the checkpoint before the update. The third preset formula is as follows: In the formula, Indicates the first The new weights after the forgetting operation is performed on each target parameter block. Indicates the first The current weights of each target parameter block, where η represents the adaptive forgetting step size; This represents the inverse of the covariance matrix of the input activation of the k-th objective parameter block. This represents the inverse of the covariance matrix of the output gradient of the k-th objective parameter block. This represents the Kronecker product operation. Indicates the first One target parameter block, Represents the gradient operator, Indicates the first Batch of training data for each target Model parameters of the large model before performing the forgetting operation. The loss value is given below, where L represents the loss value. This represents the model parameters of the large model before the forgetting operation is performed. Indicates the first Each batch of target training data.

6. The method according to claim 1, characterized in that, The verification of the forgetting completion rate of the large model yields the following verification results: The samples in the target data batch corresponding to the target parameter block are used as member samples; Data that are distributed in the same manner as the target data batch but were not used in training are used as non-member samples; A binary classification attack model is trained based on the member samples and the non-member samples to obtain the trained binary classification attack model. The trained binary classification attack model is used to classify member samples and non-member samples in the target data batch. The classification accuracy of the binary classification attack model after statistical training was calculated. Based on the classification accuracy, the forgetting completion score is calculated using the fourth preset formula. If the forgetting completion score is greater than or equal to the forgetting threshold, the verification result of the forgetting completion is "verification passed". If the forgetting completion score is less than the forgetting threshold, the verification result for forgetting completion is that the verification failed. The fourth preset formula is as follows: in, This indicates that the trained binary classification attack model is effective against the first... The classification accuracy of classifying samples in a batch of target data; Indicates the large model for the first Forgotten completion score for each batch of target data.

7. The method according to claim 1, characterized in that, After verifying the forgetting completion rate of the large model and obtaining the verification results, the process includes: When the verification result is that the verification fails, the forgetting step size is increased, and the new forgetting step size is used as the forgetting step size. The process is then returned to the step of performing the forgetting operation on the target parameter block corresponding to the cancellation instruction based on the data influence map and through the machine forgetting engine, until the verification result is that the verification passes or the maximum number of iterations is reached. If the verification result still fails when the maximum number of iterations is reached, an alarm will be generated and the parameter blocks other than the target parameter block in the large model will be frozen. The target parameter block will be retrained only based on the remaining training data after removing the revoked data.

8. The method according to claim 1, characterized in that, The utility compensation for the large model after forgetting includes: Obtain the first and second classification accuracies of the large model on the benchmark set before and after forgetting; Based on the first classification accuracy and the second classification accuracy, the utility loss rate is calculated using the fifth preset formula; The fifth preset formula is as follows: UtilityLoss = (ACC_before ACC_after) / ACC_before In the formula, UtilityLoss represents the utility loss rate; ACC_before represents the first classification accuracy of the large model on the benchmark set before forgetting; and ACC_after represents the second classification accuracy of the large model on the benchmark set after forgetting. If the utility loss rate is less than or equal to the utility threshold, then the reasoning service of the large model after forgetting is restored; If the utility loss rate is greater than the utility threshold, then synthetic data is used to compensate for the utility loss in the large model after forgetting. The optimization objective of the utility compensation is determined by a sixth preset formula, which is as follows: W_comp = argmin E(x,y)~Dsynth [L(f(x; W_after), y)] + λ||W W_after||² In the formula, W_comp represents the model parameters of the large model after utility compensation, and argmin( ) represents the parameter W_comp, E(x,y)~Dsynth[ that minimizes the entire expression. The expression ] represents the average loss calculated for samples (x, y) in the synthetic data Dsynth; L represents the loss function, f(x; W_after) represents the forward propagation function of the model, W_after represents the model parameters of the large model after forgetting, x represents the input sample, y represents the label of the synthetic data, Dsynth represents the synthetic data, λ represents the regularization coefficient, and ||W W_after||² represents the regularization term, and W represents the current model parameters of the large model.

9. The method according to claim 1, characterized in that, After performing utility compensation on the large model following the forgetting process, the following is included: Return to the step of validating the forgetting completion rate of the large model and obtaining the validation results; If the verification result is that the verification fails, then strengthen the regularization constraint on utility compensation, or reduce the learning rate of utility compensation, or reduce the number of iterations of utility compensation, and then perform utility compensation again and re-verify the forgetting completion of the large model to obtain the verification result. If the verification result is still that the verification failed, then the parameter blocks other than the target parameter block in the large model are frozen, and the target parameter block is retrained only based on the remaining training data after removing the revoked data.

10. The method according to claim 1, characterized in that, After the data sovereignty revocation instruction initiated by the data provider is captured, the following is also included: When multiple data sovereignty revocation commands are received concurrently from data providers within the same time window, they are queued according to the arrival time of the data sovereignty revocation commands and the forgetting operation is performed in sequence. For concurrent data sovereignty revocation requests with the same parameter blocks, their combined influence is calculated and then a forgetting operation is performed together.

11. A sandbox dynamic machine forgetting and utility compensation device, characterized in that, The device includes: The graph building module is configured to build data influence graphs when performing large model training within a sandbox. The forgetting operation execution module is configured to perform a forgetting operation on the target parameter block corresponding to the revocation instruction based on the data influence graph and through the machine forgetting engine when a data sovereignty revocation instruction initiated by the data provider is captured and the revocation instruction passes the verification. The forgetting completion verification module is configured to verify the forgetting completion of the large model after the forgetting operation is completed, and obtain the verification result. The utility compensation module is configured to perform utility compensation on the forgotten large model and restore the inference service of the forgotten large model if the verification result is a successful verification.

12. An electronic device, characterized in that, include: Memory, processor; The processor is configured to read and execute the computer program stored in the memory to implement the steps of the sandbox dynamic machine forgetting and utility compensation method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, implement the steps of the sandbox dynamic machine forgetting and utility compensation method according to any one of claims 1-10.

Images