A dynamic data desensitization method for sensitive field identification

Abstract

This invention discloses a dynamic data anonymization method for sensitive field identification, comprising: collaboratively training a probe model on multiple clients and, based on a federated learning framework, identifying specific data samples containing sensitive information to be anonymized without exposing the original data; calculating the degree of influence of the specific data sample on each parameter of the target neural network model and selecting the subset of key parameters most significantly affected; constructing a multi-objective optimization function including information forgetting, knowledge preservation, and regularization terms, and optimizing and updating only the subset of key parameters, thereby inversely eliminating the influence of the specific data sample in the model; verifying the anonymization effect by quantitatively evaluating the degree of forgetting of the specific data sample by the model after anonymization; this invention achieves efficient and low-loss model data anonymization through precise positioning and minimal editing, and is particularly suitable for federated learning environments with strict privacy requirements, and provides a quantifiable security verification mechanism, significantly improving the efficiency, fidelity, and credibility of data anonymization.

Description

Technical Field

[0001] This invention relates to the field of data processing technology under neural networks, and in particular to a dynamic data desensitization method for identifying sensitive fields. Background Technology

[0002] With the deep integration of artificial intelligence and big data technologies, deep neural network models have been widely applied in key fields such as finance, healthcare, and security. During this integration process, model training inevitably encounters and learns from a large amount of highly sensitive data, including personal identity, financial status, and biometrics. In recent years, with increasingly stringent global data privacy regulations, data anonymization technologies have emerged to meet compliance requirements and mitigate data leakage risks. Traditional anonymization methods primarily operate in the data preprocessing stage, using anonymization, pseudonymization, and data masking to process data before it is stored in the training set. However, these methods struggle to address the potential privacy leakage risks of already trained models, as the model parameters may have already "memorized" deep features of sensitive data and can be reverse-engineered using techniques such as member inference attacks.

[0003] To address privacy concerns at the model level, academia and industry have proposed the concept of machine learning model unlearning. The most direct method is "retraining from scratch," which involves removing the data to be deleted from the training set and retraining the entire model. While this method is thorough, it is extremely computationally expensive and impractical for scenarios with complex model structures, massive datasets, and frequent deletion requests. To address this, some researchers have proposed approximate retraining methods, such as data sharding and aggregation schemes. These schemes train multiple sub-models by splitting the data, and only retrain the relevant sub-models during deletion. However, this usually comes at the cost of sacrificing the overall generalization performance of the model. Other methods attempt to approximate the unlearning effect by applying reverse updates to the model parameters. However, these methods generally lack precise insight into the path of sensitive information's influence within the model, and their update operations are often global or heuristic, easily causing catastrophic damage to other irrelevant knowledge already learned by the model while erasing the target information. Furthermore, most current technologies are based on a pre-set centralized data environment, which makes them difficult to apply to the collaborative identification of sensitive information in distributed or federated learning scenarios where data exists in silos. Moreover, after the de-identification operation is completed, there is generally a lack of effective quantitative evaluation mechanisms to verify from the attacker's perspective whether the information has been completely removed. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a dynamic data desensitization method for identifying sensitive fields, to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a dynamic data desensitization method for sensitive field identification, wherein the method is applied to a target neural network model trained on an original training dataset containing multiple data samples, and includes the following steps:

[0007] When a desensitization request is received for at least one data sample to be desensitized in the original training dataset, the degree of influence of the data sample to be desensitized on each model parameter in the target neural network model is calculated, and a subset of key parameters is determined based on the degree of influence.

[0008] Based on an optimization objective function that includes a knowledge retention term, an information forgetting term, and a parameter regularization term, a set of parameter update amounts is calculated only for the subset of key parameters whose influence is determined, and the parameter update amounts are applied to modify the subset of key parameters to obtain a desensitized target neural network model.

[0009] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification according to the present invention, a sensitive contribution identification step is further included before receiving the desensitization request. This step includes:

[0010] A probe model is collaboratively trained within a federated learning framework consisting of a central server and multiple data-holding clients.

[0011] Each client uses the trained probe model to score the degree of influence of the data samples it holds locally, and the data samples with scores exceeding a preset contribution threshold are identified as the data samples to be de-identified.

[0012] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the collaborative training process of the probe model includes:

[0013] Each client updates its probe model locally based on its local data and the loss gradient of the target neural network model.

[0014] Each client sends the parameter changes generated by its local updates to the central server via a secure aggregation protocol;

[0015] The central server aggregates the parameter changes sent by all clients and uses the aggregation results to update a global probe model.

[0016] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the specific process of determining the subset of key parameters includes:

[0017] Calculate the gradient of the loss function of the data sample to be desensitized on the target neural network model with respect to each model parameter;

[0018] Simultaneously, an influence vector is generated, which is obtained by multiplying the gradient with a Hessian inverse matrix, wherein the Hessian inverse matrix is ​​the inverse of the Hessian matrix of the loss function of the target neural network model on the original training dataset;

[0019] The model parameters corresponding to the elements in the influence vector whose values ​​exceed a preset threshold are selected to form the key parameter subset.

[0020] In a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the product of the gradient and a Hessian inverse matrix is ​​calculated through an iterative approximation process, wherein each iteration of this process includes:

[0021] A subset of data is randomly sampled from the original training dataset, and the Hessian matrix is ​​calculated as a vector product of the Hessian matrix and the result of the previous iteration using this subset of data.

[0022] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, wherein: in the optimization objective function:

[0023] The knowledge preservation term is obtained by the loss function of the target neural network model on a dataset that does not contain the data samples to be desensitized;

[0024] The information forgetting term is obtained by the loss function of the target neural network model on the data sample to be desensitized, and this term is configured to be maximized when solving the optimization objective function;

[0025] The parameter regularization term is obtained from the norm of the parameter update quantity.

[0026] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the knowledge retention term is calculated on a calibration set, which is obtained by sampling after excluding the data samples to be desensitized from the original training dataset.

[0027] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, after obtaining a desensitized target neural network model, a desensitization effect verification step is further included, which includes:

[0028] Using a member inference attack model, the member confidence scores of the data samples to be de-identified are obtained before and after the target neural network model is modified, and the member confidence scores before and after the modification are compared.

[0029] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the desensitization effect verification step further includes:

[0030] On an independent test dataset, the performance metrics of the target neural network model before and after modification are calculated and compared.

[0031] As a preferred embodiment of the dynamic data desensitization method for sensitive field identification described in this invention, the desensitization effect verification step further includes:

[0032] A baseline model is constructed, which is obtained by retraining on a training set with the data samples to be de-identified removed;

[0033] Calculate and compare the statistical difference measure between the output probability distribution of the desensitized target neural network model and the benchmark model when processing the same input data.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. This invention introduces interpretable machine learning technology, which can accurately locate the influence path of the data to be de-identified in the neural network model, identify the key parameter subset that is strongly related to it, and modify only the subset. It is constrained by a multi-objective optimization function that includes knowledge preservation, information forgetting and parameter regularization. This fundamentally changes the blindness caused by global or heuristic updates in the existing technology. It can efficiently erase specific information while retaining the original generalization ability of the model to the greatest extent, and effectively avoids the catastrophic forgetting of irrelevant knowledge.

[0036] 2. Furthermore, in view of the limitations of existing technologies that mostly rely on centralized data environments, this invention designs a sensitive contribution collaborative identification mechanism based on a federated learning framework. By securely and collaboratively training a lightweight probe model, it can effectively identify high-risk sensitive data that has a significant impact on the global model without the data of each data holder leaving their local area. This solves the problem of sensitive target identification in data silo environments and enhances the deployability and compliance of model forgetting technology in real-world business scenarios.

[0037] 3. In addition, this invention introduces an adversarial verification mechanism from the attacker's perspective, using a member inference attack model to quantify the thoroughness of the removal of sensitive information. At the same time, by combining the performance evaluation of the model on an independent test set and the output distribution difference measurement with the "retrained from scratch" benchmark model, it provides a quantifiable objective basis for the success of the desensitization operation, ensuring the true effectiveness of the desensitization. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0039] Figure 1 This is a flowchart illustrating the overall process of a dynamic data desensitization method for sensitive field identification according to an embodiment of the present invention. Detailed Implementation

[0040] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0041] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0043] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0044] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0045] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0046] Example 1

[0047] Reference Figure 1 This is the first embodiment of the present invention, which provides a dynamic data desensitization method for identifying sensitive fields, including:

[0048] It should be noted that the solution of this invention is based on an original training dataset containing multiple data samples. Completed target neural network model The above-mentioned activities, among which, These are all the parameters of the target neural network model. , This indicates that the parameter belongs to a p-dimensional real space. (p is the dimension of the total parameters of the model); and the target neural network model includes, but is not limited to, convolutional neural networks (CNN) for image classification, recurrent neural networks (RNN) or Transformer models for sequence data processing, and multilayer perceptrons (MLP) for tabular data, etc. In the present invention, a ResNet-18 model for image classification task is used as the target neural network model.

[0049] S1. When a desensitization request is received for at least one data sample to be desensitized in the original training dataset, the degree of influence of the data sample to be desensitized on each model parameter in the target neural network model is calculated, and a subset of key parameters is determined based on the degree of influence.

[0050] It should be noted that the goal of this step is to respond to a specific sample of data to be de-identified. When requesting data anonymization, precise quantitative analysis is used to identify model parameters. The data sample to be de-identified The most significant part is determining a subset of key parameters. To avoid blindly modifying the entire model; among them, in the data samples to be de-identified middle, Represented as the feature part of the model sample. The label portion is represented as a model sample;

[0051] Furthermore, before formally performing the desensitization operation, the present invention proactively discovers the target to be desensitized through a pre-identification stage of sensitive contribution. This pre-identification stage of sensitive contribution is carried out in a federated learning framework consisting of a central server and multiple data holder clients. Moreover, each data holder client collaboratively trains a lightweight probe model without exposing its local data.

[0052] Specifically, in one feasible implementation of the present invention, the probe model is a multilayer perceptron (MLP) containing two hidden layers. The input layer dimension is consistent with the feature dimension of the data sample. The first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons, both of which use the ReLU activation function. The output layer is a single neuron that does not use an activation function and directly outputs the degree of influence score.

[0053] Furthermore, the central server first initializes a global probe model, which includes an initialized global probe model parameter and sets the total number of communication rounds R for training. Subsequently, the central server distributes the initialized global probe model parameter to all data holder clients participating in the training.

[0054] Specifically, in round t During the communication, each participating client receives global probe model parameters. After (among them) It is a global probe model (based on the parameters after the (t-1)th round of communication), perform the following local update operation:

[0055] Since the training objective of this probe model is to learn to predict a data sample The potential impact on the target neural network model; therefore, in this invention, we quantify this impact using the loss gradient norm of the target neural network model on the sample, that is, the larger the loss gradient norm of a sample, the greater the contribution of that sample to the parameter update of the model; where, unlike the aforementioned desensitized samples, Similarly, the probe model learns to predict data samples for prediction. This is represented as the feature portion of the predicted data sample. This is represented as the label portion of the predicted data sample;

[0056] For each sample in the local dataset, the client first calculates the loss gradient of the target neural network model on that sample. And obtain its L2 norm. This serves as the true impact score for the sample; where... It is the target neural network model The gradient of the loss function L with respect to the model parameters on this data sample (Loss gradient); This is the true impact score of the data sample. Loss gradient L2 norm ;

[0057] Subsequently, the client uses its local data samples and their corresponding calculated true impact scores as labels to train a local probe model replica. The optimization objective is to minimize the difference between the probe model's predicted values ​​and the true impact scores. In this minimization process, the present invention uses a mean squared error loss function. The client then uses stochastic gradient descent (SGD) to optimize this mean squared error loss function for several rounds, updating its local probe model parameters to obtain the updated probe model parameters. ;

[0058] Specifically, the mean squared error loss function Represented as:

[0059]

[0060] in, Here, c represents the mean squared error loss function of the probe model on the client side. For the client's local dataset, This is represented as the probe model parameters on the client side. Represented as probe model parameters The gradient relative to the local loss;

[0061] Furthermore, once local training is complete, each client calculates the change in its probe model parameters. That is, the difference between the locally updated parameters and the received global parameters. The data will be processed through a pre-agreed security aggregation protocol before being sent to the central server. This security aggregation protocol can be based on homomorphic encryption technology, where the client uploads encrypted parameter changes, or it can be based on differential privacy technology, where the client adds an appropriate amount of noise to the parameter changes before uploading.

[0062] It should be noted that, in order to protect the privacy of local data on each client, parameter changes cannot be uploaded directly in plaintext.

[0063] Furthermore, the central server waits for and collects the processed parameter changes sent from all clients;

[0064] Specifically, the central server performs aggregation operations. If additive homomorphic encryption is used, the central server will sum all encrypted parameter changes on the ciphertext. If a conventional federated averaging algorithm is used, the central server will perform a weighted average of the received (potentially noisy) parameter changes. Regardless of which method is used, the central server will use the aggregated parameter changes to update the global probe model.

[0065] Furthermore, the central server redistributes the updated global model parameters to all clients, starts the next round of training, and repeats the above local update operation until the set total number of communication rounds R of training is completed and the local update operation ends.

[0066] It should be noted that, through the above-mentioned federated learning collaborative training process, the present invention can obtain a global probe model that accurately assesses the impact of any data sample on the target model, and the entire process protects the data privacy of each client.

[0067] Furthermore, each client uses the trained probe model to score the degree of influence of the data samples it holds locally, and identifies the data samples with scores exceeding the preset contribution threshold as data samples to be de-identified.

[0068] Specifically, the impact scores of all data samples calculated locally by each client are sorted in descending order, and the top 1% are selected as the preset contribution threshold. That is, each client identifies the top 1% of samples with the greatest local impact as data samples to be desensitized.

[0069] Furthermore, once the aforementioned data privacy is ensured, then upon receiving samples of data to be de-identified... After submitting the anonymization request, a subset of key parameters can be determined without considering data privacy issues. ;

[0070] Specifically, determine the subset of key parameters. The steps are as follows:

[0071] Calculate the data samples to be de-identified In the current target neural network model The gradient of the loss function on the model with respect to the model parameters represents the gradient of the loss function on the model with respect to the model parameters, which represents the gradient of the loss function on the model with respect to the model parameters, in order to better fit the model to the data sample to be desensitized. The direction in which the model parameters need to be adjusted can be expressed mathematically as follows:

[0072]

[0073] in, It is the calculated loss gradient. Indicates the model parameters The differentiation operation, Representation Model exist The loss function can be either cross-entropy loss or mean squared error loss;

[0074] It should be noted that, from a technical perspective, due to the loss gradient Pointing to the fastest increasing parameter space in the model. The direction of the loss, namely "remembered" and In the opposite direction, but only by the loss gradient Making such a judgment is one-sided because it only reflects... It only considers the local information of the sample itself, without taking into account the interaction between the sample and all other training data, or the coupling relationship between model parameters. Therefore, directly using the gradient may incorrectly evaluate the importance of model parameters.

[0075] To address the aforementioned issues, this invention introduces an influence function from the field of interpretable machine learning to measure influence from a more global and precise perspective. The influence function can estimate the impact of training data. From the original training set Remove from model parameters The magnitude of the change that will occur can be approximated by the following formula:

[0076]

[0077] in, Represented as an influence vector, it is a vector that correlates with the parameters of the original target neural network model. For vectors of the same dimension, the j-th element The value directly quantifies the sample For the j-th model parameter The degree of impact; The model is on the entire original training dataset. The Hessian matrix of the average loss function, i.e., the matrix of second-order partial derivatives of the loss function with respect to the parameters, is defined as follows:

[0078]

[0079] The Hessian matrix describes the local curvature of the loss function and reflects the sensitivity of the model parameters. (As mentioned earlier...) It is represented as the inverse of the Hessian matrix, and The product operation essentially corrects the local gradient direction using global curvature information, thereby obtaining a more accurate impact assessment.

[0080] However, for modern neural network models, the number of parameters... It can reach hundreds of millions or even higher, directly calculating or storing to find the inverse. The computation of the Hessian matrix is ​​completely infeasible. To overcome this computational bottleneck, this invention employs an efficient iterative approximation algorithm (based on the idea of ​​the second-order stochastic optimization algorithm (LiSSA)) to directly calculate the Hessian inverse matrix-vector product, i.e. And no explicit construction is required throughout the entire process. or The algorithm's iterative process is as follows:

[0081] Initialization: Set the iteration count Q and initialize a vector. ;

[0082] Iterative updates: For Execute the following loop:

[0083] From the original training dataset Randomly sample a subset of data without replacement ;

[0084] Calculate the Hessian matrix of the current batch and the result of the previous iteration. The product of these two gradients can be performed using two automatic differentiations (i.e., two backpropagations), which has a computational cost that is only a few times that of conventional gradient calculations, completely avoiding the overhead of traditional gradient computation. The complexity of this product operation is as follows: First, calculate the loss in... gradient on Then calculate the scalar. Regarding parameters The gradient of is given by . ;in, It is a representation of computational complexity. The complexity is used to directly calculate the complexity of inverting the Hessian matrix (where p is the dimension of the model parameters mentioned above). It is a subset of the current batch of data. The gradient of the loss function L with respect to the model parameters (Batch gradient); Batch gradient The transpose T and the result of the previous iteration The inner product (scalar); similarly, This is the Hessian matrix of the current batch. Compared with the previous iteration result The product of (Hessian vector product).

[0085] Update the estimate of the Hessian inverse matrix-vector product using the following formula:

[0086]

[0087] in, It is the identity matrix, and this update rule is essentially solving a system of linear equations. The stochastic approximate iteration converges to the true solution. ;

[0088] Output: The vector obtained after Q iterations. That is, the influence vector High-precision approximation ;in, It is about treating desensitized samples In model parameters The influence function approximation (influence vector) is equal to... ( (The estimated vector after Q iterations of LiSSA).

[0089] Furthermore, once an approximate influence vector is obtained, the numerical values ​​of the elements in this vector intuitively reflect the importance of each parameter. Due to the high redundancy of parameters in neural networks, most elements in the vector will have very small values. Therefore, it is necessary to set an influence threshold to filter out the model parameters corresponding to elements in the influence vector whose absolute values ​​exceed this threshold, thus forming the final subset of key parameters. ;

[0090] Specifically, in one feasible implementation of the present invention, the influence threshold is set to the element in the top 1% position after the absolute values ​​of all elements in the influence vector are arranged in descending order, to ensure that the size of the key parameter subset is approximately 1% of the total number of parameters;

[0091] Specifically, this screening process is expressed using a mathematical expression:

[0092]

[0093] in, This is represented as the impact threshold. Model parameters The j-th parameter in It is the j-th element in the influence vector. Represented as arbitrary They all belong to model parameters ;

[0094] It should be noted that there are various strategies for selecting the influence threshold. For example, it can be a fixed value set based on experience; or it can be a dynamic value, such as selecting the minimum absolute value corresponding to the top k% of the absolute values ​​of all elements in the influence vector as the threshold.

[0095] It should be noted that, through the above steps of determining the subset of key parameters, the present invention can reduce the scope of subsequent desensitization operations from millions or even billions of parameters in the entire model to a much smaller but highly relevant subset of field-sensitive information.

[0096] S2. Based on an optimization objective function that includes a knowledge retention term, an information forgetting term, and a parameter regularization term, calculate a set of parameter update amounts only for the subset of key parameters whose influence is determined, and apply the parameter update amounts to modify the subset of key parameters to obtain a desensitized target neural network model.

[0097] It should be noted that after identifying the subset of key parameters that are strongly correlated with the data sample to be de-identified, a modification operation is required for this subset of key parameters. This modification operation aims to efficiently erase specific information about the data sample to be de-identified from the model, while preserving the general knowledge that the model has learned from all other data to the greatest extent possible, and ensuring the stability and perturbation of the modification process.

[0098] Specifically, to demonstrate the effect of this modification, this invention abandons gradient ascent and heuristic update methods, and instead constructs the model editing process as a multi-objective optimization problem, which does not directly optimize the model parameters. Instead, it is based on the parameter update amount. To optimize variables, and within a subset of key parameters To search in a low-dimensional space;

[0099] Furthermore, let's assume the original model parameters are as described above. The parameter update amount to be solved is The updated model parameters are Since this invention only modifies key parameters, the update amount for non-key parameters is zero. Therefore, it is only necessary to solve... That's it, this optimization problem. The definition is as follows:

[0100]

[0101] in, , , These are hyperparameters used to balance the weights of the three objective terms, and their ranges are as follows: , , ; Represented as an optimization function; This means the objective of the optimization problem is to minimize the amount of updates to a subset of key parameters. ;

[0102] Specifically, in one feasible implementation of the present invention, a grid search method is used to determine the optimal combination of hyperparameters as follows:

[0103] Reserve a small portion (e.g., 5%) of the original training dataset as a validation set. Note that this validation set does not participate in any other calculations. At the same time, select one or more samples from this validation set as simulated samples to be desensitized. Then, set a set of discrete initial values ​​for each hyperparameter. Iterate through all combinations of initial values ​​until a combination of initial values ​​has the highest accuracy on the validation set. This is the configuration value of the hyperparameter.

[0104] Specifically, Represented as the information forgetting term, it is obtained from the loss function of the target neural network model on the data samples to be de-identified. Its setting is to force the model to "forget" the data samples to be de-identified. In machine learning, a model "remembering" a sample typically manifests as a low predictive loss on that sample. Therefore, the most direct way to make it "forget" is to introduce new parameters into the model. Below, the de-identified data samples Making the worst possible prediction, i.e., maximizing its loss, is expressed as:

[0105]

[0106] in, This is represented as a desensitized target neural network model; It is the loss function L of the desensitized target neural network model on the data samples to be desensitized, which is equivalent to the loss function L in this case. That is, the updated model parameters Information forgetting in the data samples to be de-identified;

[0107] It should be noted that during the overall optimization of the model, this term is configured to maximize its loss. However, to facilitate a unified solution using gradient descent, we transform it into a minimization problem by adding a negative sign, i.e., minimizing... ;

[0108] Specifically, This term, denoted as the knowledge preservation term, is derived from the model's loss function on a dataset that does not contain the data samples to be desensitized. Its purpose is to ensure that when specific information is erased, the model does not impair its generalization ability learned on other irrelevant data, i.e., to prevent "catastrophic forgetting." Furthermore, for efficient computation, the present invention does not use all remaining training data, but rather... A calibration set of moderate size is formed by random sampling. That is, the model with new parameters The average loss on this calibration set is as follows:

[0109]

[0110] in, These are the updated model parameters. in calibration set The average loss across all samples;

[0111] It should be noted that in the overall optimization process of the model, if the model is to remain stable without using all the remaining training data, this term needs to be set as a minimized term.

[0112] Specifically, This is represented as a parameter regularization term, which is determined by the parameter update amount. The norm is obtained, and this setting is intended to limit the amount of parameter updates. The magnitude of the erase operation is important because an ideal erase operation should cause the smallest possible perturbation to the original model in the parameter space. Excessive parameter updates can lead to an unstable state or disrupt the fine structure of the model's internal feature representations. In one feasible implementation of this invention, the square of the L2 norm is used to ensure the differentiability and convexity of the model.

[0113]

[0114] It should be noted that this item is also a minimization item, used to encourage the optimizer to find a solution with minimal changes but the best effect;

[0115] Furthermore, by combining the above three terms (information forgetting term, knowledge retention term, and parameter regularization term), we can obtain the aforementioned composite optimization objective function, whose optimization variable is the parameter update amount. ;

[0116] Specifically, for the defined composite optimization objective function, this invention employs a gradient-based optimization algorithm (Adam optimizer) for iterative solution. First, the subset of key parameters to be optimized is updated. Initialize to a zero vector, i.e. Subsequently in each iteration ( Perform the following operations in )

[0117] Calculate the model parameters for the current iteration step. ,in, It is a with For parameters of the same dimension, the elements at non-critical parameter positions are always zero, while the elements at critical parameter subset positions are zero. ;

[0118] For forgotten information items, in Perform a forward propagation and calculate the loss. ;

[0119] For knowledge retention terms, from the calibration set Randomly sample a micro-batch Then, perform another forward propagation on top of that, and calculate the average loss. ;in, express yes A subset of;

[0120] Construct the overall objective function for the current iteration step and use automatic differentiation (backpropagation) to compute its function with respect to the optimization variables. gradient:

[0121]

[0122] in, Represented as gradient, Represented as the k-th iteration relative to The gradient is used for Adam optimizer updates;

[0123] It should be noted that, The gradient calculation ensures that the gradient is only propagated back to And the original model parameters Treating it as a constant greatly improves computational efficiency;

[0124] Using the Adam optimizer, based on the calculated gradient To update the subset of key parameters, we get:

[0125]

[0126] Furthermore, when the iteration reaches the preset maximum number of steps, or When the change is less than a certain threshold, the iteration operation terminates, and the optimal key parameter subset update amount is obtained. ;

[0127] Furthermore, the optimal subset of key parameters is applied to update the amount. By modifying the original target neural network model, we can obtain the desensitized target neural network model. It is important to emphasize that this modification is local and targeted.

[0128] For the subset of key parameters:

[0129]

[0130] in, Represented as a subset of the updated key parameters;

[0131] For the non-critical parameter subset :

[0132] (i.e., remain unchanged)

[0133] in, Represented as the updated subset of non-critical parameters ;

[0134] It should be noted that, through the above steps, a desensitized target neural network model can be obtained. While retaining most of the original knowledge, the model has removed significant imprints about the sample to be desensitized from the parameter level.

[0135] In addition, in order to objectively and comprehensively evaluate the effect of the desensitized target neural network model, this invention also designed a desensitization effect verification process. This process not only focuses on whether the information has been erased, but also on whether the model itself is still usable, and compares the desensitization effect with the theoretical optimal result.

[0136] Furthermore, to verify whether the data to be de-identified has been effectively removed from the perspective of potential attackers, this invention uses a Model Inversion Attack (MIA) method for verification. This method is abstracted as a member inference attack model, which is itself a binary classifier. The task of this model is to determine whether a sample belongs to the training set of the target model based on the output behavior (predicted probability vector) of the target model for a certain data sample.

[0137] Specifically, in one feasible implementation of the present invention, the member infers that the training data samples in the attack model are constructed as follows:

[0138] Half of the original training dataset is selected as positive samples (members), and the same number of samples are selected as negative samples (non-members) from an independent dataset that is not used to train the target model. These (positive and negative) samples are input into the target model, and the probability vector of its output is obtained and used as the training feature of the model. The corresponding member / non-member identity is used as the label to train the model.

[0139] Specifically, before the desensitization process, the original target neural network model is... Treatment of de-identified data samples The output (usually a predicted probability vector) is input into a pre-trained member inference attack model to obtain a modified member confidence score, since... Indeed, it is a member of the training set. An effective attack model will give a high confidence score (e.g., when the score range is [0, 1], the confidence score of the member before modification will be close to 1); subsequently, after the desensitization operation, the desensitized target neural network model... For the same data sample to be de-identified The output is input into the same member inference attack model to obtain a modified member confidence score. By comparing the member confidence score before and after modification, a successful desensitization operation should result in the modified member confidence score being significantly lower than the original member confidence score. Ideally, it should be close to the average confidence score of non-member samples. From the attacker's perspective, a significant decrease in confidence indicates... The behavior characteristics no longer resemble those of a member of the training set, and its privacy information has been successfully erased;

[0140] Furthermore, the model's generalization ability is evaluated to ensure that the desensitization process does not cause catastrophic damage to the model's general knowledge, i.e., the desensitized model remains usable.

[0141] Specifically, by preparing a separate test dataset containing data that has never been used in any training phase of the model, the target neural network model before modification is evaluated on this test dataset. And the modified desensitized target neural network model The performance of the model is evaluated by metrics that typically include, but are not limited to, accuracy, precision, recall, or F1 score. Similar to how member inference attack models determine whether a sample belongs to the training set of the target model, comparing the performance evaluation metrics before and after modification, a successful desensitization operation should result in a minimal decrease or essentially no change in the performance metrics of the desensitized model compared to the original model. For example, in an image classification task, if the decrease in model accuracy is controlled within an acceptable range or remains unchanged, it is sufficient to demonstrate that the model's generalization ability will not decrease while erasing specific information.

[0142] Furthermore, the desensitization method of the present invention is compared with the theoretical retraining from scratch method to quantify the fidelity of the present invention method;

[0143] Specifically, a baseline model is constructed by physically removing the data samples to be anonymized from the original training dataset to obtain a new training set. The model is then trained from scratch on this new training set using the exact same configuration as the original model (same model structure, initialization parameters, optimizer, learning rate, number of iterations, etc.). The resulting model is the baseline model. Subsequently, on the same independent test dataset, for each test sample, the outputs of the anonymized model and the baseline model are obtained. This output is typically the probability distribution vector of the last layer (Softmax layer) of the model. By calculating and comparing the statistical difference measure (such as KL divergence or JS divergence) between the two model output probability distributions, the average divergence of all samples on the entire test set can be used as the final fidelity index. The closer this index value is to zero, the more consistent the anonymized model obtained by the method of this invention is with the theoretical baseline model in predicting unknown data, thus proving the high fidelity of the method of this invention.

[0144] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0149] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A dynamic data de-sensitization method for sensitive field identification, the method being applied to a target neural network model trained by an original training data set containing a plurality of data samples, characterized in that, The method comprises the following steps: When receiving a de-sensitization request for at least one to-be-de-sensitized data sample in the original training data set, the influence degree of the to-be-de-sensitized data sample on each model parameter in the target neural network model is calculated, and a key parameter subset is determined according to the influence degree; Before receiving the de-sensitization request, a sensitive contribution degree identification step is further included, which comprises: In a federated learning framework composed of a central server and a plurality of data holder clients, a probe model is collaboratively trained; Each client uses the trained probe model to score the influence degree of the data samples locally held by it, and determines the data samples whose scores exceed a preset contribution degree threshold as the to-be-de-sensitized data samples; Based on an optimization objective function including a knowledge retention term, an information forgetting term and a parameter regularization term, a group of parameter update amounts are calculated only for the key parameter subset determined by the influence degree, and the key parameter subset is modified by applying the parameter update amounts to obtain a de-sensitized target neural network model.

2. The dynamic data desensitization method of sensitive field identification according to claim 1, wherein, The collaborative training process of the probe model comprises: Each client updates its probe model locally according to its local data and the loss gradient of the target neural network model; Each client sends the parameter change amount generated by local update to the central server through a secure aggregation protocol; The central server aggregates the parameter change amounts sent by all clients and updates a global probe model using the aggregation result.

3. The dynamic data desensitization method of sensitive field identification according to claim 1, wherein, The specific process of determining the key parameter subset comprises: The gradient of the loss function of the to-be-de-sensitized data sample on the target neural network model with respect to each model parameter is calculated; At the same time, an influence vector is generated, which is obtained by multiplying the gradient by a Hessian inverse matrix, wherein the Hessian inverse matrix is the inverse matrix of the Hessian matrix of the loss function of the target neural network model on the original training data set; The model parameters corresponding to the elements in the influence vector whose values exceed a preset threshold are screened out to form the key parameter subset.

4. The sensitive field identification dynamic data desensitization method of claim 3, wherein, The multiplication of the gradient and the Hessian inverse matrix is calculated through an iterative approximation process, wherein each iteration of the process comprises: A data subset is randomly sampled from the original training data set, and the vector product of the Hessian matrix and the result of the last iteration is calculated using the data subset.

5. The sensitive field identification dynamic data desensitization method of claim 1, wherein, In the optimization objective function: The knowledge retention term is obtained from the loss function of the target neural network model on a data set that does not include the to-be-de-sensitized data sample; The information forgetting term is obtained from the loss function of the target neural network model on the to-be-de-sensitized data sample, and in solving the optimization objective function, this term is configured as the term to be maximized; The parameter regularization term is obtained from the norm of the parameter update amount.

6. The sensitive field identification dynamic data desensitization method of claim 5, wherein, The knowledge retention term is calculated on a calibration set, which is obtained by sampling after excluding the to-be-de-sensitized data sample from the original training data set.

7. The sensitive field identification dynamic data desensitization method of claim 1, wherein, After obtaining the desensitized target neural network model, a desensitization effect verification step is further included, which comprises: Using a membership inference attack model, the membership confidence scores of the desensitized data samples before and after the target neural network model is modified are obtained respectively, and the membership confidence scores before and after the modification are compared.

8. The sensitive field identification and dynamic data desensitization method of claim 7, wherein, The desensitization effect verification step further comprises: On an independent test data set, the performance indicators of the target neural network model before and after modification are calculated respectively, and the performance indicators are compared.

9. The sensitive field identification dynamic data desensitization method of claim 8, wherein, The desensitization effect verification step further comprises: A benchmark model is constructed, which is obtained by retraining on a training set from which the desensitized data samples are removed; The statistical difference measure between the output probability distribution of the desensitized target neural network model and the benchmark model when processing the same input data is calculated and compared.

Images