Dynamic data desensitization method for sensitive field identification

Abstract

The invention discloses a dynamic data desensitization method for sensitive field identification, which comprises the following steps of: cooperatively training a probe model on a plurality of clients, and identifying a specific data sample containing sensitive information to be desensitized on the premise of not exposing original data on the basis of a federated learning framework; for the specific data sample, calculating the influence degree of the specific data sample on each parameter of the target neural network model, and screening out a key parameter subset which is most obviously influenced; constructing a multi-objective optimization function containing an information forgetting item, a knowledge keeping item and a regularization item, and only optimizing and updating a key parameter subset, so as to reversely eliminate the influence of a specific data sample in the model; the desensitization effect is verified by quantitatively evaluating the forgetting degree of the desensitized model to the specific data sample; according to the method, efficient and low-loss model data desensitization is achieved through accurate positioning and minimum editing, the method is particularly suitable for a federal learning environment with strict privacy requirements, a quantifiable safety verification mechanism is provided, and the efficiency, fidelity and credibility of data desensitization are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing under neural networks, and in particular to a dynamic data desensitization method for sensitive field identification. BACKGROUND

[0002] With the deep integration of artificial intelligence and big data technology, deep neural network models have been widely used in key fields such as finance, medical care, and security. In this integration process, model training inevitably involves and learns a large amount of highly sensitive data containing personal identity, property status, and biological characteristics. In recent years, with the increasing strictness of global data privacy regulations, in order to respond to compliance requirements and avoid data leakage risks, data desensitization technology has emerged. Traditional desensitization methods mainly act on the data preprocessing stage, and through anonymization, pseudonymization, and data shielding, etc. means to process before the data is stored in the training set. However, these methods are difficult to deal with the privacy leakage risk that may exist in the trained model, because the model parameters may have "remembered" the deep features of sensitive data, and can be reversed by means such as membership inference attack.

[0003] In order to solve the privacy problem at the model level, the academic and industrial circles have proposed the concept of Machine Unlearning. The most direct method is "retraining from scratch", that is, after deleting the data to be deleted from the training set, retraining the entire model, which is complete in effect but extremely high in computing cost, and is not feasible in the scene of complex model structure, large data scale, and frequent deletion requests. Therefore, some researchers have proposed some approximate retraining methods, such as a scheme based on data slicing aggregation, which trains multiple sub-models by dividing the data, and only re-trains the relevant sub-models when deleting, but this usually sacrifices the generalization performance of the whole model. In addition, some methods try to approximate the forgetting effect by applying reverse updates to the model parameters, however, these methods generally lack precise insight into the influence path of sensitive information in the model, and their update operations are often global or heuristic, which is extremely easy to cause catastrophic damage to other irrelevant knowledge learned by the model while erasing the target information. In addition, the current technology is mostly based on a pre-set centralized data environment, which is difficult to adapt to the collaborative identification of sensitive information in distributed or federated learning scenarios where data exists in the form of islands, and after the desensitization operation is completed, there is generally a lack of effective quantitative evaluation mechanism to verify from the attacker's perspective whether the information has been completely removed. SUMMARY

[0004] This section is intended to introduce some aspects of one or more embodiments of the present application, which are described below in the detail section. This section is not intended to limit the application in any way, but to provide insight into possible embodiments of the application.

[0005] The present application is proposed in view of the above-described existing problems. Therefore, the present application provides a sensitive field identification dynamic data de-sensitization method to solve the problems proposed in the background art.

[0006] To solve the above technical problems, the present application provides the following technical solutions: a sensitive field identification dynamic data de-sensitization method applied to a target neural network model trained by an original training data set containing multiple data samples, comprising the following steps:

[0007] When receiving a de-sensitization request for at least one data sample to be de-sensitized in the original training data set, the influence degree of the data sample to be de-sensitized on each model parameter in the target neural network model is calculated, and a key parameter subset is determined according to the influence degree;

[0008] Based on an optimization objective function containing a knowledge retention term, an information forgetting term and a parameter regularization term, a group of parameter update quantities is calculated only for the key parameter subset determined by the influence degree, and the key parameter subset is modified by applying the parameter update quantities to obtain a de-sensitized target neural network model.

[0009] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method of the present application, wherein: before receiving the de-sensitization request, a sensitive contribution degree identification step is further included, which comprises:

[0010] In a federated learning framework composed of a central server and multiple data holder clients, a probe model is collaboratively trained;

[0011] Each client uses the trained probe model to score the influence degree of the data samples locally held by it, and determines the data samples whose scores exceed a preset contribution degree threshold as the data samples to be de-sensitized.

[0012] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method of the present application, wherein: the collaborative training process of the probe model comprises:

[0013] Each client updates its probe model locally according to its local data and the loss gradient of the target neural network model;

[0014] Each client sends the parameter change amount generated by its local update to the central server through a secure aggregation protocol;

[0015] The central server aggregates the parameter change amounts sent by all clients and uses the aggregation result to update a global probe model.

[0016] As a preferred solution of the sensitive field identification dynamic data de-sensitization method, the specific process of determining the key parameter subset comprises:

[0017] The gradient of the loss function of the target neural network model on the to-be-de-sensitized data sample with respect to each model parameter is calculated.

[0018] Meanwhile, an influence vector is generated, which is obtained by multiplying the gradient by a Hessian inverse matrix, wherein the Hessian inverse matrix is the inverse matrix of the Hessian matrix of the loss function of the target neural network model on the original training data set.

[0019] The model parameters corresponding to the elements in the influence vector whose values exceed a preset threshold are screened out to form the key parameter subset.

[0020] As a preferred solution of the sensitive field identification dynamic data de-sensitization method, the multiplication of the gradient by a Hessian inverse matrix is calculated through an iterative approximation process, wherein each iteration of the process comprises:

[0021] A data subset is randomly sampled from the original training data set, and the vector product of the Hessian matrix and the result of the last iteration is calculated using the data subset.

[0022] As a preferred solution of the sensitive field identification dynamic data de-sensitization method, in the optimization objective function:

[0023] The knowledge preservation term is obtained from the loss function of the target neural network model on a data set that does not contain the to-be-de-sensitized data sample;

[0024] The information forgetting term is obtained from the loss function of the target neural network model on the to-be-de-sensitized data sample, and in solving the optimization objective function, the term is configured as a term to be maximized;

[0025] The parameter regularization term is obtained from the norm of the parameter update amount.

[0026] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method, wherein: the knowledge preservation term is calculated on a calibration set, and the calibration set is obtained by sampling after excluding the to-be-de-sensitized data sample from the original training data set.

[0027] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method, wherein: after obtaining the de-sensitized target neural network model, a de-sensitization effect verification step is further included, which comprises:

[0028] By using a member inference attack model, the member confidence scores of the to-be-de-sensitized data sample before and after the target neural network model is modified are obtained respectively, and the member confidence scores before and after the modification are compared.

[0029] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method, wherein: the de-sensitization effect verification step further comprises:

[0030] The performance indicators of the target neural network model before and after the modification are calculated on an independent test data set respectively, and the performance indicators are compared.

[0031] As a preferred scheme of the sensitive field identification dynamic data de-sensitization method, wherein: the de-sensitization effect verification step further comprises:

[0032] A benchmark model is constructed, and the benchmark model is obtained by retraining on a training set from which the to-be-de-sensitized data sample is removed;

[0033] The statistical difference measure between the output probability distribution of the de-sensitized target neural network model and the benchmark model when processing the same input data is calculated and compared.

[0034] Compared with the prior art, the beneficial effects of the present application are as follows:

[0035] 1、The present application can accurately locate the influence path of the to-be-de-sensitized data in the neural network model by introducing an interpretable machine learning technique, identify a key parameter subset strongly related thereto, and modify only the subset, and constrain it through a multi-objective optimization function containing knowledge preservation, information forgetting and parameter regularization, thereby fundamentally changing the blindness caused by global or heuristic updating in the prior art, efficiently erasing specific information while maximizing the original generalization ability of the model, and effectively avoiding catastrophic forgetting of irrelevant knowledge.

[0036] 2、And, in view of the limitations of the prior art mostly relying on centralized data environment, the application designs a sensitive contribution degree collaborative identification mechanism based on a federated learning framework, through safely collaboratively training a lightweight probe model, so that the high-risk sensitive data which has a great impact on the global model can be effectively identified without the data of each data holder leaving the local, solving the sensitive target identification problem in the data island environment, and enhancing the deployability and compliance of the model forgetting technology in real commercial scenarios.

[0037] 3、In addition, the application also introduces an adversarial verification mechanism from the perspective of the attacker, uses a member inference attack model to quantitatively evaluate the thoroughness of the removal of sensitive information, and at the same time, combines the performance evaluation of the model on an independent test set and the output distribution difference measurement with the "retrain from scratch" benchmark model, to provide a quantifiable objective basis for the success or failure of the desensitization operation, and ensure the authenticity and effectiveness of the desensitization. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings. Among them:

[0039] Figure 1 The overall flowchart of the dynamic data desensitization method for sensitive field identification according to an embodiment of the application. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned purposes, features and advantages of the application more apparent and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the application.

[0041] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the application, therefore the application is not limited to the specific embodiments disclosed below.

[0042] Second, the "one embodiment" or "an embodiment" referred to herein means a specific feature, structure, characteristic, or combination of features and / or characteristics described herein that can be included in at least one implementation of the present application. The various appearances of "in one embodiment" or "an embodiment" are not necessarily all referring to the same embodiment, although they can. Rather, they are used herein to connect features, structures, characteristics, or combinations of features, structures, or characteristics described herein with other features, structures, characteristics, or combinations of features, structures, or characteristics described herein that can or can not be present in

[0043] The present application is described in detail below in conjunction with the drawings, which show the device structure in cross-sectional view, which is partially enlarged without the general scale for the convenience of description, and the schematic diagram is only an example, which should not limit the scope of protection of the present application. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in actual production.

[0044] Meanwhile, in the description of the present application, it should be noted that the terms "upper, lower, inner and outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first, second or third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0045] Unless otherwise clearly specified and limited, the terms "mounting, connecting, connecting" in the present application should be understood broadly, for example: it can be fixed connection, detachable connection or integral connection; it can also be mechanical connection, electrical connection or direct connection, it can also be indirectly connected through intermediate medium, or it can be the communication between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0046] Embodiment 1

[0047] Reference Figure 1 For the first embodiment of the present application, the embodiment provides a dynamic data desensitization method for sensitive field identification, comprising:

[0048] It should be noted that the present application scheme is in an original training data set containing a plurality of data samples The target neural network model to be completed performed on, wherein, is the total parameter of the target neural network model , indicates that the parameter belongs to a p-dimensional real number space (p is the dimension of the total parameters of the model); and the target neural network model includes but is not limited to a convolutional neural network (CNN) for image classification, a recurrent neural network (RNN) or a Transformer model for sequence data processing, and a multi-layer perceptron (MLP) for table data, etc. In the present application, a ResNet-18 model for image classification task is used as the target neural network model.

[0049] S1, when receiving a de-sensitization request for at least one to-be-de-sensitized data sample in the original training data set, calculating the influence degree of the to-be-de-sensitized data sample on each model parameter in the target neural network model, and determining a key parameter subset according to the influence degree;

[0050] It should be noted that the purpose of this step is to, in response to the de-sensitization request for a specific to-be-de-sensitized data sample , through accurate quantitative analysis, identify the most significant part of the model parameter affected by the to-be-de-sensitized data sample , that is, to determine a key parameter subset , so as to avoid blind modification of the entire model; wherein in the to-be-de-sensitized data sample , represents the feature part of the model sample, represents the label part of the model sample;

[0051] In addition, before formally performing the de-sensitization operation, the present application actively discovers the target to be de-sensitized through a pre-posed sensitive contribution degree pre-identification stage, wherein the sensitive contribution degree pre-identification stage is carried out in a federated learning framework composed of a central server and a plurality of data holder clients, and a lightweight probe model is trained in collaboration by the data holder clients without exposing local data;

[0052] Specifically, in an implementable process of the present application, the probe model is a multi-layer perceptron (MLP) containing two hidden layers, the input layer dimension of which is consistent with the feature dimension of the data sample, the first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons, both of which use ReLU activation function, and the output layer is a neuron without using activation function, directly outputting the influence degree score;

[0053] Further, the central server first initializes a global probe model, which contains an initialized global probe model parameter, and sets the total communication round R of training, then the central server distributes the initialized global probe model parameter to all data holder clients participating in the training;

[0054] Specifically, in the t-th round of communication In the communication, each participating client receives the global probe model parameters After the (t-1)-th round of communication is the global probe model After the (t-1)-th round of communication), the following local update operations are performed:

[0055] Since the training target of the probe model is to learn to predict a data sample The potential impact on the target neural network model; therefore, in the present scheme, we quantify this impact through the loss gradient norm of the target neural network model on the sample, that is, the greater the loss gradient norm of a sample, the greater the contribution of the sample to the model parameter update; wherein, unlike the above-mentioned desensitization sample, The predicted data sample is learned by the predicted probe model, and for the same reason, represents the feature part of the predicted data sample, represents the label part of the predicted data sample;

[0056] For each sample in the local data set of the model, the client first calculates the loss gradient of the target neural network model on the sample , and obtains its L2 norm As the real impact score of the sample; wherein, is the loss function L of the target neural network model on the data sample with respect to the model parameters (loss gradient); is the real impact score of the data sample, is the L2 norm of the loss gradient ;

[0057] Subsequently, the client uses its local data sample and its corresponding calculated real impact score as labels to train the local probe model copy, and the optimization target is to minimize the difference between the predicted value of the probe model and the real impact score, in the minimization process, the present scheme uses the mean square error loss function, and then the client uses stochastic gradient descent (SGD) to optimize the mean square error loss function for several rounds, updates the local probe model parameters of the client, and obtains the updated probe model parameters ;

[0058] Specifically, the mean square error loss function is represented as:

[0059]

[0060] wherein, Here, c represents the mean squared error loss function of the probe model on the client side. For the client's local dataset, This is represented as the probe model parameters on the client side. Represented as probe model parameters The gradient relative to the local loss;

[0061] Furthermore, once local training is complete, each client calculates the change in its probe model parameters. That is, the difference between the locally updated parameters and the received global parameters. The data will be processed through a pre-agreed security aggregation protocol before being sent to the central server. This security aggregation protocol can be based on homomorphic encryption technology, where the client uploads encrypted parameter changes, or it can be based on differential privacy technology, where the client adds an appropriate amount of noise to the parameter changes before uploading.

[0062] It should be noted that, in order to protect the privacy of local data on each client, parameter changes cannot be uploaded directly in plaintext.

[0063] Furthermore, the central server waits for and collects the processed parameter changes sent from all clients;

[0064] Specifically, the central server performs aggregation operations. If additive homomorphic encryption is used, the central server will sum all encrypted parameter changes on the ciphertext. If a conventional federated averaging algorithm is used, the central server will perform a weighted average of the received (potentially noisy) parameter changes. Regardless of which method is used, the central server will use the aggregated parameter changes to update the global probe model.

[0065] Furthermore, the central server redistributes the updated global model parameters to all clients, starts the next round of training, and repeats the above local update operation until the set total number of communication rounds R of training is completed and the local update operation ends.

[0066] It should be noted that, through the above-mentioned federated learning collaborative training process, the present invention can obtain a global probe model that accurately assesses the impact of any data sample on the target model, and the entire process protects the data privacy of each client.

[0067] Furthermore, each client uses the trained probe model to score the degree of influence of the data samples it holds locally, and identifies the data samples with scores exceeding the preset contribution threshold as data samples to be de-identified.

[0068] Specifically, the results of the degree of influence scores of all data samples calculated locally by each client are sorted in descending order, and the top 1% is selected as the preset contribution threshold, that is, each client identifies the top 1% of the samples with the greatest local influence as the to-be-desensitized data samples;

[0069] Further, after ensuring the above data privacy, when receiving a desensitization request for the to-be-desensitized data sample , the key parameter subset can be determined without considering the data privacy problem;

[0070] Specifically, the step of determining the key parameter subset is as follows:

[0071] Calculate the gradient of the loss function of the to-be-desensitized data sample on the current target neural network model with respect to each parameter of the model, which represents the direction in which the model parameters need to be adjusted in order to better fit the to-be-desensitized data sample , which is mathematically expressed as:

[0072]

[0073] wherein, is the calculated loss gradient, denotes the derivative operation on the model parameter , and denotes the loss function of the model on , which can be cross-entropy loss or mean square error loss;

[0074] It should be noted that, from a technical principle, since the loss gradient points to the direction in which the model parameter space most quickly increases the loss, that is, it "remembers" the opposite direction of , but relying solely on the loss gradient for judgment is one-sided, because it only reflects the local information of itself and does not consider the interaction between this sample and all other training data, nor does it consider the coupling relationship between model parameters, so directly using the gradient may incorrectly evaluate the importance of model parameters;

[0075] To solve the above problem, the present application introduces the influence function (influence function) in the field of explainable machine learning to measure the influence from a more global and accurate perspective, wherein the influence function can estimate that if the trained is removed from the original training set , the model parameters What change will occur, the amount of change can be approximated by the following formula:

[0076]

[0077] where, is the impact vector, which is a vector with the same dimension as the original target neural network model parameters , whose j-th element directly quantifies the degree of influence of sample on the j-th model parameter ; is the Hessian matrix of the average loss function of the model on the entire original training data set , that is, the second-order partial derivative matrix of the loss function with respect to the parameters, which is defined as:

[0078]

[0079] where the Hessian matrix describes the local curvature of the loss function, reflecting the sensitivity of the model parameters, and in the foregoing, is the inverse matrix of the Hessian matrix, and is the product operation, which essentially corrects the local gradient direction through global curvature information to obtain a more accurate impact assessment;

[0080] However, for modern neural network models, the number of parameters can reach tens of millions or even higher, and directly calculating or storing a Hessian matrix to find its inverse is completely infeasible in terms of computation. To solve this computational bottleneck, the present solution employs an efficient iterative approximation algorithm (based on the idea of the second-order stochastic optimization algorithm (LiSSA)) to directly calculate the Hessian inverse matrix-vector product, i.e. without explicitly constructing or during the entire process. The iterative process of the algorithm is as follows:

[0081] Initialization: Set the number of iterations Q and initialize a vector ;

[0082] Iterative update: For , perform the following loop:

[0083] Randomly sample a data subset from the original training data set without replacement;

[0084] Calculate the Hessian matrix of the current batch and the last iteration result The product operation can be done by two automatic differentiations (i.e. two backpropagations) with only a few times of the computation cost of the regular gradient computation, completely avoiding the complexity of

[0085] The estimation of the Hessian inverse matrix-vector product is updated using the following formula:

[0086]

[0087] is the identity matrix. This update rule is essentially a stochastic approximation iteration for solving the linear system

[0088] Output: After Q iterations, the vector is obtained, which is a high-precision approximation of the influence vector is the estimation vector after Q iterations of LiSSA.

[0089] ​​​​​​​​​​​​​​​​​​​​​​​​​Further, when the approximate influence vector is obtained, the element value size in the influence vector intuitively reflects the importance of each parameter. Due to the high redundancy of the parameters of the neural network, most of the elements in the vector have small values, and therefore an influence threshold is set to filter out the model parameters corresponding to the elements in the influence vector whose absolute values exceed the threshold to form a final key parameter subset ;

[0090] Specifically, in an implementable process of the scheme, the influence threshold is set as the elements located in the top 1% positions after all the element absolute values in the influence vector are arranged in descending order, so as to ensure that the size of the key parameter subset is about 1% of the total parameter amount.

[0091] Specifically, the filtering process is obtained by a mathematical expression as follows:

[0092]

[0093] wherein, the influence threshold is represented as , the model parameters are represented as , the jth parameter in the model parameters is represented as , the jth element in the influence vector is represented as , any of the model parameters is represented as , and all the model parameters belong to . ;

[0094] It should be noted that the influence threshold can be selected by various strategies, for example, it can be a fixed value set according to experience, or it can be a dynamic value, such as selecting the minimum absolute value corresponding to the elements ranked in the top k% positions of all the element absolute values in the influence vector as the threshold.

[0095] It should be noted that by the above steps of determining the key parameter subset, the subsequent desensitization operation range can be reduced from the millions or even billions of parameters of the entire model to a key subset which is much smaller in size but highly related to the field sensitive information.

[0096] S2, based on an optimization objective function including a knowledge retention term, an information forgetting term and a parameter regularization term, a group of parameter update amounts are calculated only for the key parameter subset determined according to the influence degree, and the parameter update amounts are applied to modify the key parameter subset to obtain a desensitized target neural network model.

[0097] It should be noted that after identifying the key parameter subset strongly related to the to-be-desensitized data sample, a modification operation needs to be performed on the key parameter subset, which aims to both efficiently erase specific information about the to-be-desensitized data sample in the model and maximize the retention of general knowledge learned by the model from all other data, while ensuring the stability and perturbation of the modification process;

[0098] Specifically, in order to demonstrate the effect achieved by the modification operation, the present application discards the gradient ascent and heuristic update method, and instead constructs the editing process of the model as a multi-objective optimization problem, which is not directly optimizing the model parameters , but taking the parameter update amount as the optimization variable and searching in a low-dimensional space limited to the key parameter subset ;

[0099] Further, assuming that the original model parameters are the aforementioned , and the parameter update amount to be solved is , then the updated model parameters are , since only the key parameters are modified in the present application scheme, the update amount of the non-key parameters is zero, so only needs to be solved, and the optimization problem is defined as follows:

[0100]

[0101] wherein , , are hyperparameters used to balance the weights of the three objective terms, and their ranges are respectively: , , ; denotes the optimization function; denotes that the objective of the optimization problem is to minimize the key parameter subset update amount ;

[0102] Specifically, in an implementable process of the present application scheme, in order to determine the optimal combination of hyperparameters, a grid search method is used as follows:

[0103] A small part of data (for example, 5%) is reserved from the original training data set as a validation set, and it should be noted that the validation set does not participate in any other calculations, and one or more samples are selected from the validation set as simulated to-be-desensitized samples; then, a set of discrete initial values is set for each hyperparameter, and all combinations of initial values are traversed until a combination of initial values has the highest accuracy on the validation set, i.e., the configuration value of the hyperparameters;

[0104] Specifically, is the loss function of the target neural network model on the data sample to be de-biased, which is set to force the model to "forget" the data sample to be de-biased In machine learning, the model "remembers" a sample usually shows as a very low prediction loss on the sample, thus, to make it "forget", the most direct way is to let the model make as bad prediction as possible on the data sample to be de-biased, i.e. maximize its loss, denoted as:

[0105]

[0106] wherein, is the de-biased target neural network model; is the loss function L of the de-biased target neural network model on the data sample to be de-biased, which is equivalent to in this place, i.e. the updated model parameters on the data sample to be de-biased;

[0107] It should be noted that in the overall optimization process of the model, since this term is configured to maximize its loss, but in order to make it easy to use gradient descent method for unified solution, we convert it into a minimization problem by adding a negative sign, i.e. minimizing ;

[0108] Specifically, is the knowledge retention term, which is obtained by the loss function of the model on a data set that does not contain the data sample to be de-biased, and the purpose of setting it is to ensure that the model does not damage its generalization ability learned on other irrelevant data when erasing specific information, i.e. to prevent "catastrophic forgetting", and, in order to efficiently calculate, the present solution does not use all the remaining training data, but randomly samples from to form a calibration set with a moderate size , i.e. the average loss of the model on the calibration set under the new parameters :

[0109]

[0110] wherein, is the updated model parameters on the calibration set ;

[0111] It should be noted that in the overall optimization process of the model, if the model is to remain stable under the condition of not using all the remaining training data, this term needs to be set as a minimization term; ​​

[0112] Specifically, is expressed as a parameter regularization term, which is obtained by the norm of the parameter update , and the setting of this term aims to limit the amplitude of the parameter update , because an ideal erasing operation should be as small a disturbance as possible to the original model in the parameter space, and a too large parameter update amount may cause the model to enter an unstable state or destroy the fine structure of the internal feature representation of the model; in an implementable case of the present application, the square of the L2 norm is used to ensure the differentiability and convexity of the model:

[0113]

[0114] It should be noted that this term is also a minimized term for encouraging the optimizer to find a solution that changes the least but works best;

[0115] Further, after combining the above three terms (information forgetting term, knowledge preserving term, and parameter regularization term), the aforementioned composite optimization objective function can be obtained, and the optimization variable of this function is the parameter update ;

[0116] Specifically, for the defined composite optimization objective function, the present application uses a gradient-based optimization algorithm (Adam optimizer) to iteratively solve it, first initializing the key parameter subset update to be a zero vector, i.e. , and then performing the following operations in each iteration (t) :

[0117] Calculate the model parameters of the current iteration step, where is a parameter of the same dimension as , whose elements at non-key parameter positions are always zero, and whose elements at key parameter subset positions are ;

[0118] For the information forgetting term, perform a forward propagation on to calculate the loss ;

[0119] For the knowledge preserving term, randomly sample a mini-batch from the calibration set , and also perform a forward propagation on its basis to calculate the average loss ; where represents is a subset of ;

[0120] The total objective function of the current iteration step is constructed, and its gradient with respect to the optimization variables is calculated using automatic differentiation (backpropagation):

[0121]

[0122] where, denotes the gradient of , and denotes the gradient of the kth iteration with respect to , which is used for the Adam optimizer update;

[0123] It should be noted that, the gradient calculation ensures that the gradient is only returned to , while the original model parameters are treated as constants, greatly improving computational efficiency;

[0124] Using the Adam optimizer, the key parameter subset update is updated according to the calculated gradient , resulting in:

[0125]

[0126] Furthermore, when the iteration reaches the preset maximum number of steps, or the change in is less than a certain threshold, the iteration operation is terminated, and the optimal key parameter subset update is obtained;

[0127] Further, the optimal key parameter subset update is used to modify the original target neural network model, and the desensitized target neural network model is obtained. It should be particularly emphasized that this modification operation is local and targeted:

[0128] For the key parameter subset:

[0129]

[0130] where, denotes the updated key parameter subset;

[0131] For the non-key parameter subset :

[0132] (i.e., remains unchanged)

[0133] where, denotes the updated non-key parameter subset ;

[0134] It should be noted that through the above steps, a desensitized target neural network model is obtained , which retains most of the original knowledge while removing the significant imprint of the desensitized sample from the parameter level;

[0135] In addition, in order to objectively and comprehensively evaluate the effect of the desensitized target neural network model, the present application also designs a desensitization effect verification process, which not only focuses on whether the information is erased, but also focuses on whether the model itself is still available, and compares the desensitization effect with the optimal result in theory;

[0136] Further, from the perspective of potential attackers, whether the desensitized data has been effectively removed, the present application uses a model inversion attack (MIA) method for verification, which is abstracted as a member inference attack model, and the model itself is a binary classifier. The task of this model is to determine whether a certain data sample belongs to the training set of the target model according to the output behavior (predicted probability vector) of the target model on the sample;

[0137] Specifically, in an implementable process of the present application scheme, the training data samples in the member inference attack model are constructed as follows:

[0138] Half of the original training data set is selected as positive samples (members), and the same number of samples are selected from an independent data set that is not used to train the target model as negative samples (non-members). These (positive and negative) samples are input into the target model to obtain the output probability vector as the training features of the model, and the corresponding member / non-member identity is used as the label to train the model;

[0139] Specifically, before desensitization, the original target neural network model is used to process the desensitized data sample The output (usually the predicted probability vector) is input into a pre-trained member inference attack model to obtain a modified member confidence score. Since the desensitized data sample is indeed a training set member, an effective attack model will give a higher confidence score (for example, when the score range is [0, 1], the modified member confidence score will be close to 1); then, after desensitization, the desensitized target neural network model is used to process the same desensitized data sample The output is input into the same member inference attack model to obtain a modified member confidence score; by comparing the modified member confidence score with the modified member confidence score, a successful desensitization operation should make the modified member confidence score significantly lower than the modified member confidence score, and in an ideal case, it should be close to the average confidence score of the non-member sample, and a significant decrease in confidence from the perspective of the attacker indicates that The behavior characteristics of the member are no longer like those of the training set, and the privacy information has been successfully erased;

[0140] Further, the generalization ability of the model is evaluated to ensure that the desensitization process does not cause catastrophic damage to the general knowledge of the model, i.e. the desensitized model is still available;

[0141] Specifically, by preparing an independent test data set, the data in the data set has never been used in any training stage of the model, then on the test data set, the performance of the modified target neural network model And the desensitized target neural network model The performance evaluation index of the performance evaluation index generally includes but is not limited to accuracy (Accuracy), precision (Precision), recall (Recall) or F1 score; as the aforementioned member inference attack model judges whether the sample belongs to the training set of the target model, comparing the performance evaluation index before and after modification, then a successful desensitization operation should make the performance index of the desensitized model compared with the original model, the decline is extremely small or basically unchanged, for example, in an image classification task, the decline of the model accuracy is controlled within an acceptable range, or remains unchanged, which is sufficient to show that while erasing specific information, the generalization ability of the model remains unchanged;

[0142] Further, the desensitization method of the present application is compared with the theoretical retraining from scratch to quantify the fidelity of the present application method;

[0143] Specifically, a baseline model is constructed, i.e., the to-be-de- sensitized data samples are physically removed from the original training data set to obtain a new training set, and the same configuration (the same model structure, initialization parameters, optimizer, learning rate, iteration number, etc.) as that of training the original model is used to train the model from scratch on the new training set, and the final obtained model is the baseline model; subsequently, on the same independent test data set, for each test sample, the outputs of the de-sensitized model and the baseline model are obtained respectively, the output is usually the probability distribution vector output by the last layer (i.e., the Softmax layer) of the model, and by calculating and comparing the statistical difference measure (such as the KL divergence or the JS divergence) between the probability distribution outputs of the two models, the average divergence of all samples on the entire test set can be taken as the final fidelity index, and the closer the index value is to zero, the more consistent the to-be-de-sensitized model obtained by the method of the present application is with the theoretical baseline model in the prediction behavior on unknown data, thereby proving the high fidelity of the method of the present application.

[0144] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0145] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices generate a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0146] These computer program instructions can also be stored in a computer readable storage medium capable of guiding the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0147] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0148] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and variations can be made thereto without departing from the spirit and scope of the application. It is therefore intended that the appended claims cover all such modifications and variations as fall within the scope of the application.

[0149] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore intended that the application be covered within the scope of the claims, and that the scope of the claims be interpreted not to be limited by the embodiments set forth in the specification.

Claims

1. A dynamic data de-sensitization method for sensitive field identification, the method being applied to a target neural network model trained by an original training data set containing a plurality of data samples, characterized in that, The method comprises the following steps: When receiving a de-sensitization request for at least one to-be-de-sensitized data sample in the original training data set, the influence degree of the to-be-de-sensitized data sample on each model parameter in the target neural network model is calculated, and a key parameter subset is determined according to the influence degree; Based on an optimization objective function including a knowledge preservation term, an information forgetting term and a parameter regularization term, a set of parameter update amounts is calculated only for the key parameter subset determined according to the influence degree, and the parameter update amounts are applied to modify the key parameter subset to obtain a de-sensitized target neural network model.

2. The dynamic data desensitization method of sensitive field identification according to claim 1, wherein, Before receiving the de-sensitization request, a sensitive contribution degree identification step is further included, which comprises: In a federated learning framework composed of a central server and a plurality of data holder clients, a probe model is collaboratively trained; Each client uses the trained probe model to score the influence degree of the data samples locally held by it, and determines the data samples whose scores exceed a preset contribution degree threshold as the to-be-de-sensitized data samples.

3. The dynamic data desensitization method of sensitive field identification according to claim 2, wherein, The collaborative training process of the probe model comprises: Each client updates its probe model locally according to its local data and the loss gradient of the target neural network model; Each client sends the parameter change amount generated by local update to the central server through a secure aggregation protocol; The central server aggregates the parameter change amounts sent by all clients and updates a global probe model using the aggregation result.

4. The sensitive field identification dynamic data desensitization method of claim 1, wherein, The specific process of determining the key parameter subset comprises: The gradient of the loss function of the to-be-de-sensitized data sample on the target neural network model with respect to each model parameter is calculated; At the same time, an influence vector is generated, which is obtained by multiplying the gradient by a Hessian inverse matrix, wherein the Hessian inverse matrix is the inverse matrix of the Hessian matrix of the loss function of the target neural network model on the original training data set; The model parameters corresponding to the elements in the influence vector whose values exceed a preset threshold are screened out to form the key parameter subset.

5. The sensitive field identification dynamic data desensitization method of claim 4, wherein, The multiplication of the gradient by the Hessian inverse matrix is calculated through an iterative approximation process, wherein each iteration of the process comprises: A data subset is randomly sampled from the original training data set, and the vector product of the Hessian matrix and the result of the last iteration is calculated using the data subset.

6. The sensitive field identification dynamic data desensitization method of claim 1, wherein, In the optimization objective function: The knowledge preservation term is obtained from the loss function of the target neural network model on a data set that does not include the to-be-de-sensitized data sample; The information forgetting term is obtained from the loss function of the target neural network model on the to-be-de-sensitized data sample, and in solving the optimization objective function, this term is configured as the term to be maximized; The parameter regularization term is obtained from the norm of the parameter update amount.

7. The sensitive field identification dynamic data desensitization method of claim 6, wherein, The knowledge preservation term is calculated on a calibration set obtained by sampling after excluding the to-be-de-sensitized data sample from the original training data set.

8. The sensitive field identification and dynamic data desensitization method of claim 1, wherein, After obtaining the desensitized target neural network model, a desensitization effect verification step is further included, which comprises: Using a membership inference attack model, the membership confidence scores of the desensitized data samples before and after the target neural network model is modified are obtained respectively, and the membership confidence scores before and after the modification are compared.

9. The sensitive field identification and dynamic data desensitization method of claim 8, wherein, The desensitization effect verification step further comprises: On an independent test data set, the performance indicators of the target neural network model before and after modification are calculated respectively, and the performance indicators are compared.

10. The sensitive field identification and dynamic data desensitization method of claim 9, wherein, The desensitization effect verification step further comprises: A benchmark model is constructed, which is obtained by retraining on a training set from which the desensitized data samples are removed; The statistical difference measure between the output probability distribution of the desensitized target neural network model and the benchmark model when processing the same input data is calculated and compared.

Images