Long-tail identification data privacy protection method and system based on forgetting learning algorithm

Abstract

The invention discloses a long-tail identification data privacy protection method and system based on a forgetting learning algorithm, and the method comprises the steps: firstly obtaining an original long-tail identification data set, and obtaining a dynamic anchor point set of a forgetting data set and a non-forgetting data set through a deep neural network model; constructing a forgetting loss function based on the dynamic anchor point sets of the forgetting data set and the non-forgetting data set; and finally, based on a forgetting loss function, establishing a comprehensive model gradient to update deep neural network model parameters, and realizing forgetting of forgetting data by long-tail identification data. According to the method, the forgotten data set and the non-forgotten data set are processed by using the deep neural network model, the relevance between privacy information and the deep neural network model is eliminated, meanwhile, core features required in a long-tail recognition task are reserved, and the forgotten data prediction capability of the model is destroyed by constructing a forgotten loss function, so that the long-tail recognition efficiency is improved. And a comprehensive model gradient is established to update model parameters to realize data forgetting, and the generalization ability of the model is maintained while privacy information is protected.

Description

Technical Field

[0001] The present invention belongs to the field of information security, and in particular relates to a long-tail recognition data privacy protection method and system based on a forgetting learning algorithm. Background Art

[0002] In the field of image classification in the real world, categories often exhibit the characteristics of long-tail distribution. For example, in the recognition and classification of natural scene images, the image data of common object categories such as people and cars are extremely rich, while the image data of some rare object categories such as rare animals and specific ancient cultural relics are relatively scarce. Since the model is exposed to more category samples with large amounts of data during the training process, the features and classification rules it learns will tend to these majority class image samples. As a result, when faced with minority class image data, the recognition accuracy of the model will be greatly reduced, and it will be unable to effectively classify and identify all categories in the data set, thereby limiting the widespread application and precise implementation of related technologies in actual complex scenarios. Therefore, in order to overcome this dilemma, long-tail recognition technology came into being.

[0003] Long-tail identification aims to solve classification problems in situations where data distribution is uneven. However, the data used for long-tail identification often contains sensitive information. For example, in the medical field, long-tail disease identification involves collecting patient case data to identify rare diseases. This case data contains private information such as the patient's personal identity information and medical history. Long-tail identification technology is particularly vulnerable to various security and privacy attacks. Therefore, protecting the privacy information contained in long-tail identification data is a scientific issue that must be addressed in the field of machine learning security.

[0004] To protect the privacy information contained in long-tail recognition data, several data forgetting algorithms have been studied. However, these data forgetting algorithms are specific to the case where there are a sufficient number of forgetting data samples. However, the number of data samples in long-tail recognition data is uneven, and there are rare object categories with very few samples. Therefore, developing a forgetting algorithm that is suitable for the case of insufficient forgetting data samples is an urgent problem to be solved in long-tail recognition technology. Summary of the Invention

[0005] The purpose of the present invention is to address the above-mentioned problems existing in the prior art and to provide a forgetting algorithm for the situation where the number of forgotten data samples is insufficient, and a long-tail recognition data privacy protection method and system based on the forgetting learning algorithm.

[0006] To achieve the above objectives, the technical solutions of the present invention are as follows:

[0007] In a first aspect, the present invention proposes a long-tail recognition data privacy protection method based on a forgetting learning algorithm, comprising:

[0008] S1. Obtain an original long-tail recognition dataset and use a deep neural network model to obtain a dynamic anchor point set of a forgotten dataset and a non-forgotten dataset, wherein the original long-tail recognition dataset includes a forgotten dataset that requires privacy protection and a normal non-forgotten dataset;

[0009] S2. Construct a forgetting loss function based on a dynamic anchor point set of the forgotten dataset and the non-forgotten dataset;

[0010] S3. Based on the forgetting loss function, a comprehensive model gradient is established to update the deep neural network model parameters to achieve the forgetting of the long-tail recognition data for the forgotten data.

[0011] Said S1 comprises:

[0012] S11. Input all categories of the forgotten and non-forgotten datasets into the feature extractor of the deep neural network model, and use the following formula to extract the deep feature vectors of each category in the forgotten and non-forgotten dataset samples:

[0013] z i =E(x i );

[0014] z c =E(x c );

[0015] In the above formula, z i is the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...z i，L}, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K};

[0016] S12. Take the mean of all deep feature vectors of each category as the initial anchor point, and use the following formula to calculate the temporary anchor points of each category in the forgotten dataset and the non-forgotten dataset samples:

[0017]

[0018] In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the initial anchor point of category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c, is the initial anchor point of category c;

[0019] S13. Weighted fusion of temporary anchor points of each category with the initial anchor points to obtain dynamic anchor points of each category in the forgotten and non-forgotten dataset samples:

[0020]

[0021] In the above formula, a i is the dynamic anchor point of category i in the forgetting dataset, β is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset;

[0022] S14. Traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and obtain the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,...,a f}, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c ,...,a r}.

[0023] The forgetting loss function of S2 is:

[0024]

[0025] In the above formula, is the forgetting loss function, is the similarity loss function of the forgotten dataset, λ is the weight hyperparameter that balances the contribution of the two types of losses, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset, P i For p ic The probability matrix composed of is the forgotten data x of category i i The predicted probability matrix obtained by forward propagation of the deep neural network model, r is the total number of categories of the non-forgetting dataset, p ic is the probability distribution that category i of the forgotten dataset belongs to category c of the non-forgotten dataset, For pic The predicted probability matrix of forward propagation, a c is the dynamic anchor of category c in the non-forgetting dataset.

[0026] The S3 includes:

[0027] S31, calculating the forgetting gradient and memory gradient of the forgotten dataset relative to the deep neural network model, wherein the forgetting gradient is calculated based on the forgetting loss function;

[0028] S32. Calculate the cosine similarity of the forgetting gradient and the memory gradient, and determine whether the cosine similarity is negative. If the cosine similarity of the forgetting gradient and the memory gradient is negative, perform an orthogonal adjustment on the forgetting gradient, and linearly superimpose the orthogonalized forgetting gradient and the memory gradient to form a comprehensive model gradient. If the cosine similarity of the forgetting gradient and the memory gradient is not negative, the forgetting gradient remains unchanged, and the forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient.

[0029] S33. Using the integrated model gradient, update the deep neural network model parameters using the following formula:

[0030] w′=w-η·g;

[0031] In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient;

[0032] S34. The deep neural network model achieves forgetting of forgotten data based on the updated model parameters.

[0033] The forgetting gradient of S31 is calculated using the following formula:

[0034]

[0035] In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function;

[0036] The memory gradient of S31 is calculated using the following formula:

[0037]

[0038] In the above formula, g r is the memory gradient, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y jis the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample;

[0039] The comprehensive model gradient of S32 is expressed by the following formula:

[0040]

[0041] In the above formula, g′ f is the forgotten gradient after orthogonal adjustment, cos(g f , g r ) is the cosine similarity between the forget gradient and the memory gradient.

[0042] In the second aspect, the present invention proposes a long-tail recognition data privacy protection system based on the forgetting learning algorithm, including a dynamic anchor point set acquisition module, a forgetting loss function construction module, and a model parameter update module;

[0043] The dynamic anchor point set acquisition module is used to obtain an original long-tail recognition dataset and obtain a dynamic anchor point set of a forgotten dataset and a non-forgotten dataset using a deep neural network model, wherein the original long-tail recognition dataset includes a forgotten dataset that requires privacy protection and a normal non-forgotten dataset;

[0044] The forgetting loss function construction module is used to construct a forgetting loss function based on a dynamic anchor point set of a forgetting dataset and a non-forgetting dataset;

[0045] The model parameter updating module is used to establish a comprehensive model gradient based on the forgetting loss function to update the deep neural network model parameters, so as to realize the forgetting of the long-tail recognition data to the forgotten data.

[0046] The dynamic anchor point set acquisition module includes a deep feature vector extraction unit, a temporary anchor point calculation unit, a dynamic anchor point weighting unit, and a dynamic anchor point set unit;

[0047] The deep feature vector extraction unit is used to input all categories of the forgotten dataset and the non-forgotten dataset into the feature extractor of the deep neural network model, and extract the deep feature vectors of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula:

[0048] z i =E(x i );

[0049] z c =E(x c );

[0050] In the above formula, z iis the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...z i，L}, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K};

[0051] The temporary anchor point calculation unit is used to take the mean of all deep feature vectors of each category as the initial anchor point, and calculate the temporary anchor points of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula:

[0052]

[0053] In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the initial anchor point of category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c, is the initial anchor point of category c;

[0054] The dynamic anchor point weighting unit is used to weight the temporary anchor points of each category and the initial anchor points to obtain the dynamic anchor points of each category in the forgotten dataset and the non-forgotten dataset samples:

[0055]

[0056] In the above formula, a i is the dynamic anchor point of category i in the forgetting dataset, β is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset;

[0057] The dynamic anchor point set unit is used to traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and obtain the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,...,a f}, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c,...,a r}.

[0058] The forgetting loss function of the forgetting loss function building module is:

[0059]

[0060]

[0061] In the above formula, is the forgetting loss function, is the similarity loss function of the forgotten dataset, λ is the weight hyperparameter that balances the contribution of the two types of losses, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset, P i For p ic The probability matrix composed of is the forgotten data x of category i i The predicted probability matrix obtained by forward propagation of the deep neural network model, r is the total number of categories of the non-forgetting dataset, p ic is the probability distribution that category i of the forgotten dataset belongs to category c of the non-forgotten dataset, For p ic The predicted probability matrix of forward propagation, a c is the dynamic anchor of category c in the non-forgetting dataset.

[0062] The model parameter updating module includes a gradient calculation unit, a comprehensive model gradient forming unit, a model parameter updating unit, and a data forgetting unit;

[0063] The gradient calculation unit is used to calculate the forgetting gradient and memory gradient of the forgotten data set relative to the deep neural network model, and the forgetting gradient is calculated based on the forgetting loss function;

[0064] The comprehensive model gradient forming unit is used to calculate the cosine similarity of the forgetting gradient and the memory gradient, and determine whether the cosine similarity is negative. If the cosine similarity of the forgetting gradient and the memory gradient is negative, the forgetting gradient is orthogonalized and adjusted, and the orthogonalized forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient; if the cosine similarity of the forgetting gradient and the memory gradient is not negative, the forgetting gradient remains unchanged, and the forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient;

[0065] The model parameter updating unit is used to update the deep neural network model parameters using the following formula using the comprehensive model gradient:

[0066] w′=w-η·g;

[0067] In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient;

[0068] The data forgetting unit is used in the deep neural network model to forget the forgotten data based on the updated model parameters.

[0069] The forgetting gradient of the gradient calculation unit is calculated using the following formula:

[0070]

[0071] In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function;

[0072] The memory gradient of the gradient calculation unit is calculated using the following formula:

[0073]

[0074] In the above formula, g r is the memory gradient, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y j is the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample;

[0075] The comprehensive model gradient of the comprehensive model gradient forming unit is expressed by the following formula:

[0076]

[0077] In the above formula, g′ f is the forgotten gradient after orthogonal adjustment, cos(g f , g r ) is the cosine similarity between the forget gradient and the memory gradient.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. This invention proposes a method and system for long-tail recognition data privacy protection based on a forgetting learning algorithm. This method first obtains an original long-tail recognition dataset and, using a deep neural network model, obtains a dynamic anchor set for the forgotten and non-forgotten datasets. The original long-tail recognition dataset contains both the forgotten and normal datasets. A forgetting loss function is then constructed based on the dynamic anchor set for the forgotten and non-forgotten datasets. Finally, based on the forgetting loss function, a comprehensive model gradient is established to update the deep neural network model parameters, achieving forgetting of the long-tail recognition data for the forgotten data. This method uses the deep neural network model to extract representative features of the data as dynamic anchor points, capturing more discriminative semantic information. This method can continuously reflect dynamic changes in data distribution, enhance the ability of subsequent steps to accurately locate private data, and provide a reusable quantitative benchmark for forgetting operations targeting specific categories. This avoids the resource consumption of repeatedly calculating anchor points, significantly improving the algorithm's execution efficiency. While weakening the relevance of the target private data, it also reduces interference with non-private data, maintaining the stability of the model's overall performance.

[0080] 2. The present invention proposes a long-tail recognition data privacy protection method and system based on the forgetting learning algorithm. This method forms a forgetting loss function by weighted fusion of similarity loss and probability distribution loss, which destroys the deep neural network model's ability to predict forgotten data, synergistically weakens the model's predictive relevance for private data, optimizes the model's ability to discriminate between similar categories, and avoids the model performance imbalance problem caused by single loss optimization. In actual scenarios such as medical imaging and biometrics, this method significantly improves the systematicity of privacy protection and the robustness of model updates.

[0081] 3. The present invention proposes a long-tail recognition data privacy protection method and system based on a forgetting learning algorithm. This method linearly superimposes the forgetting gradient and the memory gradient to form a comprehensive model gradient. The gradient orthogonalization technique is used to eliminate the directional conflict between the forgetting gradient and the memory gradient. The optimization direction is dynamically adjusted according to the relationship between the forgetting target and the similarity category. While eliminating the correlation of private information, it avoids model performance fluctuations caused by gradient conflicts, ensures that the forgetting operation of private data during the model update process does not affect the discrimination accuracy of similar categories, and improves the stability of the forgetting algorithm in scenarios with high inter-class similarity. It can not only achieve efficient forgetting of sensitive data, but also ensure the model's robustness in recognizing similar features, thereby enhancing the synergistic effect of privacy protection and model practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0083] Figure 2 1 is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0084] The present invention will be further described in detail below with reference to specific embodiments and the accompanying drawings.

[0085] The present invention proposes a long-tail recognition data privacy protection method and system based on a forgetting learning algorithm. The method obtains the original long-tail recognition data set, applies a deep neural network model and a dynamic anchor selection algorithm to process the forgotten data set and the non-forgotten data set to eliminate the correlation between privacy information and the deep neural network model, while retaining the core features required in the long-tail recognition task; constructs a forgetting loss function that combines similarity loss with a probability distribution mechanism to destroy the model's predictive ability for forgotten data; adopts gradient orthogonalization technology to eliminate the negative impact of forgetting gradients and memory gradients on similar categories in the comprehensive model gradient; and realizes data forgetting by updating model parameters through cross-entropy loss and gradient optimization, thereby maintaining the generalization ability of the model while protecting privacy information.

[0086] Example 1:

[0087] like Figure 1 As shown in the figure, the long-tail recognition data privacy protection method based on the forgetting learning algorithm is carried out in the following steps:

[0088] 1. Obtain the original long-tail recognition dataset and use the deep neural network model to obtain the dynamic anchor point set of the forgotten dataset and the non-forgotten dataset;

[0089] The original long-tail recognition dataset is represented as Include The data images of each category have the same label. The original long-tail recognition dataset contains the forgotten dataset and the data images that need to be protected. In the original long-tail recognition dataset, the data images of a specific category containing privacy information are defined as the forgotten dataset, which is expressed as D f ={x f ,y f}, x f is the pixel information of the forgotten data, y f is the label of the forgotten data; the category data image that does not contain private information is defined as a normal non-forgotten dataset, denoted as D c ={x c ,y c}, x c is the pixel information of the non-forgotten data, y c is the label of the non-forgotten data;

[0090] For each category in the forgotten and non-forgotten datasets, feature data is extracted from the last convolutional layer of the deep neural network model. The feature information of the dynamic anchor is captured by the output of the deep convolutional layer of the model, which can effectively represent the abstract semantic features of the category.

[0091] The deep neural network model is The structure before the middle convolutional layer of the deep neural network model is the feature extractor E. All categories of the forgotten dataset and the unforgotten dataset are input into the feature extractor of the deep neural network model. The following formula is used to extract the deep feature vectors of each category in the samples of the forgotten dataset and the unforgotten dataset:

[0092] z i =E(x i );

[0093] z c =E(x c );

[0094] In the above formula, z i is the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...z i，L}, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K};

[0095] In the initial stage, a cold start strategy is adopted to directly use the mean of all deep feature vectors of each category as the initial anchor point to ensure the stability and representativeness of the anchor point in the early stage and avoid the anchor point offset problem caused by initial data fluctuations:

[0096]

[0097] In the above formula, is the initial anchor point of category i, L is the total number of categories in the forgotten dataset, z i，l is the lth deep feature vector of category i, is the initial anchor point of category c, K is the total category book of the non-forgetting dataset, z c，k is the k-th deep feature vector of category c;

[0098] The following formula is used to calculate the temporary anchor points of each category in the samples of the forgotten dataset and the non-forgotten dataset, and the feature vector with the highest cosine similarity with the initial anchor point in each category is selected as the temporary anchor point:

[0099]

[0100] In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c;

[0101] Using the momentum smoothing update strategy, the temporary anchor points of each category are weightedly fused with the initial anchor points according to the momentum coefficient to obtain the dynamic anchor points of each category in the forgotten dataset and the non-forgotten dataset samples to reflect the dynamic changes of the data distribution:

[0102]

[0103] In the above formula, a i is the dynamic anchor point of category i in the forgetting dataset, β is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset;

[0104] Traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and get the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,...,a f}, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c ,...,a r The dynamic anchor point dataset contains the anchor point features obtained by the dynamic anchor point selection method and the original input image data corresponding to the anchor point;

[0105] Before implementing the forgetting algorithm, by extracting representative features from the last convolutional layer of the deep neural network model as dynamic anchor points, more discriminative semantic information can be captured, which can continuously reflect the dynamic changes of data distribution, enhance the forgetting algorithm's ability to accurately locate private data, and provide a reusable quantitative benchmark for subsequent forgetting operations for specific categories. It avoids the resource consumption of repeated calculation of anchor points, significantly improves the algorithm execution efficiency, and reduces interference with non-private data while weakening the correlation of target private data, maintaining the stability of the overall model performance. Especially when processing large-scale long-tail data sets, it can quickly respond to multi-category privacy protection needs and enhance the practical deployment feasibility of the algorithm.

[0106] 2. Construct a forgetting loss function based on a dynamic anchor set of the forgotten dataset and the non-forgotten dataset;

[0107] First, based on the cosine similarity between the deep feature vectors of each category in the forgotten dataset and the dynamic anchor points, a similarity loss function is constructed. By minimizing the average cosine similarity of all forgotten data samples, the prediction correlation of the deep neural network model on the forgotten data is weakened:

[0108]

[0109] In the above formula, is the similarity loss function of the forgotten dataset, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset;

[0110] In the optimization process of the deep neural network model, the similarity loss function is calculated by back propagation Relative to the gradient of the deep neural network model parameters, adjust the weights of the feature extractor E and the classifier so that the deep feature vector z of the forgotten dataset i Gradually moving away from its original category anchor point a in the vector space i , effectively reducing the correlation between the forgotten data features and their category anchors, thereby destroying the model's ability to remember private information;

[0111] Then, a probability allocation mechanism is introduced to generate prediction probabilities that conform to the data distribution, ensuring that the prediction probabilities of the forgotten data are reasonably distributed;

[0112] Calculate the cosine similarity between the deep feature vectors of each category of the forgotten dataset and the dynamic anchor points of the non-forgotten dataset, and generate the probability distribution of the forgotten data belonging to each non-forgotten category:

[0113]

[0114] In the above formula, p ic is the probability distribution that the category i of the forgotten dataset belongs to the category c of the non-forgotten dataset, z i is the deep feature vector of category i in the forgotten dataset, a c is the dynamic anchor of category c in the non-forgetting dataset, especially p ic = 0, thus obtaining the probability matrix P of category i in the forgotten data set for all categories in the non-forgotten data set i ={p i1 ,...,p ic ,...,p ir};

[0115] The relative entropy (Kullback-Leibler divergence, KL divergence) is used to measure the difference between the original predicted probability distribution of the deep neural network model for the forgotten data and the target probability distribution, and a probability distribution loss function is constructed;

[0116] The forgotten data x of category i in the forgotten dataset will be i The prediction probability matrix is ​​obtained through forward propagation of the deep neural network model And calculate the difference through KL divergence, the formula is:

[0117]

[0118] Then construct the probability distribution loss function, the formula is:

[0119]

[0120] In the above formula, P i For p ic The probability matrix composed of is the forgotten data x of category i i The prediction probability matrix obtained by forward propagation of the deep neural network model, For p ic The predicted probability matrix of the forward propagation, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i;

[0121] By minimizing the probability distribution loss function Force the deep neural network model's prediction distribution of forgotten data to tend towards the target probability distribution, eliminating its prediction preference for the privacy category of the original forgotten data while retaining the reasonable discrimination logic for similar categories;

[0122] Finally, the parameters of the deep neural network model are updated by constructing a total forgetting loss that combines the similarity loss function with the probability assignment loss function:

[0123]

[0124] In the above formula, is the forgetting loss function, and λ is the weight hyperparameter that balances the contribution of the two types of losses;

[0125] By weightedly fusing similarity loss and probability distribution loss, a total forgetting loss function is formed to undermine the deep neural network model's ability to predict forgotten data, synergistically weaken the model's predictive relevance for private data, and optimize the model's ability to discriminate between similar categories. This avoids the model performance imbalance caused by single-loss optimization, significantly improving the systematicity of privacy protection and the robustness of model updates in practical scenarios such as medical imaging and biometrics.

[0126] Long-tail recognition has broad application prospects in multiple fields, including network security, intelligent security, and biometrics, and is of great significance in optimizing management and services. In biometric recognition systems, fingerprint or iris recognition systems can replace traditional clock-in systems to achieve more efficient and accurate attendance management. In intelligent security scenarios, long-tail recognition technology can ensure that only authorized personnel can enter specific areas.

[0127] Long-tail recognition technology also faces some security challenges. For example, in biometric systems, as time goes by and technology develops, new types of artificial fingerprints or fake iris technologies continue to emerge. These are minority-class abnormal data, while the real biometric data of normal users is the majority-class data. The system needs to be updated in a timely manner to adapt to new security threats. However, due to the scarcity and uniqueness of this data, traditional model update mechanisms may not be able to quickly and effectively weaken the model's memory ability for fake data. By introducing similarity loss, the model's relevance to abnormal private data can be accurately quantified, minimizing its feature similarity with dynamic anchor points, thereby destroying the model's judgment logic for fake data; at the same time, combined with a probability distribution mechanism, the similarity between abnormal data and other legitimate categories is normalized into a probability distribution, forcing the model prediction to tend towards a uniform distribution and eliminating the preference for fake categories;

[0128] Large-scale model updates for the entire biometric recognition system may face issues such as high costs and system compatibility. In intelligent security systems, the majority of image data consists of normal scenes of people and environments, belonging to the majority class. However, some special cases, such as illegal intruders disguised as normal people wearing special camouflage clothing or attackers using adversarial sample technology to interfere with surveillance cameras, constitute long-tail data. During the long-tail recognition process, due to the presence of data noise and adversarial samples, and their small proportion in the dataset, the model may not fully learn how to effectively distinguish this special long-tail data from normal data during training. By coordinating the model optimization direction through gradient orthogonalization technology, while eliminating the correlation of adversarial samples and retaining the ability to distinguish normal data, it can effectively prevent illegal intruders from using disguised or adversarial samples to break through the recognition defenses of the monitoring system and avoid threatening the safety of people and property. Therefore, combining similarity loss with probability distribution mechanism in long-tail recognition technology to protect data privacy is extremely important. It can not only respond to new security threats but also reduce the cost and complexity of model updates.

[0129] 3. Based on the forgetting loss function, a comprehensive model gradient is established to update the parameters of the deep neural network model, realizing the forgetting of the long-tail recognition data for the forgotten data;

[0130] When the deep neural network model performs the forgetting operation, it calculates the gradient of the forgetting loss function relative to the model parameters through backpropagation to obtain the forgetting gradient:

[0131]

[0132] In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function;

[0133] Since the forgotten gradient may interfere with the decision boundary of the deep neural network model for distinguishing similar categories, in order to solve this problem, the deep neural network model's discriminative knowledge of similar categories is retained by constructing a memory gradient:

[0134] Set the probability threshold to ∈, and forget the probability matrix P of each category of the dataset i In the above example, we filter out the non-forgotten category set that is highly similar to the forgotten data and record it as the similar category set Y sim =D j ={c|p ic >∈}, and extract similar category set D j Pixel information of dynamic anchor points corresponding to non-forgotten categories (i.e. dynamic anchor point a j Corresponding original data), build a memory data set Among them, y j is the true category label of the dynamic anchor sample;

[0135] The memorized dataset Input the deep neural network model, calculate the cross entropy loss of the memory data set, and obtain the memory gradient through back propagation. The formula is:

[0136]

[0137] In the above formula, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y j is the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample, g r is the memory gradient;

[0138] Since the forgotten gradient g f and memory gradient g r The two optimization goals are often in conflict with each other, so the cosine similarity of the gradient is used to coordinate the two optimization goals. The gradient orthogonalization technique is combined to eliminate the directional conflict between the forgetting gradient and the memory gradient, and dynamically coordinate the update direction of the model parameters. The formula is:

[0139]

[0140] If cos(g f , g r )≥0, the forgetting gradient remains unchanged. If cos(g f , g r )<0, indicating whether there is a conflicting component between the two gradients, then the gradient g will be forgotten f In the memory gradient g r The component in the direction is removed, that is, the forgotten gradient is orthogonalized to obtain g′ f , the formula is:

[0141]

[0142] The forget gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient so that the two gradients no longer conflict:

[0143]

[0144] Using the integrated model gradient, the deep neural network model parameters are updated using the following formula:

[0145] w′=w-η·g;

[0146] In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient;

[0147] The deep neural network model forgets the forgotten data based on the updated model parameters;

[0148] There are many types of deep neural network models. Taking the common ResNet network as an example, when using common optimizers such as torch.optim.SGD or torch.optim.Adam, the parameters passed in using model.parameters() will be registered with the optimizer. These parameters usually include: weights and biases of convolutional layers (Conv Layers), weights and biases of fully connected layers (Linear Layers); if other model structures are defined, the parameters of these structures will also be passed in. The most common is to update the weights and biases of the convolutional layers and the weights and bias parameters of the fully connected layers in the deep neural network;

[0149] Gradient orthogonalization technology is used to eliminate the directional conflict between the forgetting gradient and the memory gradient, and the optimization direction is dynamically adjusted according to the relationship between the forgetting target and the similar categories. While eliminating the correlation of private information, it avoids model performance fluctuations caused by gradient conflicts and ensures that the forgetting operation of private data during the model update process does not affect the discrimination accuracy of similar categories. This improves the stability of the forgetting algorithm in scenarios with high inter-class similarity, achieves efficient forgetting of sensitive data, and ensures the model's robustness in recognizing similar features, thereby enhancing the synergistic effect of privacy protection and model practicality.

[0150] 4. Test the classification accuracy of the deep neural network model on forgotten data and the success rate of membership inference attacks to verify the model's effect on forgetting private information;

[0151] The method for determining the protection of privacy information in long-tail identification data is as follows: the classification accuracy and the membership inference attack success rate are used to test the prediction ability of the deep neural network model for forgotten data. When the classification accuracy and the membership inference attack success rate are both lower than the preset threshold, it is determined that the privacy protection is successful and the deep neural network model has effectively eliminated the relevance of the target private data.

[0152] Specifically, the classification accuracy is used to measure the degree of degradation of the deep neural network model's predictive ability for forgotten data, which is defined as the model's performance on the forgotten dataset D. f The proportion of the number of correctly predicted samples to the total number of samples is as follows:

[0153]

[0154] When the accuracy rate is lower than the preset threshold of 5%, it indicates that the deep neural network model cannot effectively identify the original category of the forgotten data, and the relevance of private information has been significantly weakened;

[0155] The membership inference attack success rate is used to detect the risk of residual memory of forgotten data in deep neural network models. It is defined as the probability that an attacker successfully infers whether a forgotten data sample belongs to the original training set. That is, the attacker uses a shadow model or a confidence threshold method to infer the difference in the predicted output of the model between the forgotten data and external non-member data. The formula is:

[0156]

[0157] When the success rate of the member inference attack is lower than the preset threshold of 10%, it indicates that the memory traces of the private data in the deep neural network model have been effectively eliminated, and the attacker cannot infer the data ownership by predicting the output.

[0158] Example 2:

[0159] like Figure 2 As shown in the figure, the long-tail recognition data privacy protection system based on the forgetting learning algorithm includes a dynamic anchor set acquisition module, a forgetting loss function construction module, and a model parameter update module;

[0160] The dynamic anchor point set acquisition module is used to obtain an original long-tail recognition dataset and obtain a dynamic anchor point set of a forgotten dataset and a non-forgotten dataset using a deep neural network model, wherein the original long-tail recognition dataset includes a forgotten dataset that requires privacy protection and a normal non-forgotten dataset;

[0161] The forgetting loss function construction module is used to construct a forgetting loss function based on a dynamic anchor point set of a forgetting dataset and a non-forgetting dataset;

[0162] The model parameter updating module is used to establish a comprehensive model gradient based on the forgetting loss function to update the deep neural network model parameters, so as to realize the forgetting of the long-tail recognition data to the forgotten data.

[0163] The dynamic anchor point set acquisition module includes a deep feature vector extraction unit, a temporary anchor point calculation unit, a dynamic anchor point weighting unit, and a dynamic anchor point set unit;

[0164] The deep feature vector extraction unit is used to input all categories of the forgotten dataset and the non-forgotten dataset into the feature extractor of the deep neural network model, and extract the deep feature vectors of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula:

[0165] z i =E(x i );

[0166] z c =E(x c );

[0167] In the above formula, z i is the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...z i，L}, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K};

[0168] The temporary anchor point calculation unit is used to take the mean of all deep feature vectors of each category as the initial anchor point, and calculate the temporary anchor points of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula:

[0169]

[0170]

[0171] In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the initial anchor point of category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c, is the initial anchor point of category c;

[0172] The dynamic anchor point weighting unit is used to weight the temporary anchor points of each category and the initial anchor points to obtain the dynamic anchor points of each category in the forgotten dataset and the non-forgotten dataset samples:

[0173]

[0174] In the above formula, a i is the dynamic anchor point of category i in the forgetting dataset, β is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset;

[0175] The dynamic anchor point set unit is used to traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and obtain the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,,...,a f}, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c ,...,a r}.

[0176] The forgetting loss function of the forgetting loss function building module is:

[0177]

[0178] In the above formula, is the forgetting loss function, is the similarity loss function of the forgotten dataset, λ is the weight hyperparameter that balances the contribution of the two types of losses, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset, P i For p ic The probability matrix composed of is the forgotten data x of category i i The predicted probability matrix obtained by forward propagation of the deep neural network model, r is the total number of categories of the non-forgetting dataset, p ic is the probability distribution that category i of the forgotten dataset belongs to category c of the non-forgotten dataset, For p ic The predicted probability matrix of forward propagation, a c is the dynamic anchor of category c in the non-forgetting dataset.

[0179] The model parameter updating module includes a gradient calculation unit, a comprehensive model gradient forming unit, a model parameter updating unit, and a data forgetting unit;

[0180] The gradient calculation unit is used to calculate the forgetting gradient and memory gradient of the forgotten data set relative to the deep neural network model, and the forgetting gradient is calculated based on the forgetting loss function;

[0181] The comprehensive model gradient forming unit is used to calculate the cosine similarity of the forgetting gradient and the memory gradient, and determine whether the cosine similarity is negative. If the cosine similarity of the forgetting gradient and the memory gradient is negative, the forgetting gradient is orthogonalized and adjusted, and the orthogonalized forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient; if the cosine similarity of the forgetting gradient and the memory gradient is not negative, the forgetting gradient remains unchanged, and the forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient;

[0182] The model parameter updating unit is used to update the deep neural network model parameters using the following formula using the comprehensive model gradient:

[0183] w′=w-η·g;

[0184] In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient;

[0185] The data forgetting unit is used in the deep neural network model to forget the forgotten data based on the updated model parameters.

[0186] The forgetting gradient of the gradient calculation unit is calculated using the following formula:

[0187]

[0188] In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function;

[0189] The memory gradient of the gradient calculation unit is calculated using the following formula:

[0190]

[0191] In the above formula, g r is the memory gradient, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y j is the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample;

[0192] The comprehensive model gradient of the comprehensive model gradient forming unit is expressed by the following formula:

[0193]

[0194]

[0195] In the above formula, g′ f is the forgotten gradient after orthogonal adjustment, cos(g f , g r ) is the cosine similarity between the forget gradient and the memory gradient.

Claims

1. A long-tail identification data privacy protection method based on a forgetting learning algorithm, characterized in that: The method comprises: S1. Obtain an original long-tail recognition dataset and use a deep neural network model to obtain a dynamic anchor point set of a forgotten dataset and a non-forgotten dataset, wherein the original long-tail recognition dataset includes a forgotten dataset that requires privacy protection and a normal non-forgotten dataset; S2. Construct a forgetting loss function based on a dynamic anchor point set of the forgotten dataset and the non-forgotten dataset; S3. Based on the forgetting loss function, a comprehensive model gradient is established to update the parameters of the deep neural network model to achieve the forgetting of the forgotten data by the long-tail recognition data.

2. The long-tail identification data privacy protection method based on the forgetting learning algorithm according to claim 1 is characterized in that: Said S1 comprises: S11. Input all categories of the forgotten and non-forgotten datasets into the feature extractor of the deep neural network model, and use the following formula to extract the deep feature vectors of each category in the forgotten and non-forgotten dataset samples: z i =E(x i ); z c =E(x c ); In the above formula, z i is the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...,z i，L }, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K }; S12. Take the mean of all deep feature vectors of each category as the initial anchor point, and use the following formula to calculate the temporary anchor points of each category in the forgotten dataset and the non-forgotten dataset samples: In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the initial anchor point of category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c, is the initial anchor point of category c; S13. Weighted fusion of temporary anchor points of each category with the initial anchor points to obtain dynamic anchor points of each category in the forgotten and non-forgotten dataset samples: In the above formula, a i is the dynamic anchor point of category i in the forgotten dataset, γ is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset; S14. Traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and obtain the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,...,a f }, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c ,...,a r }.

3. The long-tail identification data privacy protection method based on the forgetting learning algorithm according to claim 1 is characterized in that: The forgetting loss function of S2 is: In the above formula, is the forgetting loss function, is the similarity loss function of the forgotten dataset, λ is the weight hyperparameter that balances the contribution of the two types of losses, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset, P i For p ic The probability matrix composed of is the forgotten data x of category i i The predicted probability matrix obtained by forward propagation of the deep neural network model, r is the total number of categories of the non-forgetting dataset, p ic is the probability distribution that category i of the forgotten dataset belongs to category c of the non-forgotten dataset, For p ic The predicted probability matrix of forward propagation, a c is the dynamic anchor of category c in the non-forgetting dataset.

4. The long-tail identification data privacy protection method based on the forgetting learning algorithm according to claim 1 is characterized in that: The S3 includes: S31, calculating the forgetting gradient and memory gradient of the forgotten dataset relative to the deep neural network model, wherein the forgetting gradient is calculated based on the forgetting loss function; S32. Calculate the cosine similarity of the forgetting gradient and the memory gradient, and determine whether the cosine similarity is negative. If the cosine similarity of the forgetting gradient and the memory gradient is negative, perform an orthogonal adjustment on the forgetting gradient, and linearly superimpose the orthogonalized forgetting gradient and the memory gradient to form a comprehensive model gradient. If the cosine similarity of the forgetting gradient and the memory gradient is not negative, the forgetting gradient remains unchanged, and the forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient. S33. Using the integrated model gradient, update the deep neural network model parameters using the following formula: w′=w-η·g; In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient; S34. The deep neural network model achieves forgetting of forgotten data based on the updated model parameters.

5. The long-tail identification data privacy protection method based on the forgetting learning algorithm according to claim 4 is characterized in that: The forgetting gradient of S31 is calculated using the following formula: In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function; The memory gradient of S31 is calculated using the following formula: In the above formula, g r is the memory gradient, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y j is the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample; The comprehensive model gradient of S32 is expressed by the following formula: In the above formula, g′ f is the forgotten gradient after orthogonal adjustment, cos(g f , g r ) is the cosine similarity between the forget gradient and the memory gradient.

6. The long-tail identification data privacy protection system based on the forgetting learning algorithm is characterized by: The system includes a dynamic anchor point set acquisition module, a forgetting loss function construction module, and a model parameter updating module; The dynamic anchor point set acquisition module is used to obtain an original long-tail recognition dataset and obtain a dynamic anchor point set of a forgotten dataset and a non-forgotten dataset using a deep neural network model, wherein the original long-tail recognition dataset includes a forgotten dataset that requires privacy protection and a normal non-forgotten dataset; The forgetting loss function construction module is used to construct a forgetting loss function based on a dynamic anchor point set of a forgetting dataset and a non-forgetting dataset; The model parameter updating module is used to establish a comprehensive model gradient based on the forgetting loss function to update the deep neural network model parameters, so as to realize the forgetting of the long-tail recognition data to the forgotten data.

7. The long-tail identification data privacy protection system based on the forgetting learning algorithm according to claim 6 is characterized in that: The dynamic anchor point set acquisition module includes a deep feature vector extraction unit, a temporary anchor point calculation unit, a dynamic anchor point weighting unit, and a dynamic anchor point set unit; The deep feature vector extraction unit is used to input all categories of the forgotten dataset and the non-forgotten dataset into the feature extractor of the deep neural network model, and extract the deep feature vectors of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula: z i =E(x i ); z c =E(x c ); In the above formula, z i is the deep feature vector of category i in the forgotten dataset, E is the feature extractor of the deep neural network model, x i To forget the pixel information of category i in the dataset, z c is the deep feature vector of category c in the non-forgetting dataset, x c is the pixel information of category c in the non-forgetting dataset, where all deep feature vectors of category i are represented as {z i，1 , z i，2 ,...,z i，L }, all deep feature vectors of category c are represented as {z c，1 , z c，2 ,...z c，K }; The temporary anchor point calculation unit is used to take the mean of all deep feature vectors of each category as the initial anchor point, and calculate the temporary anchor points of each category in the forgotten dataset and the non-forgotten dataset samples using the following formula: In the above formula, is the temporary anchor point of category i in the forgotten dataset, z i，l is the lth deep feature vector in category i, is the initial anchor point of category i, is the temporary anchor point of category c in the non-forgetting dataset, z c，k is the k-th deep feature vector in category c, is the initial anchor point of category c; The dynamic anchor point weighting unit is used to weight the temporary anchor points of each category and the initial anchor points to obtain the dynamic anchor points of each category in the forgotten dataset and the non-forgotten dataset samples: In the above formula, a i is the dynamic anchor point of category i in the forgotten dataset, γ is the momentum coefficient, a c is the dynamic anchor point of category c in the non-forgetting dataset; The dynamic anchor point set unit is used to traverse the dynamic anchor points of all categories of the forgotten dataset and the non-forgotten dataset in the original long-tail dataset, and obtain the dynamic anchor point set of the forgotten dataset as D f，a ={a1, ..., a i ,...,a f }, the dynamic anchor point set of the non-forgetting dataset is D r，a ={a1, ..., a c ,...,a r }.

8. The long-tail identification data privacy protection system based on the forgetting learning algorithm according to claim 6 is characterized in that: The forgetting loss function of the forgetting loss function building module is: In the above formula, is the forgetting loss function, is the similarity loss function of the forgotten dataset, λ is the weight hyperparameter that balances the contribution of the two types of losses, Assign loss function to probability, D f is the forgotten data set, x i is the pixel information of category i, z i is the deep feature vector of category i in the forgotten dataset, a i is the dynamic anchor point of category i in the forgotten dataset, P i For p ic The probability matrix composed of is the forgotten data x of category i i The predicted probability matrix obtained by forward propagation of the deep neural network model, r is the total number of categories of the non-forgetting dataset, p ic is the probability distribution that category i of the forgotten dataset belongs to category c of the non-forgotten dataset, For p ic The predicted probability matrix of forward propagation, a c is the dynamic anchor of category c in the non-forgetting dataset.

9. The long-tail identification data privacy protection system based on the forgetting learning algorithm according to claim 6 is characterized in that: The model parameter updating module includes a gradient calculation unit, a comprehensive model gradient forming unit, a model parameter updating unit, and a data forgetting unit; The gradient calculation unit is used to calculate the forgetting gradient and memory gradient of the forgotten data set relative to the deep neural network model, and the forgetting gradient is calculated based on the forgetting loss function; The comprehensive model gradient forming unit is used to calculate the cosine similarity of the forgetting gradient and the memory gradient, and determine whether the cosine similarity is negative. If the cosine similarity of the forgetting gradient and the memory gradient is negative, the forgetting gradient is orthogonalized and adjusted, and the orthogonalized forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient; if the cosine similarity of the forgetting gradient and the memory gradient is not negative, the forgetting gradient remains unchanged, and the forgetting gradient and the memory gradient are linearly superimposed to form a comprehensive model gradient; The model parameter updating unit is used to update the deep neural network model parameters using the following formula using the comprehensive model gradient: w′=w-η·g; In the above formula, w′ is the updated deep neural network model parameter, w is the deep neural network model parameter, η is the learning rate, and g is the comprehensive model gradient; The data forgetting unit is used in the deep neural network model to forget the forgotten data based on the updated model parameters.

10. The long-tail identification data privacy protection system based on the forgetting learning algorithm according to claim 9 is characterized in that: The forgetting gradient of the gradient calculation unit is calculated using the following formula: In the above formula, g f is the forgetting gradient, w is the deep neural network model parameter, is the forgetting loss function; The memory gradient of the gradient calculation unit is calculated using the following formula: In the above formula, g r is the memory gradient, is the cross entropy loss, is a set of similar categories D j Pixel information of the dynamic anchor points corresponding to the non-forgotten categories, is the memory dataset, y j is the true category label of the dynamic anchor sample, is the predicted probability of the deep neural network model for the dynamic anchor point sample; The comprehensive model gradient of the comprehensive model gradient forming unit is expressed by the following formula: In the above formula, g′ f is the forgotten gradient after orthogonal adjustment, cos(g f , g r ) is the cosine similarity between the forget gradient and the memory gradient.

Images