Machine learning data forgetting method inspired by Hebb plasticity

Abstract

The invention discloses a machine learning data forgetting method based on Hebb plasticity inspiration. The method comprises the following steps: dividing a given data set into a reservation set and a forgetting set according to a forgetting task; establishing a first recursive lateral connection and a second recursive lateral connection between layers of the deep network model, which are respectively used for extracting feature subspaces of a given data set and a forgetting set; based on the feature subspaces of the given data set and the forgotten set, obtaining a core gradient space of the reserved set; constructing an orthogonal projection matrix of the core gradient space; forward propagation is carried out on samples in the forgotten set, and an error signal is generated through a loss function; performing back propagation on the samples in the forgotten set, and correcting the weight gradient of the deep network model based on the error signal and the orthogonal projection matrix; and updating the weight of the deep network model by using the corrected weight gradient to complete the forgetting task. By means of the method, forgetting of specific data can be efficiently and accurately achieved under the condition that the model is not retrained.

Description

Technical Field

[0001] The present invention relates to the field of machine forgetting technology, and more specifically to a Hebb plasticity-inspired machine learning data forgetting method. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, data privacy and security issues have become increasingly prominent. During the training process of large-scale data sets, machine learning models may capture and retain sensitive personal information, leading to potential privacy leakage risks. For example, membership inference attacks and model inversion attacks may leak sensitive information in training data, leading to privacy risks. Under this premise, AI's dependence on massive training data, data privacy and regulatory requirements have gradually given rise to new needs, namely, to remove specific samples and their impact from machine learning models. The most direct response strategy is to retrain the model, that is, to retrain the entire model after removing specific samples. However, this process is usually time-consuming and expensive. Therefore, developing a more efficient model update method has become an urgent problem to be solved.

[0003] Machine Unlearning (MU) technology has emerged, which aims to quickly remove specific samples and their effects without retraining the entire model. In addition to meeting legal and regulatory requirements, MU can also improve the performance and robustness of the model, especially when removing abnormal or erroneous samples. Figure 1 As shown in Figure 1, currently, machine forgetting technologies are mainly divided into three categories: based on data deletion, based on data perturbation, and based on model update. Among them, for deep learning models trained by stochastic gradient descent, the model update-based method for approximate machine forgetting is the most promising. This type of method achieves forgetting of specific samples by minimizing adjustments to the model structure and parameters. Common techniques include regularization, transfer learning, model distillation, model pruning, and model inversion. However, the existing technology still has the following shortcomings:

[0004] 1. High cost of retraining: Currently, the most intuitive way to remove the impact of a single data point from a trained model is to remove the samples to be deleted (i.e., the forgotten set) from the original training data, and then retrain the model based on the remaining data (i.e., the retained set). However, this strategy faces serious feasibility issues in practical applications. First, retraining requires a lot of computing resources and time, especially for large models and large-scale data sets, this process becomes extremely expensive. For example, training a complex deep learning model may require days or even weeks of computing resources, resulting in extremely high costs. In addition, due to privacy regulations, the original training data may not be kept indefinitely, which makes retraining more difficult or even impossible. Therefore, the retraining scheme is difficult to meet the needs of efficient and low-cost forgetting in actual scenarios.

[0005] 2. The computational complexity of the method based on influence function is huge: Many existing methods that approximate machine forgetting rely on the influence function, which estimates the impact of training data on the model and finds an update process to reverse the impact of forgotten samples. However, the calculation of the influence function involves the estimation of the Hessian matrix, which is a process with high computational complexity, especially when the model is large, the amount of calculation increases exponentially. Therefore, in application scenarios such as large models such as convolutional neural networks (CNN) and Transformer, methods based on influence functions often find it difficult to significantly reduce computational costs, and still take a lot of time to perform weight adjustments, which cannot meet the needs of fast and efficient forgetting.

[0006] 3. Performance limits based on orthogonal projection: Another machine forgetting technique is based on orthogonal projection, which regularizes the gradient projection of the model on the retention set and limits the weight update to orthogonal directions that take into account high-dimensional linear transformations. This method avoids the direct calculation of the influence function, but introduces the singular value decomposition (SVD) step, which results in it being able to effectively estimate the gradient subspace only on small batches of data. When processing large-scale or streaming data, the efficiency and accuracy of this method will be limited, making it difficult to meet the needs of online data processing, and the additional calculation steps introduced will still have an impact on the overall performance.

[0007] In summary, with increasingly stringent data privacy regulations and users paying more attention to their personal data rights, traditional models are difficult to promptly and effectively remove specific data samples and their impact from the model after training, and cannot meet the current stringent data privacy regulations and users' requirements for personal data rights. In particular, although retraining the model can achieve the purpose of removing specific data, the high computational cost and time overhead make it infeasible in practical applications.

[0008] Therefore, how to efficiently and accurately achieve the forgetting of specific data without retraining the model is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0009] In view of the above problems, the present invention provides a Hebb plasticity-inspired machine learning data forgetting method to at least solve some of the technical problems mentioned in the above background technology.

[0010] In order to achieve the above object, the present invention adopts the following technical solution:

[0011] A Hebb plasticity-inspired method for forgetting machine learning data, comprising the following steps:

[0012] S1. According to the forgetting task, the given data set is divided into a retention set and a forgetting set;

[0013] S2. Establishing a first recursive lateral connection and a second recursive lateral connection between layers of the deep network model, respectively for extracting feature subspaces of the given data set and the forgotten set;

[0014] S3, based on the feature subspace of the given data set and the feature subspace of the forgotten set, obtaining the core gradient space of the retained set;

[0015] S4, constructing an orthogonal projection matrix of the core gradient space;

[0016] S5, forward propagating the samples in the forgetting set, and generating an error signal through a loss function;

[0017] S6, performing back propagation on the samples in the forgetting set, and correcting the weight gradient of the deep network model based on the error signal and the orthogonal projection matrix;

[0018] S7. Use the corrected weight gradient to update the weight of the deep network model to complete the forgetting task.

[0019] Furthermore, the step S2 specifically includes:

[0020] Traversing and forward propagating samples in the given data set, and updating the weights of the first recursive lateral connections using the Hebb rule to extract a feature subspace of the given data set;

[0021] The samples in the forgetting set are traversed and forward propagated, and the weights of the second recursive lateral connections are updated using the Hebb rule to extract the feature subspace of the forgetting set.

[0022] Furthermore, the first recursive lateral connection and the second recursive lateral connection have antisymmetric weights.

[0023] Furthermore, the step S3 specifically includes:

[0024] S31, based on the feature subspace of the given data set and the feature subspace of the forgotten set, obtaining the feature subspace of the retained set; expressed as:

[0025]

[0026] in, Represents the input feature matrix of the lth linear layer in the retained set; Represents the feature subspace of the lth linear layer in the retained set; Represents the input feature matrix of the lth linear layer in a given dataset; Represents the feature subspace of the lth linear layer in a given data set; Represents the input feature matrix of the lth linear layer in the forgetting set; represents the feature subspace of the lth linear layer in the forgetting set; T represents transposition;

[0027] S32, performing SVD decomposition on the feature subspace of the reserved set, expressed as:

[0028]

[0029] in, Represents the eigenvector matrix of the lth linear layer in the retained set; express The corresponding eigenvalue matrix; Represents the feature vector matrix of the lth linear layer in a given dataset; express The corresponding eigenvalue matrix; Represents the eigenvector matrix of the lth linear layer in the forgetting set; express The corresponding eigenvalue matrix;

[0030] S33, take the eigenvector matrix The first k eigenvectors in , span the core gradient space of the retained set; expressed as:

[0031]

[0032] Among them, CGS represents the core gradient space of the retention set; span represents span; Represents the eigenvector matrix The kth eigenvector in .

[0033] Furthermore, the orthogonal projection matrix of the core gradient space is expressed as:

[0034]

[0035] in, represents the orthogonal projection matrix; I represents a total of I linear layers; The matrix form representing the core gradient space is the principal component matrix of the retained set; Represents the principal component matrix of a given data set; Represents the principal component matrix of the forgotten set.

[0036] Furthermore, the step S5 specifically includes:

[0037] S51, performing forward propagation on the samples in the forgetting set to obtain the predicted labels output by the deep network model for the samples in the forgetting set;

[0038] S52: Based on the predicted label and in combination with the corresponding true label, an error signal is obtained through a loss function.

[0039] Furthermore, for the classification task, the loss function is expressed as:

[0040]

[0041] in, represents the loss function; represents the forgotten set; i represents the i-th sample in the forgotten set; C represents a total of C categories; Indicates whether the i-th sample in the forgetting set belongs to the true label of the c-th category; Forget whether the i-th sample in the set belongs to the predicted label of the c-th category; Represents a constant; represents the maximization coefficient, .

[0042] Furthermore, the step S6 specifically includes:

[0043] S61, performing back propagation on the samples in the forgetting set, and in the back propagation process, correcting the input features of the intermediate layer using an orthogonal projection matrix;

[0044] S62, based on the corrected intermediate layer input features, combined with the error signal, the weight gradient of the deep network model is corrected. Taking a single-layer network as an example, the above process is expressed as:

[0045]

[0046] in, Represents the corrected single-layer network weight gradient; represents the error signal; x represents the input feature of this layer; represents the weight of the first recursive lateral connection after updating using the Hebb rule; represents the weight of the second recursive lateral connection after updating using the Hebb rule.

[0047] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a Hebb plasticity-inspired machine learning data forgetting method, which has the following beneficial effects:

[0048] The present invention can significantly improve the efficiency of data forgetting; specifically, the present invention avoids the high cost of retraining the model by introducing the orthogonal projection method inspired by Hebb plasticity. When processing large-scale, streaming data, this method can efficiently and accurately remove the influence of specified samples, making it possible to perform data forgetting in practical applications. This is particularly important for scenarios that require frequent model updates or real-time processing of user data deletion requests.

[0049] The present invention can ensure the performance and robustness of the model; specifically, the present invention retains the performance of the deep network model on the reserved set to the greatest extent while forgetting specific data samples. By precisely adjusting the model parameters, only the influence of the target sample is removed, avoiding the negative impact on the overall performance of the model. This ensures the robustness and reliability of the machine learning model, meets the high requirements of practical applications for model performance, and is expected to be widely used in sensitive data processing, personalized services, medical health and other fields.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 Schematic diagram of the types of machine forgetting techniques provided for related prior art.

[0053] Figure 2 A schematic flow chart of a Hebb plasticity-inspired machine learning data forgetting method provided in an embodiment of the present invention.

[0054] Figure 3 A schematic diagram of a process for constructing a feature space of a retained set based on a machine forgetting task provided in an embodiment of the present invention.

[0055] Figure 4A schematic diagram of the Hebb forward update process of the lateral connection antisymmetric weights provided in an embodiment of the present invention.

[0056] Figure 5 A schematic diagram of the data forgetting process of machine learning based on Hebb plasticity inspiration provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] See also Figure 2 As shown, the embodiment of the present invention discloses a Hebb plasticity-inspired machine learning data forgetting method, comprising the following steps:

[0059] S1. According to the forgetting task, the given data set is divided into a retention set and a forgetting set;

[0060] S2, establishing a first recursive lateral connection and a second recursive lateral connection between the layers of the deep network model, respectively for extracting feature subspaces of a given data set and a forgotten set;

[0061] S3, based on the feature subspace of the given data set and the feature subspace of the forgotten set, obtain the core gradient space of the retained set;

[0062] S4, constructing an orthogonal projection matrix of the core gradient space;

[0063] S5, forward propagation is performed on the samples in the forgetting set, and an error signal is generated through the loss function;

[0064] S6, back-propagating the samples in the forgetting set, and correcting the weight gradient of the deep network model based on the error signal and the orthogonal projection matrix;

[0065] S7. Use the corrected weight gradient to update the weights of the deep network model to complete the forgetting task.

[0066] Next, each of the above steps will be described in detail.

[0067] In the above step S1, the deep network model (such as ResNet-19) is initialized; the deep network model is fully back-propagated on a given dataset (such as ImageNet); then the forgetting task is established, see Figure 3 As shown, according to the forgetting task, the given data set is divided into a retention set and a forgetting set;

[0068] Specifically, the machine's forgetting task is mathematically described as follows: Definition For a given data set, is a parameterized deep network model, such as CNN, where the parameter W is trained on a given dataset D. Define the forgetting set As a subset of the training set, the information it contains is to be Removed from the definition of the reserved set for The complement of D, that is and , the knowledge it contains will be preserved.

[0069] In the above step S2, see Figure 4 As shown, a first recursive lateral connection and a second recursive lateral connection are established between the layers of the deep network model, which are used to extract the feature subspace of the given data set and the forgotten set respectively;

[0070] Specifically, the samples in the given data set D are traversed and forward propagated, and the weights of the first recursive lateral connection are updated using the Hebb rule to extract the feature subspace of the given data set; The samples in are traversed and forward propagated, and the weights of the second recursive lateral connection are updated using the Hebb rule to extract the feature subspace of the forgotten set;

[0071] (1) Hebb rule

[0072] The above Hebb learning rule is the Hebbian theory, which describes the basic principle of synaptic plasticity, that is, the continuous and repeated stimulation of the presynaptic neuron to the postsynaptic neuron can lead to an increase in the efficiency of synaptic transmission. This theory was proposed by Donald Hebb in 1949 and is also known as Hebb's rule, Hebb's postulate, etc. Oja's Rule is a variant of Hebb's learning rule. For a single output neuron with weight h, input x and output y, the weight update process following the Oja rule is as follows:

[0073]

[0074] It is proven that it can converge to the principal components of the input, thus achieving the most advanced convergence speed in the streaming principal component analysis problem. The matrix form of the update process can be written as:

[0075]

[0076] Therefore, the lateral connection weight H can be updated according to the above rules, and the cyclic connection weight The antisymmetry with respect to H is maintained during the update process.

[0077] The Hebb update process is unsupervised and does not rely on back propagation. Therefore, for a given dataset D and a forgetting set In the process of approximate learning of the principal components of the features, it is only necessary to perform forward propagation on the small batch data stream and update the lateral weights according to the rules to complete the principal component estimation of each layer of features.

[0078] (2) Extracting feature subspace

[0079] When the deep network model completes the training process on a given data set, it is necessary to calculate the feature space of the given data set (only once), and then calculate the feature space of the forgotten set according to different forgotten sets to construct the projection matrix of the Core Gradient Space (CGS).

[0080] Extracting the feature subspace of the l-th layer weight of the deep network model on a given dataset D requires sampling from the given dataset D representative training samples; for each convolutional layer or linear layer l, the training samples Perform a forward pass to obtain the output features of the l-1th layer , and serves as the input feature of the lth layer Based on this, we get features; all d-dimensional input feature vectors are concatenated along the columns to form an input feature matrix , the characteristic subspace it spans is ;in, yes A subset of Input representation. Can be calculated in batch mode;

[0081] Forgotten Collection The process of extracting the feature subspace of the l-th layer weights can refer to the process of extracting the feature subspace of the given data set D mentioned above.

[0082] In the above step S3, when actually determining the machine forgetting task, for a model pre-trained on a training set, there are often different forgetting subset task requirements. In order to adapt to different tasks, the embodiment of the present invention intends to establish a given data set D and a forgetting set The characteristic subspace of , indirectly and quickly calculates the reserved set feature subspace; that is, whenever we receive the forget set When a data deletion request is made, the retention set can be calculated The characteristic subspace of ; expressed as:

[0083]

[0084] in, Represents the input feature matrix of the lth linear layer in the retained set; Represents the feature subspace of the lth linear layer in the retained set; Represents the input feature matrix of the lth linear layer in a given dataset; Represents the feature subspace of the lth linear layer in a given data set; Represents the input feature matrix of the lth linear layer in the forgetting set; represents the feature subspace of the lth linear layer in the forgetting set; T represents transposition;

[0085] If we follow the general steps of the gradient projection method, in order to simplify the size of the feature space representation, we need to perform SVD decomposition on the feature subspace of the retained set, which can be expressed as:

[0086]

[0087] in, Represents the eigenvector matrix of the lth linear layer in the retained set; express The corresponding eigenvalue matrix; Represents the feature vector matrix of the lth linear layer in a given dataset, ; express The corresponding eigenvalue matrix is, ; Represents the eigenvector matrix of the lth linear layer in the forgetting set; express The corresponding eigenvalue matrix;

[0088] Therefore, the core gradient space on the retained set is composed of the eigenvector matrix The first k eigenvectors of are expressed as:

[0089]

[0090] Among them, CGS represents the core gradient space of the retention set; span represents span; Represents the eigenvector matrix The kth eigenvector in .

[0091] In the above step S4, an orthogonal projection matrix of the core gradient space is constructed, specifically:

[0092] The core gradient space CGS of the retained set can be expressed in matrix form , then the corresponding orthogonal projection matrix of the gradient projection method is ;

[0093] To avoid redundant computations caused by different forgetting requirements, use and The first k eigenvectors of represent the orthogonal projection matrix of the core gradient space as:

[0094]

[0095] in, represents the orthogonal projection matrix; I represents a total of I linear layers; The matrix form representing the core gradient space is the principal component matrix of the retained set; Represents the principal component matrix of a given data set; Represents the principal component matrix of the forgotten set.

[0096] In the above step S5, forward propagation is performed on the samples in the forgetting set to generate an error signal through a loss function; specifically, the step includes:

[0097] S51, forward propagating the samples in the forgotten set to obtain the predicted labels output by the deep network model for the samples in the forgotten set;

[0098] S52. Based on the predicted label and the corresponding true label, an error signal is obtained through a loss function.

[0099] In the framework of machine forgetting tasks, it is necessary to define the loss function of the machine forgetting or "reversing machine learning" process. Taking the classification task as an example, the cross entropy loss of the machine learning process is rewritten as follows in the forgetting process:

[0100]

[0101] in, represents the loss function; represents the forgotten set; i represents the i-th sample in the forgotten set; C represents a total of C categories; Indicates whether the i-th sample in the forgetting set belongs to the true label of the c-th category; Forget whether the i-th sample in the set belongs to the predicted label of the c-th category; Represents a constant; represents the maximization coefficient, ;

[0102] In the above loss function formula, the predicted label It is expressed as:

[0103]

[0104] in, express the cth element of ; Representation sample When inputting to the deep network model The output feature of the corresponding j-th category.

[0105] In the above loss function formula, the first term It is an “inverted” cross entropy loss that attempts to minimize the predicted probability of the true label class. is a small constant to avoid large gradients at the beginning of training; the second Try to maximize the information entropy of the output value of the deep network model so that all categories have the same confidence score. This term helps to remove cross-category label information in the forgotten data (for example, birds and airplanes may have similar background information) and reduce the focus on features related to the forgotten labels.

[0106] In the above step S6, see Figure 5 As shown, the samples in the forgetting set are back-propagated, and the weight gradient of the deep network model is corrected based on the error signal and the orthogonal projection matrix, including:

[0107] S61, back-propagating the samples in the forgetting set, and in the back-propagation process, using the orthogonal projection matrix to correct the input features of the intermediate layer;

[0108] The input feature of the middle layer of the network here refers to the state or activity of a neuron in the neural network when it receives input signals from other neurons (presynaptic neurons); the linear correction of the input feature x can be implemented using recursive lateral connections with antisymmetric weights. Specifically, first x is propagated laterally to the subspace neurons with connections H in the form of The form of y is connected by The loop is propagated back to the original feature space. The corrected input features It is expressed as:

[0109]

[0110] Therefore, as long as the lateral connection H approximates the principal component matrix M, the lateral connection structure of the neural circuit can achieve gradient orthogonal projection, and Hebb learning can quickly iterate to make the lateral weight H approximate the principal component matrix M, thereby achieving an efficient and biologically reasonable forgetting process. In addition, in order to represent the core gradient space, two sets of lateral connections are required to extract D and , and are selectively activated in their respective approximate learning processes;

[0111] In short, the core gradient space represents the gradient direction of the information that needs to be retained, while the orthogonal space of the core gradient is used to remove unnecessary information, that is, to selectively forget the influence of certain data samples. Since the above gradient update direction after being corrected by the orthogonal projection matrix can be orthogonal to the core gradient space, it effectively ensures that the specific information in the forgotten set is removed without affecting the learning results of the retained set data.

[0112] S62, based on the corrected intermediate layer input features, combined with the error signal, the weight gradient of the deep network model is corrected. Taking a single-layer network as an example, the above process is expressed as:

[0113]

[0114] in, Represents the corrected single-layer network weight gradient; represents the error signal; x represents the input feature of this layer; represents the weight of the first recursive lateral connection after updating using the Hebb rule; Represents the weight of the second recursive lateral connection after updating using the Hebb rule; it can be seen that the weight gradient of the deep network model can be quickly corrected by only implementing layer-by-layer linear correction of the feature map during the back propagation process.

[0115] In the above step S7, the weights of the deep network model are updated using the corrected weight gradients, and the direction of the updated weight changes will be restricted within the core gradient space, thereby selectively forgetting the influence of the data samples in the forgetting set while keeping the performance of the model unchanged on the retained set, thus completing the forgetting task.

[0116] In summary, the embodiments of the present invention provide a method for machine learning data forgetting inspired by Hebb plasticity, which utilizes lateral connections between neurons, combined with Hebb and anti-Hebb learning mechanisms, to extract the core gradient space of the model on the retained set streaming data. By projecting the gradient direction of the synaptic weight update to the orthogonal space of the core gradient, selective forgetting of specific data samples is achieved without retraining the entire model. Compared with traditional methods, the present invention effectively reduces the computational cost of data forgetting, and at the same time can process large batches of streaming data in a more efficient and unbiased manner, and construct the core gradient space online, thereby achieving data forgetting while ensuring the maintenance of model performance and the efficiency of the forgetting process, thereby ensuring the stability and reliability of model performance.

[0117] Due to the high efficiency and applicability of the method provided by the present invention, machine learning models can be more widely used in fields that need to process sensitive data (such as medical health, personalized services, etc.). The present invention improves the generalization ability and adaptability of the model by improving the robustness of the model to abnormal or erroneous data, and further improves the competitiveness of the model in the market.

[0118] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0119] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Hebb plasticity-inspired method for machine learning data forgetting, characterized in that: The steps include: S1. According to the forgetting task, the given data set is divided into a retention set and a forgetting set; S2. Establishing a first recursive lateral connection and a second recursive lateral connection between layers of the deep network model, respectively for extracting feature subspaces of the given data set and the forgotten set; S3, based on the feature subspace of the given data set and the feature subspace of the forgotten set, obtaining the core gradient space of the retained set; S4, constructing an orthogonal projection matrix of the core gradient space; S5, forward propagating the samples in the forgetting set, and generating an error signal through a loss function; S6, performing back propagation on the samples in the forgetting set, and correcting the weight gradient of the deep network model based on the error signal and the orthogonal projection matrix; S7. Use the corrected weight gradient to update the weight of the deep network model to complete the forgetting task.

2. A Hebb plasticity-inspired machine learning data forgetting method according to claim 1, characterized in that: The step S2 specifically includes: Traversing and forward propagating samples in the given data set, and updating the weights of the first recursive lateral connections using the Hebb rule to extract a feature subspace of the given data set; The samples in the forgetting set are traversed and forward propagated, and the weights of the second recursive lateral connections are updated using the Hebb rule to extract the feature subspace of the forgetting set.

3. The Hebb plasticity-inspired machine learning data forgetting method according to claim 1, characterized in that: The first recursive lateral connection and the second recursive lateral connection have antisymmetric weights.

4. The method of Hebb plasticity-inspired machine learning data forgetting according to claim 1, characterized in that: The step S3 specifically includes: S31, based on the feature subspace of the given data set and the feature subspace of the forgotten set, obtaining the feature subspace of the retained set; expressed as: ； in, Represents the input feature matrix of the lth linear layer in the retained set; Represents the feature subspace of the lth linear layer in the retained set; Represents the input feature matrix of the lth linear layer in a given dataset; Represents the feature subspace of the lth linear layer in a given data set; Represents the input feature matrix of the lth linear layer in the forgetting set; represents the feature subspace of the lth linear layer in the forgetting set; T represents transposition; S32, performing SVD decomposition on the feature subspace of the reserved set, expressed as: ； in, Represents the eigenvector matrix of the lth linear layer in the retained set; express The corresponding eigenvalue matrix; Represents the feature vector matrix of the lth linear layer in a given dataset; express The corresponding eigenvalue matrix; Represents the eigenvector matrix of the lth linear layer in the forgetting set; express The corresponding eigenvalue matrix; S33, take the eigenvector matrix The first k eigenvectors in , span the core gradient space of the retained set; expressed as: ； Among them, CGS represents the core gradient space of the retention set; span represents span; Represents the eigenvector matrix The kth eigenvector in .

5. The Hebb plasticity-inspired machine learning data forgetting method according to claim 1, characterized in that: The orthogonal projection matrix of the core gradient space is expressed as: ； in, represents the orthogonal projection matrix; I represents a total of I linear layers; The matrix form representing the core gradient space is the principal component matrix of the retained set; Represents the principal component matrix of a given data set; Represents the principal component matrix of the forgotten set.

6. The method of Hebb plasticity-inspired machine learning data forgetting according to claim 1, characterized in that: The step S5 specifically includes: S51, performing forward propagation on the samples in the forgetting set to obtain the predicted labels output by the deep network model for the samples in the forgetting set; S52: Based on the predicted label and in combination with the corresponding true label, an error signal is obtained through a loss function.

7. A Hebb plasticity-inspired machine learning data forgetting method according to claim 6, characterized in that: For the classification task, the loss function is expressed as: ； in, represents the loss function; represents the forgotten set; i represents the i-th sample in the forgotten set; C represents a total of C categories; Indicates whether the i-th sample in the forgetting set belongs to the true label of the c-th category; Forget whether the i-th sample in the set belongs to the predicted label of the c-th category; Represents a constant; represents the maximization coefficient, .

8. A Hebb plasticity-inspired machine learning data forgetting method according to claim 2, characterized in that: The step S6 specifically includes: S61, performing back propagation on the samples in the forgetting set, and in the back propagation process, using an orthogonal projection matrix to correct the input features of the intermediate layer; S62, based on the corrected intermediate layer input features, combined with the error signal, the weight gradient of the deep network model is corrected; taking a single-layer network as an example, the above process is expressed as: ； in, Represents the corrected single-layer network weight gradient; represents the error signal; x represents the input feature of this layer; represents the weight of the first recursive lateral connection after updating using the Hebb rule; represents the weight of the second recursive lateral connection after updating using the Hebb rule.

Images