A face recognition model forgetting method and system based on ensemble learning

Abstract

The present invention discloses a face recognition model forgetting method and system based on ensemble learning, which relates to the field of model forgetting technology. First, a face data set is obtained and divided into sub-data sets according to category; each sub-data set is divided into data blocks, and the blocks are sequentially input into corresponding single-category classifiers for training, while recording the model parameters of each data block after training; the trained single classifiers are subjected to decision aggregation to obtain a face recognition model; the corresponding single-category classifier is retrained based on the data block containing the user's target data and the model parameters corresponding to the previous data block as the starting point of the retraining process; the retrained single-category classifier is subjected to decision aggregation with other single-category classifiers to obtain a retrained face recognition model. The present invention can significantly reduce the amount of model forgetting calculations, has good processing efficiency and model forgetting effect when facing a large number of data removal applications, and keeps normal model performance unaffected.

Description

Technical Field

[0001] The present invention relates to the field of model forgetting technology, and more particularly to a face recognition model forgetting method and system based on ensemble learning. Background Art

[0002] In recent years, the issue of unauthorized use of private information has attracted widespread attention. When a machine learning model owner receives a data removal request from a data provider, they must fulfill the obligation to delete the target data. This is the so-called "right to be forgotten" granted to the data provider.

[0003] However, if the target data has already participated in model training, the knowledge about the target data has already been learned by the model and stored in the model's memory as parameters. Therefore, simply removing it from the training dataset will not completely "forget" it. For example, model inversion attacks can restore training set samples using only labels and model output behavior, and membership inference attacks can determine whether specific data was used to train the target model. Therefore, model forgetting learning techniques have been proposed to help model owners eliminate the influence of trained target data on the model.

[0004] For face recognition models, when users want to remove part of the training data, the existing model forgetting method usually requires retraining the entire model, facing problems such as large data calculation volume and unsatisfactory forgetting effect. Therefore, how to overcome this technical defect is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a face recognition model forgetting method and system based on ensemble learning. Through ensemble learning and data preprocessing, the model forgetting operation overhead is greatly saved. When faced with a large number of face data removal applications, it has good processing efficiency and model forgetting effect, while ensuring that the normal performance of the model is not affected.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A forgetting method for a face recognition model based on ensemble learning, comprising the following steps:

[0008] Step 1: Obtain a face dataset and classify it to obtain several sub-datasets of different categories;

[0009] Step 2: Input each sub-dataset into the corresponding single-category classifier for initial training. The training process is as follows:

[0010] Step 2.1: Divide each sub-dataset into data blocks;

[0011] Step 2.2: Input the data blocks of each sub-dataset into the corresponding single-class classifier for training, and record the model parameters after training the single-class classifier for each data block;

[0012] Step 3: Aggregate the trained single classifiers to obtain a face recognition model.

[0013] Step 4: Based on the user's data forgetting request, the corresponding single-category classifier is retrained using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process;

[0014] Step 5: Re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain a retrained face recognition model.

[0015] Optionally, in step 1, after obtaining several sub-datasets of different categories, the sub-datasets are compressed based on information theory to generate new sub-datasets.

[0016] Optionally, in step 2.1, each type of sub-data set is evenly divided into data blocks according to a preset size.

[0017] Optionally, in step 2.2, the data blocks of each type of sub-dataset are sorted according to the probability of being deleted, and are sequentially input into the corresponding single-category classifier from low to high for training.

[0018] Optionally, the face recognition model consists of a single-category classifier and a decision aggregation model, where:

[0019] The single-category classifier generates a distribution feature representation of the corresponding category sub-dataset, and then calculates the outlier value of the target sample, compares the outlier value with the preset threshold, and determines whether the target sample belongs to this category;

[0020] Decision aggregation aggregates the judgment results of several single-category classifiers to determine the decision aggregation result of the target sample as the output content of the face recognition model.

[0021] Optional, single-class classifier f i The goal of (x; θ) is to extract the distribution feature representation of the corresponding category sub-dataset, and the objective function F of the training process is expressed as:

[0022]

[0023] Where r i is the characteristic representation value of the i-th sub-dataset, x i is the sample of the i-th sub-dataset, is the weight decay regularizer, λ is a hyperparameter greater than 0, W[l] is the weight of the lth layer.

[0024] Optionally, the outlier value is calculated as:

[0025] anomaly_score=||f i (x i ;θ)-r i || 2 ;

[0026] Where r i is the characteristic representation value of the i-th sub-dataset, x i is a sample of the i-th sub-dataset, and the single-category classifier f i (x i ; θ) is the feature representation value of the target sample.

[0027] Optionally, during the decision aggregation, if only one single-category classifier determines that the target sample belongs to this category, then this category is directly used as the decision aggregation result; if more than one single-category classifier determines that the target sample belongs to this category, then the category with the lowest outlier value is used as the decision aggregation result.

[0028] A face recognition model forgetting system based on ensemble learning, comprising:

[0029] The dataset acquisition module is used to obtain the face dataset and classify it to obtain several sub-datasets of different categories;

[0030] The classifier initialization training module is used to input each sub-dataset into the corresponding single-category classifier for initialization training. The training process is as follows:

[0031] Each type of sub-dataset is divided into data blocks;

[0032] The data blocks of each sub-dataset are sequentially input into the corresponding single-category classifier for training, and the model parameters after each data block is trained on the single-category classifier are recorded;

[0033] The first decision aggregation module is used to aggregate the decisions of several trained single classifiers to obtain a face recognition model;

[0034] The classifier retraining module is used to retrain the corresponding single-category classifier based on the user's data forgetting request, using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process;

[0035] The second decision aggregation module is used to re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain a retrained face recognition model.

[0036] Through the above technical solutions, it can be seen that the present invention provides a face recognition model forgetting method and system based on ensemble learning, which has the following beneficial effects compared with the existing technology:

[0037] The present invention first employs ensemble learning during the model retraining phase, converting the normal retraining process into numerous single-category classification tasks to reduce retraining complexity and computational effort. Simultaneously, it introduces a variety of data preprocessing methods, including representative data compression, training state storage, and training data block sorting, to further reduce retraining complexity and accelerate the efficiency of the model forgetting process. This invention significantly reduces the computational effort required to mitigate model forgetting, achieving excellent processing efficiency and model forgetting effectiveness when faced with large-scale data removal applications, while maintaining normal model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 It is a schematic flow chart of the method of the present invention;

[0040] Figure 2 It is a schematic diagram of the overall process of model training of the present invention;

[0041] Figure 3 Schematic diagram comparing the retraining method of the present invention and the existing retraining method. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] The embodiment of the present invention discloses a face recognition model forgetting method based on ensemble learning, wherein the initial training process of the model is called the initialization training process, and the training process when the model handles the forgetting application is called the retraining process. Figure 1 and Figure 2 , specifically including the following steps:

[0044] Step 1: Get the face dataset and classify it to get several sub-datasets of different categories. For example, for dataset S, there are c label categories, and the sub-datasets S after division are i Only all samples of the corresponding category are included, so there are:

[0045]

[0046] In the specific implementation process, it is also possible to compress each sub-dataset based on information theory without losing prediction accuracy, and select samples containing as much effective information as possible to form a new sub-dataset, thereby reducing the number of retraining samples in subsequent forgetting learning and improving the efficiency of retraining. The expression is:

[0047]

[0048] Where S′ i Represents the generated new sub-dataset of type i, X1,...,X n Represents S′ i Samples of x1,...,x n They are samples X1,...,X n For different values ​​in P(x1,...,x n ) is the probability of these values ​​appearing simultaneously, in particular, when P(x1,...,x n )=0, let P(x1,...,x n )logP(x1,...,x n )=0.

[0049] Step 2: Input each sub-dataset into the corresponding single-category classifier for initial training. The training process is as follows:

[0050] Step 2.1: Divide each sub-dataset into data blocks evenly according to the preset size. Assume that the sub-dataset S i Split into m data blocks;

[0051] Step 2.2, sort the data blocks of each type of sub-dataset according to the probability of being deleted, that is, put the data with a higher probability of being applied for deletion in the last data block during initial training. The so-called probability of being deleted can be estimated by auxiliary information (for example, the demands of the data provider, the privacy level of the data source, etc.), and the present invention does not impose any restrictions on this. This method of sorting data blocks according to the probability of being deleted makes it possible to forget k samples with a high probability of being deleted, and only needs to be trained after sorting, compared with the simple retraining method that may require retraining k data blocks. data blocks. Generally, the retraining speed can be increased to k times by increasing the sorting time consumption which can be ignored. After sorting, the data blocks are input into the corresponding single-category classifier in order of probability from low to high for training. Each data block B ij After input and training, record the corresponding model parameters θ of the iterative update ij , where the final parameter θ i(m-1) is the parameter θ of the sub-model i , and record the model parameters after training the single-category classifier for each data block. This allows for subsequent forgetting and retraining to start directly from the data block where the deleted sample is located and the corresponding parameter state, without having to repeat the previous parameter update process.

[0052] Further explanation, assuming that the target data to be forgotten is in the sub-dataset S i The jth data block B ij In B, the model owner can ij Delete the target data and get a new data block B′ ij , and take out the stored state θ i(j-1) As the starting point of the model parameters, in B′ ij Retraining on the data after and thus avoiding B′ ij Unnecessary retraining of the previous j-1 data blocks.

[0053] For the above data sorting, assuming that the samples to be forgotten are from the 0th data block to the m-1th data block, it may be necessary to train from the mth data block to the 1st data block. According to the statistical average of m, there are a total of (m+1)m / 2 data blocks that need to be retrained. For direct retraining, this process always requires m 2 data blocks, so the training state is stored. While occupying only a portion of the storage space, the training speed can be increased to 2m / (m+1) times based on direct training.

[0054] Step 3: Aggregate the decisions of several trained single classifiers to obtain a face recognition model.

[0055] Step 4: Based on the user's data forgetting request, the corresponding single-category classifier is retrained using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process. Figure 3 , which is a schematic diagram comparing the retraining method of the present invention and the existing conventional retraining method.

[0056] Step 5: Re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain the retrained face recognition model, thus completing the forgetting process of the face recognition model.

[0057] The face recognition model of the present invention includes two parts: a single-category classifier and a decision aggregation. For the target sample to be classified, that is, the face image to be recognized, the single-category classifier in the retrained face recognition model first obtains several category judgment results. Then, the several category judgment results are aggregated by decision to obtain the recognition result of the face image to be recognized. Specifically:

[0058] (1) Single-category classifier:

[0059] Single-class classifier f i The goal of (x; θ) is to extract the distribution feature representation of the corresponding category sub-dataset. The objective function F in the training process is expressed as:

[0060]

[0061] Where r i is the characteristic representation value of the i-th sub-dataset (determined empirically, such as the average output of the target category sample in the middle layer of the network), x i is the sample of the i-th sub-dataset, is the weight decay regularizer, λ is a hyperparameter greater than 0, W [l] is the weight of the lth layer; the single-category classifier after training can extract the representation value of the target sample on the common features of the category, that is, f i (x i ; θ). Minimizing this objective function enables the model to learn parameters θ that can effectively extract category features;

[0062] Based on the difference between the feature representation value and the class representation value, the outlier value of the target sample is calculated using the following formula:

[0063] anomaly_score=||f i (x i ;θ)-r i || 2 ;

[0064] Where r i is the characteristic representation value of the i-th sub-dataset, x i is a sample of the i-th sub-dataset, and the single-category classifier f i (x i ; θ) is the characteristic representation value of the target sample;

[0065] Compare the outlier value with the preset threshold. If it is lower than the threshold, it is considered to belong to this category; if it is higher than the threshold, it is considered not to belong to this category.

[0066] (2) Decision aggregation: Aggregate the judgment results of several single-category classifiers. If only one single-category classifier determines that the target sample belongs to this category, then directly use this category as the decision aggregation result; if more than one single-category classifier determines that the target sample belongs to this category, then use the category with the lowest outlier value as the decision aggregation result. The final decision aggregation result is output as the output content of the face recognition model, which is the final classification result of the target sample (the face image to be recognized).

[0067] Another embodiment further discloses a face recognition model forgetting system based on ensemble learning, comprising:

[0068] The dataset acquisition module is used to obtain the face dataset and classify it to obtain several sub-datasets of different categories;

[0069] The classifier initialization training module is used to input each sub-dataset into the corresponding single-category classifier for initialization training. The training process is as follows:

[0070] Each type of sub-dataset is divided into data blocks;

[0071] The data blocks of each sub-dataset are sequentially input into the corresponding single-category classifier for training, and the model parameters after each data block is trained on the single-category classifier are recorded;

[0072] The first decision aggregation module is used to aggregate the decisions of several trained single classifiers to obtain a face recognition model;

[0073] The classifier retraining module is used to retrain the corresponding single-category classifier based on the user's data forgetting request, using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process;

[0074] The second decision aggregation module is used to re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain a retrained face recognition model.

[0075] As for the system modules disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0076] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0077] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A forgetting method for face recognition model based on ensemble learning, characterized in that: The following steps are involved: Step 1: Obtain a face dataset and classify it to obtain several sub-datasets of different categories; Step 2: Input each sub-dataset into the corresponding single-category classifier for initial training. The training process is as follows: Step 2.1: Divide each sub-dataset into data blocks; Step 2.2: Input the data blocks of each sub-dataset into the corresponding single-class classifier for training, and record the model parameters after training the single-class classifier for each data block; Step 3: Aggregate the trained single classifiers to obtain a face recognition model. Step 4: Based on the user's data forgetting request, the corresponding single-category classifier is retrained using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process; Step 5: Re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain a retrained face recognition model. The face recognition model consists of two parts: the single-category classifier and the decision aggregation, where: The single-category classifier generates a distribution feature representation of the corresponding category sub-dataset, and then calculates the outlier value of the target sample, compares the outlier value with the preset threshold, and determines whether the target sample belongs to this category; Decision aggregation aggregates the judgment results of several single-category classifiers to determine the decision aggregation result of the target sample as the output content of the face recognition model; Single-class classifier f i The goal of (x; θ) is to extract the distribution feature representation of the corresponding category sub-dataset, and the objective function F of the training process is expressed as: Where r i is the characteristic representation value of the i-th sub-dataset, x i is the sample of the i-th sub-dataset, is the weight decay regularizer, λ is a hyperparameter greater than 0, W [l] is the weight of the lth layer.

2. The forgetting method for face recognition model based on ensemble learning according to claim 1, characterized in that: In step 1, after obtaining a number of sub-datasets of different categories, the sub-datasets are compressed based on information theory to generate new sub-datasets.

3. The face recognition model forgetting method based on ensemble learning according to claim 1, characterized in that: In step 2.1, each type of sub-dataset is evenly divided into data blocks according to a preset size.

4. The forgetting method for face recognition model based on ensemble learning according to claim 1, characterized in that: In step 2.2, the data blocks of each type of sub-dataset are sorted according to the probability of being deleted, and are sequentially input into the corresponding single-category classifier from low to high for training.

5. The forgetting method for face recognition model based on ensemble learning according to claim 1, characterized in that: The calculation formula for outliers is: anomaly_score=||f i (x i ;θ)-r i || 2 ; Where r i is the characteristic representation value of the i-th sub-dataset, x i is a sample of the i-th sub-dataset, and the single-category classifier f i (x i ; θ) is the feature representation value of the target sample.

6. The forgetting method for face recognition model based on ensemble learning according to claim 1, characterized in that: During the decision aggregation, if only one single-category classifier determines that the target sample belongs to this category, then this category is directly used as the decision aggregation result; if more than one single-category classifier determines that the target sample belongs to this category, then the category with the lowest outlier value is used as the decision aggregation result.

7. A face recognition model forgetting system based on ensemble learning, characterized in that: include: The dataset acquisition module is used to obtain the face dataset and classify it to obtain several sub-datasets of different categories; The classifier initialization training module is used to input each sub-dataset into the corresponding single-category classifier for initialization training. The training process is as follows: Each type of sub-dataset is divided into data blocks; The data blocks of each sub-dataset are sequentially input into the corresponding single-category classifier for training, and the model parameters after each data block is trained on the single-category classifier are recorded; The first decision aggregation module is used to aggregate the decisions of several trained single classifiers to obtain a face recognition model; The classifier retraining module is used to retrain the corresponding single-category classifier based on the user's data forgetting request, using the data block where the user's target data is located and the model parameters corresponding to the previous data block as the starting point of the retraining process; The second decision aggregation module is used to re-aggregate the decision of the retrained single-category classifier and the unchanged single-category classifier to obtain a retrained face recognition model. The face recognition model consists of two parts: the single-category classifier and the decision aggregation, where: The single-category classifier generates a distribution feature representation of the corresponding category sub-dataset, and then calculates the outlier value of the target sample, compares the outlier value with the preset threshold, and determines whether the target sample belongs to this category; Decision aggregation aggregates the judgment results of several single-category classifiers to determine the decision aggregation result of the target sample as the output content of the face recognition model; Single-class classifier f i The goal of (x; θ) is to extract the distribution feature representation of the corresponding category sub-dataset, and the objective function F of the training process is expressed as: Where r i is the characteristic representation value of the i-th sub-dataset, x i is the sample of the i-th sub-dataset, is the weight decay regularizer, λ is a hyperparameter greater than 0, W [l] is the weight of the lth layer.

Images