A privacy-protected face anti-counterfeiting method, system, terminal, and medium

Abstract

The present invention provides a privacy-protected facial anti-counterfeiting method and system, which migrates a pre-trained facial anti-counterfeiting model to a user end, including: obtaining a generalizable pre-trained facial anti-counterfeiting model as a first model; inputting a user end facial image into the first model to generate a first pseudo-label; using the first model as an initialization to iteratively update and obtain a second model; conditionally aligning the classifier weight features of the first model with the features output by the second model; dividing the user end facial image into blocks and shuffling the order, using the first model as an initialization, and iterating with the disordered facial image blocks to obtain a third model; combining the third model and the aligned facial image features to update the second model for user end facial anti-counterfeiting. The present invention migrates the pre-trained facial anti-counterfeiting model to the user end for use in the absence of pre-training data to protect facial privacy, effectively improving the accuracy of users using the facial anti-counterfeiting model in different real-world environments.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a privacy-protected face anti-counterfeiting method, system, terminal, and medium. Background Art

[0002] With the development of artificial intelligence (AI), computer vision is increasingly being used in our daily lives and production. Facial recognition, a relatively mature technology, is now used in many interactive intelligent systems, such as identity verification, mobile payments, and access control. Despite its widespread adoption, facial recognition systems also present security risks and are vulnerable to deliberate attacks, such as printed photo attacks, video playback attacks, and 3D mask attacks. To address these security risks, facial anti-counterfeiting technologies have been developed to distinguish genuine from fake faces. Early facial anti-counterfeiting technologies used traditional hand-crafted feature extraction operators, such as HoG and LBP, to model facial features. With the rise of deep learning, more powerful neural network models have been applied to facial anti-counterfeiting tasks. While deep learning methods have achieved excellent performance in internal dataset tests, they generally generalize poorly across datasets. This is due to differences in dataset distribution, such as lighting, imaging resolution, and ethnicity. This hinders the application of facial anti-counterfeiting technologies in complex, real-world scenarios. For example, a company trains a facial anti-counterfeiting model on its own dataset and provides it to users. However, due to the distribution differences between the company's data and the user's data, the performance of the company's model will degrade when used on the user side. Domain adaptation methods from computer vision have been introduced to facial anti-counterfeiting to improve the model's transferability. Domain adaptation methods utilize both the company's pre-trained data and the user's data for training. By aligning the marginal distributions of the pre-trained data and the user's data in the feature space, the pre-trained knowledge is transferred to the user side. However, current domain adaptation methods require the use of the company's pre-training data, which undermines the privacy of facial data and infringes on the company's privacy interests. Furthermore, marginal distribution alignment destroys the category information in the feature space. Therefore, more effective and privacy-preserving facial anti-counterfeiting methods are needed.

[0003] Existing face anti-counterfeiting methods have the following shortcomings: (1) Face anti-counterfeiting models trained on specific datasets have poor migration capabilities and poor accuracy in cross-dataset testing, which affects users' applications in real scenarios; (2) Current face anti-counterfeiting methods require the use of company-pre-trained data, which undermines the privacy of face image data; (3) Existing face anti-counterfeiting methods implement edge distribution alignment in feature space, which destroys the category information in feature space. Summary of the Invention

[0004] The present invention addresses the deficiencies of the prior art and aims to provide a privacy-preserving facial anti-counterfeiting method, system, terminal, and medium.

[0005] According to one aspect of the present invention, a privacy-preserving face anti-counterfeiting method is provided, comprising:

[0006] Obtain a generalizable pre-trained face anti-counterfeiting model as the first model;

[0007] Inputting the user-side face image into the first model to generate a first pseudo label;

[0008] Based on the first pseudo label, using the first model as an initialization, iteratively updating according to the user-side face image to obtain a second model;

[0009] Using the classifier weights of the first model as features, and performing conditional distribution alignment with the facial image features output by the second model;

[0010] Divide the user-side face image into blocks and shuffle the order, perform data augmentation, use the first model as initialization, and iteratively update the shuffled face image blocks to obtain a third model;

[0011] The second model is updated by combining the third model and the aligned facial image features for user-side facial anti-counterfeiting.

[0012] Preferably, the step of obtaining a generalizable pre-trained face anti-counterfeiting model as the first model includes:

[0013] Get face image x S and its corresponding authenticity label y S Construct the pre-training dataset D S ={x S ,y S}, authenticity label y s 0 represents a fake face, and 1 represents a real face;

[0014] The pre-training dataset D S Randomly divide into training subsets and validation subset in and represents the face images of the training subset and the validation subset, and is the corresponding authenticity label;

[0015] Pre-trained models By a classifier h s and a feature extractor g s Composition, initialization parameters are randomly generated;

[0016] The training set face images Input pre-trained model f s , and get the model’s predicted output The cross entropy loss is calculated based on the corresponding true and false labels yStrain:

[0017] Based on the cross entropy loss, the model parameters are updated using gradient descent training until the model training converges to obtain the pre-trained face anti-counterfeiting model f s As the first model.

[0018] Preferably, inputting the face image into the first model to generate a first pseudo label includes:

[0019] The user's face image D T ={x T} Input to the first model f s , get the classification results output by the model Pseudo-labeling and confidence values where h t2s is a classifier, argmax returns the maximum value of an array, and σ is the sigmoid function, that is,

[0020] Filter the confidence value The pseudo labels greater than the threshold γ As the first pseudo label, the threshold γ has a value range of [0, 1].

[0021] Preferably, the first model is used as an initialization based on the first pseudo label, and the second model is obtained by iteratively updating according to the user-side face image, including:

[0022] Initialize the second model , the second model consists of classifier h t and feature extractor g t Its structure is the same as the first model, and its initialization weight parameters are the first model The weight parameter of

[0023] The user end face image D T ={x T} Input to the second model f t , output the classification results of the second model Pseudo-labeling and confidence

[0024] Filter the confidence value The pseudo label greater than the threshold γ is used as the second pseudo label The value range of the threshold γ is [0,1];

[0025] According to the first pseudo label The classification results of the second model output Calculating cross entropy loss

[0026] According to the second pseudo label The classification results of the second model output Calculating cross entropy loss

[0027] The cross entropy loss L supervised by the first pseudo label SR The cross entropy loss L supervised by the second pseudo-label ST Added as the overall loss L SSR =(1-α)·L SR +α·L ST , where the hyperparameter α gradually increases from 0 to 1 as the number of training iterations increases. epoch represents the number of iterations of the current training, and total_epoch represents the total number of iterations of the training.

[0028] According to the overall loss L SSR , using the gradient descent method, update the weight parameters of the second model until convergence.

[0029] Preferably, the method of using the classifier weights of the first model as features and performing conditional distribution alignment with the facial image features output by the second model includes:

[0030] The classifier weights of the first model are used as feature prototypes to characterize the pre-training data, and the feature prototypes to characterize the real face and fake face of the pre-training data are obtained as follows: in Represents the weight parameter corresponding to the true face in the classifier, Represents the weight parameter of the corresponding fake face in the classifier. Input the user's face image into the second model and obtain the feature z output by the model. t =g t (x T ), pseudo labels and confidence values

[0031] Conditional distribution alignment using supervised contrastive loss function,

[0032]

[0033] where N tIndicates the total number of user-side data. represents the output features obtained by inputting the second model into the face image of the i-th user side, Represents the weight parameter corresponding to the mth category in the classifier, Represents the weight parameter corresponding to the jth category in the classifier. τ represents the temperature coefficient, which is set to 0.1. M represents the total number of categories (M = 2 means two categories, true and false).<x,y> Represents the inner product of vectors x and y. This maximizes the similarity between the features of real (fake) faces in the user-side data and the features of the same category in the pre-training data to achieve conditional distribution alignment, while minimizing the similarity between the features of real (fake) faces in the user-side data and the features of different categories in the pre-training data to improve discrimination.

[0034] Preferably, the method of dividing the user-side facial image into blocks and disrupting the order, performing data enhancement, using the first model as initialization, and iteratively updating the randomized facial image blocks to obtain a third model comprises:

[0035] Input user-side face image x T Data enhancement using image block scrambling destroys the structural information of the face and generates a face image x′ with random blocks. T ;

[0036] Use the first model to initialize the third model as the teacher model of the second model in represents a classifier, Represents a feature extractor.

[0037] Preferably, the updating of the second model by combining the third model and the aligned facial image features for user-side facial anti-counterfeiting includes:

[0038] The user-side face image x T and block-random face image x′ T Alternately feed the second model f t and the third model Get two pairs of output probability distributions {P stu ,P′ tea} and {P′ stu ,P tea}, where P stu =σ(h t (g t (x T )) / η),P′ stu =σ(h t (g t (x′ T )) / η), η represents the temperature coefficient, which is set to 0.1;

[0039] Use Kullback Leibler (KL) deviation to measure the difference in probability distribution and use KL deviation as the loss function L TSE =D KL (P′ tea ||P stu )+D KL (P tea ||P′ stu ), to train and update the second model f t The parameter θ t ;

[0040] Based on the second model f t The parameter θ t , update the third model using sliding average Parameters

[0041] The pseudo-label loss function L SSR , the supervised contrast loss function L CDA and the KL deviation loss function L TsE Add them together to get the overall loss function:

[0042] L=L SR +λ1·L CDA +λ2·L TSE

[0043] Where λ1 and λ2 are the coefficients of the loss function;

[0044] According to the overall loss function L, gradient descent is used to train and update model parameters, and the second model is trained until convergence. The finally trained second model is used for face anti-counterfeiting recognition.

[0045] According to a second aspect of the present invention, a privacy-protected face anti-counterfeiting system is provided, comprising:

[0046] A first model module, which obtains a generalizable pre-trained face anti-counterfeiting model as a first model;

[0047] A first pseudo label module, which inputs the user's face image into the first model to generate a first pseudo label;

[0048] A second model module, which is based on the first pseudo label, uses the first model as an initialization, and iteratively updates the user-side face image to obtain a second model;

[0049] an alignment module, which uses the classifier weights of the first model as features and performs conditional distribution alignment with the facial image features output by the second model;

[0050] A third model module, which divides the user-side facial image into blocks and shuffles the order, performs data enhancement, uses the first model as an initialization, and iteratively updates the shuffled facial image blocks to obtain a third model;

[0051] A face anti-counterfeiting module combines the third model and the aligned face image features to update the second model for user-side face anti-counterfeiting.

[0052] According to a third aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute the above-mentioned method or to execute the above-mentioned system when executing the program.

[0053] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the above-mentioned method, or to run the above-mentioned system.

[0054] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0055] (1) The privacy-preserving face anti-counterfeiting method in the embodiment of the present invention can effectively improve the migration performance of the face anti-counterfeiting model, improve its accuracy in user-side testing, and help improve its reliability in real scenarios.

[0056] (2) The privacy-preserving face anti-counterfeiting method in the embodiment of the present invention effectively utilizes the pre-trained model parameters, and effectively migrates the pre-trained model to the user end while protecting the privacy of the facial image data and without accessing the pre-trained data, thereby improving the migration performance on the user end data.

[0057] (3) The privacy-preserving face anti-counterfeiting method in the embodiment of the present invention maximizes the similarity between the features of the real (fake) face in the user-side data and the features of the same category in the pre-training data to achieve conditional distribution alignment, and at the same time minimizes the similarity between the features of the real (fake) face in the user-side data and the features of different categories in the pre-training data to improve the discrimination ability.

[0058] (4) The privacy-preserving face anti-counterfeiting method, computer device, and readable storage medium in the embodiments of the present invention solve the problem of migrating a pre-trained model to differently distributed user terminals in the absence of pre-trained data, and have broad industrial application prospects. For example, using the present invention, a company can provide a pre-trained face anti-counterfeiting model to users, and use a small amount of user-side data to fine-tune the model to improve the migration performance. At the same time, it ensures that the company's data is not accessed during the migration process, thereby protecting the privacy security of the company's data. Therefore, the privacy-preserving face anti-counterfeiting method provided by the present invention can effectively promote the actual deployment of face anti-counterfeiting models. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0060] Figure 1 This is a module principle diagram of a face anti-counterfeiting method with privacy protection according to an embodiment of the present invention;

[0061] Figure 2 This is an attention activation map of the face anti-counterfeiting recognition result according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0063] See also Figure 1 The present invention provides an embodiment of a privacy-preserving face anti-counterfeiting method, which migrates a pre-trained face anti-counterfeiting model to a user end, including:

[0064] S100, obtaining a generalizable pre-trained face anti-counterfeiting model as the first model;

[0065] S200, inputting the user-side face image into the first model obtained in S100 to generate a first pseudo label;

[0066] S300, based on the first pseudo label of S200 and using the first model of S100 as initialization, iteratively updating the user-side face image to obtain a second model;

[0067] S400, using the classifier weights of the first model in S100 as features and performing conditional distribution alignment with the facial image features output by the second model in S300;

[0068] S500, dividing the user's face image into blocks and shuffling the order, performing data enhancement, using the first model of S100 as initialization, and iteratively updating the shuffled face image blocks to obtain a third model;

[0069] S600, combining the third model of S500 and the aligned facial image features to update the second model of S300 for user-side facial anti-counterfeiting.

[0070] This embodiment migrates the trained pre-trained facial anti-counterfeiting model to the user end in the absence of pre-trained data to protect facial privacy, which can effectively improve the accuracy of users using the facial anti-counterfeiting model in different real-world environments, thereby ensuring the privacy and practicality of the facial anti-counterfeiting method.

[0071] In a preferred embodiment of the present invention, S100 is implemented. The specific process is as follows:

[0072] S11. Get face image x S and its corresponding authenticity label y S Construct the pre-training dataset D S ={x S ,y S}, authenticity label y S 0 represents a fake face, and 1 represents a real face;

[0073] S12, the pre-training data set D S Randomly divide into training subsets and validation subset in and represents the face images of the training subset and the validation subset, and is the corresponding authenticity label;

[0074] S13. Pre-trained model By a classifier h s and a feature extractor g s Composition, initialization parameters are randomly generated;

[0075] S14, training set face images Input pre-trained model f s , and get the model’s predicted output The cross entropy loss is calculated based on the corresponding true and false labels yStrain:

[0076] S15. Based on the cross entropy loss, the model parameters are updated using gradient descent training until the model training converges to obtain the pre-trained face anti-counterfeiting model f s As the first model.

[0077] This embodiment uses a large amount of pre-training data to train the model to converge, improve the model's generalization ability, and obtain a generalizable face anti-counterfeiting model. This generalizable face anti-counterfeiting model facilitates subsequent migration and use on user-side face images without causing a significant loss in face anti-counterfeiting discrimination accuracy.

[0078] In a preferred embodiment of the present invention, S200 is implemented, which specifically includes the following steps:

[0079] S21, the user's face image D T ={x T} Input to the first model f s , get the classification results output by the model Pseudo-labeling and confidence values where h t2s is a classifier, argmax returns the maximum value of an array, and σ is the sigmoid function, that is,

[0080] S22. Screening confidence value Pseudo labels greater than the threshold γ As the first pseudo label, the threshold γ here ranges from [0,1].

[0081] This embodiment uses the first model to generate pseudo-labels for unlabeled facial images on the user side, which are then used to train the second model. Simultaneously, a threshold is used to filter pseudo-labels with high confidence values, eliminating the noise introduced by pseudo-labels and enhancing the second model's ability to maintain anti-counterfeiting accuracy even for perturbed facial images.

[0082] In a preferred embodiment of the present invention, S300 is implemented, which specifically includes the following steps:

[0083] S31, initializing the second model , the second model consists of feature extractor h t and classifier g t The second model structure is the same as the first model, and the weight parameters are initialized to the first model. The weight parameter of the second model ;

[0084] S32, the user end face image D T ={x T} Input to the second model f t , output the classification results of the second model Pseudo-labeling and confidence

[0085] S33. Screening confidence value The pseudo label greater than the threshold γ is used as the second pseudo label The value range of the threshold γ is [0,1];

[0086] S34, according to the first pseudo label The classification results output by the second model Calculating cross entropy loss

[0087] S35: Based on the second pseudo label The classification results output by the second model Calculating cross entropy loss

[0088] S36, the cross entropy loss L supervised by the first pseudo label SR Cross entropy loss L with the second pseudo-label supervision ST Added as the overall loss L SSR =(1-α)·L SR +α·L ST , where the hyperparameter α gradually increases from 0 to 1 as the number of training iterations increases. epoch represents the number of iterations of the current training, and total_epoch represents the total number of iterations of the training.

[0089] S37, according to the overall loss L SSR , using the gradient descent method, update the weight parameters of the second model until convergence.

[0090] In this embodiment, the second model has two classifiers, h t2s and h t , respectively and Calculate the loss corresponding to the first and second pseudo labels respectively.

[0091] This embodiment uses a generalizable first model as initialization to improve the generalization ability of the model. At the same time, the first pseudo-label and the second pseudo-label are used to jointly train the user-side facial image, which can eliminate the influence of pseudo-label noise and improve the stability and generalization of the trained and updated second model, which is conducive to the facial image features generated in the subsequent S400 to more effectively distinguish between real and fake faces.

[0092] In a preferred embodiment of the present invention, S400 is implemented, which specifically includes the following steps:

[0093] S41, using the classifier weight of the first model as the feature prototype to characterize the pre-training data, the feature prototypes to characterize the real face and fake face of the pre-training data are obtained respectively as in Represents the weight parameter corresponding to the true face in the classifier, Represents the weight parameter corresponding to the fake face in the classifier;

[0094] S42, input the face image into the second model, and obtain the feature z output by the model t =g t (x T ), pseudo labels and confidence values

[0095] S43. Using supervised contrast loss function,

[0096]

[0097] where N t Indicates the total number of user-side data. represents the output features obtained by inputting the second model into the face image of the i-th user side, Represents the weight parameter corresponding to the mth category in the classifier, represents the weight parameter corresponding to the jth category in the classifier, τ represents the temperature coefficient, which is set to 0.1, and M represents the total number of categories (M = 2 means two categories: true and false).<x,y> Represents the inner product of vectors x and y.

[0098] The model parameters are trained using a supervised contrastive loss function to maximize the similarity of features within the same category, achieving conditional distribution alignment. Features of different categories have lower similarity, and the greater the difference in categories, the lower the similarity. This results in a model that can identify the category of a user's facial image based on similar features, improving its discrimination capabilities.

[0099] This embodiment can achieve conditional distribution alignment of the features of the user-side data with the features of the same category in the pre-training data, which is conducive to transferring the knowledge learned on the pre-training data to the facial images of the user side, and enhancing the anti-counterfeiting identification accuracy of the second model when deployed and used on the user side.

[0100] In a preferred embodiment of the present invention, the implementation of S500 specifically includes the following steps:

[0101] S51. Input user-side face image x T Data enhancement using image block scrambling destroys the structural information of the face and generates a face image x′ with random blocks. T ;

[0102] S52, use the first model to initialize the third model as the teacher model of the second model in represents the classifier, Represents a feature extractor.

[0103] In a preferred embodiment of the present invention, S600 is implemented, which specifically includes the following steps:

[0104] S61, the user's face image x T and block-random face image x′ T Alternately feed into the second model f t and the third model Get two pairs of output probability distributions {P stu ,P′ tea} and {P′ stu ,P tea}, where P stu =σ(h t (g t (x T )) / η),P′ stu =σ(h t (g t (x′ T )) / η), η represents the temperature coefficient, which is set to 0.1;

[0105] S62. Since the original data and the data after block shuffling contain the same true and false details, the probability distributions they output should be consistent. Use KL deviation to measure the difference in probability distribution and use KL deviation as the loss function L TSE =D KL (P′ tea ||P stu )+D KL (P tea ||P′ stu ), to train and update the second model f t The parameter θ t ;

[0106] S63: Update the third model using sliding average Parameters

[0107] S64, the pseudo label loss function L SSR , supervised contrast loss function L CDA and KL deviation loss function L TSE Add them together to get the overall loss function:

[0108] L=L SR +λ1·L CDA +λ2·L TSE

[0109] Where λ1 and λ2 are the coefficients of the loss function, λ1 = 0.3, λ2 = 0.15. Based on the overall loss function L, gradient descent is used to train and update the model parameters. The second model is trained until convergence. The final trained second model is used for face anti-counterfeiting recognition.

[0110] This embodiment uses the third model as a teacher model to assist the second model in training, which is beneficial for the model to mine the facial anti-counterfeiting features corresponding to this part of the facial image on the user side, and further improve the anti-counterfeiting identification accuracy of the second model when deployed and used on the user side.

[0111] Based on the same inventive concept, in other embodiments of the present invention, a privacy-protected face anti-counterfeiting system is provided, including a first model module, a first pseudo-label module, a second model module, an alignment module, a third model module and a face anti-counterfeiting module; the first model module obtains a generalizable pre-trained face anti-counterfeiting model as a first model; the first pseudo-label module inputs the user-side face image into the first model to generate a first pseudo-label; the second model module is based on the first pseudo-label, takes the first model as initialization, and iteratively updates according to the user-side face image to obtain a second model; the alignment module uses the classifier weight of the first model as a feature, and performs conditional distribution alignment with the face image features output by the second model; the third model module divides the user-side face image into blocks and shuffles the order, performs data enhancement, takes the first model as initialization, and iteratively updates according to the disordered face image blocks to obtain a third model; the face anti-counterfeiting module updates the second model in combination with the third model and the aligned face image features for user-side face anti-counterfeiting.

[0112] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the terminal can be used to execute the above-mentioned method, or to run the above-mentioned system.

[0113] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute the above method or run the above system.

[0114] In order to verify the effectiveness of the above embodiments, experiments were conducted on five public datasets, namely OULU (O), CASIA (C), Replay (I), MSU (M) and CelebA-Spoof (CA). The test protocol of the experiment is to train on several pre-trained datasets. After the training is completed, a converged pre-trained face anti-counterfeiting model is obtained. The pre-trained face anti-counterfeiting model is used to initialize the user-side face anti-counterfeiting model, and the user-side face anti-counterfeiting model is trained on a training set of a small amount of unlabeled user-side data. After the training is completed, it is tested on the test set of the user-side data. For example, experiments are conducted on four datasets, namely OULU (O), CASIA (C), Replay (I) and MSU (M). O&M&I-to-C means training on the three datasets of O, M, and I, training on the training set of the C dataset, and testing on the test set of the C dataset. Evaluation indicators include HTER and AUC. HTER is the mean total error rate, which is equal to the average of the false acceptance rate and the false rejection rate; AUC is the area under the ROC curve (receiver operating characteristic curve). The smaller the HTER, the larger the AUC, indicating better model performance.

[0115] The specific comparison results can be seen in Tables 1, 2, 3, and 4 below. Table 1 is a comparison of the migration accuracy evaluation results of an embodiment of the present invention (on the O, C, I, and M datasets), Table 2 is a comparison of the migration accuracy evaluation results of an embodiment of the present invention (on the C, I, and M datasets), Table 3 is a comparison of the migration accuracy evaluation results of an embodiment of the present invention (on the M, I, C datasets and the M, I, O datasets), and Table 4 is a comparison of the migration accuracy evaluation results of an embodiment of the present invention (on the M, C, O, and CA datasets). In the table, the lower the HTER and the higher the AUC, the better the migration performance of the model.

[0116] Table 1

[0117]

[0118] Table 2

[0119]

[0120] Table 3

[0121]

[0122] Table 4

[0123]

[0124] like Figure 2The attention activation map of the face anti-counterfeiting recognition results shown in the figure shows that for real face samples, the method in this embodiment shows a dense attention map, effectively capturing the physical structure of the face. For forged face samples, our method focuses on the printed paper, finger parts, and device edges to distinguish forged faces.

[0125] The above experiments show that the privacy-preserving face anti-counterfeiting method proposed in the embodiments of the present invention can achieve higher test accuracy of user-side applications in the absence of pre-training data and protect the privacy of facial data, and achieve a lower average error rate and a higher area under the curve, indicating that the model has better migration performance and thus has a wider range of application scenarios.

[0126] The privacy-preserving face anti-counterfeiting method in the embodiment of the present invention migrates the company's pre-trained face anti-counterfeiting model to the user end to achieve conditional distribution alignment when there is no pre-training data to protect the privacy of the face. The similarity between the features of the real (fake) face in the user-end data and the features of the same category in the pre-training data is maximized, and the similarity between the features of the real (fake) face in the user-end data and the features of different categories in the pre-training data is minimized to improve the recognition ability. It can effectively improve the accuracy of the face anti-counterfeiting model in different real-world environments, ensure the privacy of the face anti-counterfeiting method, and the overall algorithm is simple and effective, which can achieve a higher accuracy of face anti-counterfeiting generalization, so that users can deploy it in complex real-world scenarios to perform face anti-counterfeiting tasks.

[0127] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A privacy-preserving face anti-counterfeiting method, which migrates a pre-trained face anti-counterfeiting model to the user end, characterized in that: include: Obtain a generalizable pre-trained face anti-counterfeiting model as the first model; Inputting the user-side face image into the first model to generate a first pseudo label; Based on the first pseudo label, using the first model as an initialization, iteratively updating according to the user-side face image to obtain a second model; Using the classifier weights of the first model as features, and performing conditional distribution alignment with the facial image features output by the second model; Divide the user-side face image into blocks and shuffle the order, perform data augmentation, use the first model as initialization, and iteratively update the shuffled face image blocks to obtain a third model; The second model is updated by combining the third model and the aligned facial image features for user-side facial anti-counterfeiting.

2. The privacy-preserving face anti-counterfeiting method according to claim 1, characterized in that: The method of obtaining a generalizable pre-trained face anti-counterfeiting model as a first model includes: Get face image x S and its corresponding authenticity label y S Construct the pre-training dataset D S ={x S ,y S }, authenticity label y S 0 represents a fake face, and 1 represents a real face; The pre-training dataset D S Randomly divide into training subsets and validation subset in and represents the face images of the training subset and the validation subset, and is the corresponding authenticity label; Pre-trained models By a classifier h s and a feature extractor g s Composition, initialization parameters are randomly generated; The training set face images Input pre-trained model f s , and get the model’s predicted output According to the corresponding authenticity label Calculate the cross entropy loss as Based on the cross entropy loss, the model parameters are updated using gradient descent training until the model training converges to obtain the pre-trained face anti-counterfeiting model f s As the first model.

3. The privacy-preserving face anti-counterfeiting method according to claim 2, characterized in that: Inputting the user-side face image into the first model to generate a first pseudo label includes: The user's face image Dx = {x T } Input to the first model f s , get the classification results output by the model Pseudo-labeling and confidence values where h t2s is the classifier, g s is a feature extractor, argmax returns the maximum value of an array, and σ is the sigmoid function, i.e. Filter the confidence value The pseudo label greater than the threshold γ is used as the first pseudo label The value range of the threshold γ is [0, 1].

4. The privacy-preserving face anti-counterfeiting method according to claim 3, characterized in that: The method of obtaining a second model based on the first pseudo label and using the first model as an initialization and iteratively updating the user-side face image includes: Initialize the second model The second model consists of the classifier h t and feature extractor g t Its structure is the same as the first model, and its initialization weight parameters are the first model The weight parameter of The user end face image D T ={x T } Input to the second model f t , output the classification results of the second model Pseudo-labeling and confidence Filter the confidence value The pseudo label greater than the threshold γ is used as the second pseudo label The value range of the threshold γ is [0,1]; According to the first pseudo label The classification results of the second model output Calculating cross entropy loss According to the second pseudo label The classification results of the second model output Calculating cross entropy loss The cross entropy loss L supervised by the first pseudo label SR The cross entropy loss L supervised by the second pseudo-label ST Added as the overall loss L SSR =(1-α)·L SR +α·L ST , where the hyperparameter α gradually increases from 0 to 1 as the number of training iterations increases. epoch represents the number of iterations of the current training, and total_epoch represents the total number of iterations of the training; According to the overall loss L SSR , using the gradient descent method, update the weight parameters of the second model until convergence.

5. The privacy-preserving face anti-counterfeiting method according to claim 1, characterized in that: The method of using the classifier weight of the first model as a feature and performing conditional distribution alignment with the facial image feature output by the second model includes: The classifier weights of the first model are used as feature prototypes to characterize the pre-training data, and the feature prototypes to characterize the real face and fake face of the pre-training data are obtained as follows: in Represents the weight parameter corresponding to the true face in the classifier, Represents the weight parameter corresponding to the fake face in the classifier; Input the user's face image into the second model to obtain the feature z output by the model t =g t (x T ), pseudo labels and confidence values Conditional distribution alignment using supervised contrastive loss function, where N t Indicates the total number of user-side data. represents the output features obtained by inputting the second model into the face image of the i-th user side, Represents the weight parameter corresponding to the mth category in the classifier, Represents the weight parameter corresponding to the jth category in the classifier, τ represents the temperature coefficient, which is set to 0.1, M represents the total number of categories, and M=2 represents two categories: true and false.<x,y> Represents the inner product of vectors x and y.

6. The privacy-preserving face anti-counterfeiting method according to claim 5, characterized in that: The method of dividing the user-side face image into blocks and disrupting the order, performing data enhancement, using the first model as initialization, and iteratively updating the randomized face image blocks to obtain a third model includes: Input user-side face image x T Data enhancement using image block scrambling destroys the structural information of the face and generates a face image x′ with random blocks. T ; Use the first model to initialize the third model as the teacher model of the second model in represents a classifier, Represents a feature extractor.

7. The privacy-preserving face anti-counterfeiting method according to claim 6, characterized in that: The updating of the second model by combining the third model and the aligned facial image features for user-side facial anti-counterfeiting includes: The user-side face image x T and block-random face image x′ T Alternately feed the second model f t and the third model Get two pairs of output probability distributions {P stu ,P′ tea } and {P′ stu ,P tea }, where P stu =σ(h t (g t (x T )) / η),P′ stu =σ(h t (g t (x′ T )) / η), η represents the temperature coefficient, which is set to 0.1; Use KullbackLeibler, that is, KL deviation to measure the difference in probability distribution, and use KL deviation as the loss function L TSE =D KL (P′ tea ||P stu )+D KL (P tea ||P′ stu ), to train and update the second model f t The parameter θ t ; Based on the second model f t The parameter θ t , update the third model using sliding average Parameters The pseudo-label loss function L SSR , the supervised contrast loss function L CDA and the KL deviation loss function L TSE Add them together to get the overall loss function: L=L SR +λ1·L CDA +λ2·L TSE Where λ1 and λ2 are the coefficients of the loss function; According to the overall loss function L, gradient descent is used to train and update model parameters, and the second model is trained until convergence. The finally trained second model is used for face anti-counterfeiting recognition.

8. A privacy-protected face anti-counterfeiting system, characterized in that: include: A first model module, which obtains a generalizable pre-trained face anti-counterfeiting model as a first model; A first pseudo label module, which inputs the user's face image into the first model to generate a first pseudo label; A second model module, which is based on the first pseudo label, uses the first model as an initialization, and iteratively updates the user-side face image to obtain a second model; an alignment module, which uses the classifier weights of the first model as features and performs conditional distribution alignment with the facial image features output by the second model; A third model module, which divides the user-side facial image into blocks and shuffles the order, performs data enhancement, uses the first model as an initialization, and iteratively updates the shuffled facial image blocks to obtain a third model; A face anti-counterfeiting module, which combines the third model and the aligned face image features to update the second model for user-side face anti-counterfeiting.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to perform the method according to any one of claims 1 to 7, or run the system according to claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to perform the method according to any one of claims 1 to 7, or to run the system according to claim 8.

Images