A face forgery detection method and system based on federated incremental learning

Abstract

The application discloses a face forgery detection method and system based on federal incremental learning, comprising the following steps: constructing a federal incremental learning framework, each client is provided with a local face forgery detection model, and a global server is provided with a global face forgery detection model; the local face forgery detection model is subjected to cross-entropy training in a basic training stage, generates a forgery substitute sample in an incremental training stage, and is used as training input together with a real face sample and a forged face sample; after training, the model parameters are uploaded to the global server for aggregation; the global face forgery detection model is updated based on a difference perception aggregation strategy; an adversarial perturbation is trained based on the global face forgery detection model converged in the present stage and the real face sample, and a perturbation pool is refreshed until a preset termination condition is reached; and a to-be-tested face image is identified as real or fake based on the trained global face forgery detection model. The application can relieve catastrophic forgetting and improve cross-task generalization ability and robustness.

Description

Technical Field

[0001] This invention relates to the field of face forgery detection technology, specifically to a face forgery detection method and system based on federated incremental learning. Background Technology

[0002] Most existing face forgery detection methods employ a centralized training scheme, where all data is aggregated and trained once on a central server to learn the features of forged faces. However, as new forgery techniques and attacks emerge, traditional methods often require retraining the model using all historical data to maintain detection performance. Otherwise, the model's detection performance against older types of forgeries will drop sharply. This repeated full-scale training is computationally expensive and makes it difficult to adapt to new threats in real time.

[0003] For example, the incremental learning method and system for detecting forged face images and videos disclosed in patent publication number CN116310754A introduces a dynamically scalable incremental learning framework, which retains the memory of old knowledge while the model absorbs new knowledge. However, this scheme mainly operates in centralized scenarios, requiring the uploading of a large amount of face image data from various terminals to the server, which brings serious data privacy risks. It relies on saving and replaying old task samples to suppress forgetting, and in order to maintain the balance of positive and negative samples, it needs to construct training samples in pairs, which brings additional data storage and training burden. When the size of old samples is limited or the distribution is significantly different from that of the new task, the model is still prone to overfitting the current task and catastrophic forgetting. In addition, this method lacks a cross-data domain collaborative update mechanism, making it difficult to utilize forged samples scattered across multiple institutions and devices to improve generalization. Summary of the Invention

[0004] To overcome the defects and shortcomings of existing technologies, this invention provides a face forgery detection method and system based on federated incremental learning. This invention adopts a federated incremental learning framework, which enables the model to continuously adapt to various forgery types as the task flow changes, while ensuring the data privacy of all participants. This alleviates catastrophic forgetting and significantly improves the model's cross-task generalization ability and robustness, making it more suitable for real-world face forgery threat scenarios that are multi-source, heterogeneous, and continuously evolving.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention provides a face forgery detection method based on federated incremental learning, comprising the following steps:

[0007] A federated incremental learning framework is constructed, consisting of multiple clients and a global server. Each client has a local face spoofing detection model, and the global server has a global face spoofing detection model.

[0008] Obtain a training dataset of facial images and train a local face forgery detection model, including a basic training phase and an incremental training phase.

[0009] During the basic training phase, the client performs cross-entropy training on the local face spoofing model based on the face image training dataset;

[0010] During the incremental training phase, the client adds adversarial perturbations to real face samples to generate fake substitute samples, and inputs them along with real face samples and fake face samples into the local face spoofing model for model training.

[0011] After the local face forgery detection model is trained, the model parameters are uploaded to the global server for aggregation.

[0012] The global face forgery detection model is updated based on a difference-aware aggregation strategy.

[0013] The global server trains adversarial perturbation based on the global face forgery detection model that has converged in this stage and real face samples, refreshes the perturbation pool, and sends it to the client in the next stage until the global face forgery detection model reaches the preset termination condition.

[0014] The trained global face forgery detection model is used to identify the authenticity of the face image under test.

[0015] As a preferred technical solution, during the incremental training phase, a total loss function is constructed, expressed as:

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] in, Represents the total loss function. This represents the true / false classification loss function. Represents the total training entropy. This represents the knowledge distillation loss of soft-labels. The feature knowledge distillation loss represents the real sample. This represents the feature knowledge distillation loss of the forged surrogate sample. A dataset for training face images. Image tags, Client-side local dataset, This indicates a local face spoofing detection model. This represents the entropy of each stage obtained by random sampling from the adversarial perturbation pool. The total training entropy is obtained by summing the results. , Fake substitute samples Indicates the student model on the sample The probability of it being identified as counterfeit. Indicates the teacher model on the sample The probability of it being identified as counterfeit. This represents the sample set for distillation. This represents the feature extractor for the current stage, corresponding to the student model. This represents the feature extractor from the previous stage, corresponding to the teacher model. This represents a real face sample at the current stage.

[0023] As a preferred technical solution, the global face forgery detection model is updated based on a difference-aware aggregation strategy, specifically including:

[0024] The parameters of the local face forgery detection model uploaded in this round are sampled M times independently. In the m-th sampling, the average model of the e selected local face forgery detection models is calculated, denoted as . , represented as:

[0025] ;

[0026] in, Local face forgery detection model;

[0027] Perform a weighted summation on the e local face forgery detection models sampled in the m-th round to obtain the aggregated model weighted aggregated model under this round of sampling. , represented as:

[0028] ;

[0029] in, Indicates the first The weight of each client;

[0030] After M samplings, the average of the M weighted aggregation models is taken to obtain the final global face forgery detection model for this round.

[0031] As a preferred technical solution, the first The weight of each client is represented as follows:

[0032] ;

[0033] in, For the first The sample size of data held by each client For all The sum of local client data at each stage. For hyperparameters, For unnormalized weights, The weights are normalized to [0,1]. = The difference between the local face forgery detection model and the average model. Due to model parameter differences, Norm.

[0034] As a preferred technical solution, the global server trains adversarial perturbation based on the global face forgery detection model converged in this stage and real face samples, and refreshes the perturbation pool, specifically as follows:

[0035] ;

[0036] in, For symbolic functions, for learning rate, For the global face forgery detection model trained in stage t, Represents a real image Countering disturbances Forged substitute samples obtained by pixel-level addition Countering disturbances Iterate continuously, adding it to the adversarial disturbance pool P, and then sending it to the client at the start of the next phase.

[0037] This invention also provides a face forgery detection system based on federated incremental learning, comprising: a federated incremental learning framework construction module, a client data receiving module, a client data sending module, a global server data receiving module, a global server data sending module, a training module, a forgery substitution sample generation module, a difference-aware aggregation module, an adversarial perturbation training module, and a training control module.

[0038] The incremental learning framework building module is used to build a federated incremental learning framework, which includes multiple clients and a global server. Each client has a local face spoofing detection model, and the global server has a global face spoofing detection model.

[0039] The client data receiving module is used to receive the face image training dataset and the parameters and adversarial perturbation pool of the global face forgery detection model issued by the global server.

[0040] The client data sending module is used to send parameters of the local face forgery detection model;

[0041] The global server data receiving module is used to receive parameters of the local face forgery detection model and receive real face samples;

[0042] The global server data sending module is used to send global face forgery detection model parameters and adversarial perturbation pool to each client.

[0043] The training module is used for training a local face forgery detection model, including a basic training phase and an incremental training phase.

[0044] During the basic training phase, the client performs cross-entropy training on the local face spoofing model based on the face image training dataset;

[0045] During the incremental training phase, the client inputs fake substitute samples, real face samples, and fake face samples into the local face spoofing model for model training;

[0046] After the local face forgery detection model is trained, the model parameters are uploaded to the global server for aggregation.

[0047] The fake substitute sample generation module is used to superimpose adversarial perturbations onto real face samples to generate fake substitute samples.

[0048] The difference-aware aggregation module is used to update the global face forgery detection model based on the difference-aware aggregation strategy.

[0049] The adversarial perturbation training module is used to train adversarial perturbations based on the global face forgery detection model converged in this stage and real face samples, refresh the perturbation pool, and send it to the client in the next stage.

[0050] The training control module is used to determine whether the global face forgery detection model has reached the preset termination condition.

[0051] The face forgery detection is used to identify the authenticity of the face image to be tested based on the trained global face forgery detection model.

[0052] As a preferred technical solution, during the incremental training phase, a total loss function is constructed, expressed as:

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] in, Represents the total loss function. This represents the true / false classification loss function. Represents the total training entropy. This represents the knowledge distillation loss of soft-labels. The feature knowledge distillation loss represents the real sample. This represents the feature knowledge distillation loss of the forged surrogate sample. A dataset for training face images. Image tags, Client-side local dataset, This indicates a local face spoofing detection model. This represents the entropy of each stage obtained by random sampling from the adversarial perturbation pool. The total training entropy is obtained by summing the results. , Fake substitute samples Indicates the student model on the sample The probability of it being identified as counterfeit. Indicates the teacher model on the sample The probability of it being identified as counterfeit. This represents the sample set for distillation. This represents the feature extractor for the current stage, corresponding to the student model. This represents the feature extractor from the previous stage, corresponding to the teacher model. This represents a real face sample at the current stage.

[0060] As a preferred technical solution, the difference-aware aggregation module is used to update the global face forgery detection model based on the difference-aware aggregation strategy, specifically including:

[0061] The parameters of the local face forgery detection model uploaded in this round are sampled M times independently. In the m-th sampling, the average model of the e selected local face forgery detection models is calculated, denoted as . , represented as:

[0062] ;

[0063] in, Local face forgery detection model;

[0064] Perform a weighted summation on the e local face forgery detection models sampled in the m-th round to obtain the aggregated model weighted aggregated model under this round of sampling. , represented as:

[0065] ;

[0066] in, Indicates the first The weight of each client;

[0067] After M samplings, the average of the M weighted aggregation models is taken to obtain the final global face forgery detection model for this round.

[0068] As a preferred technical solution, the first The weight of each client is represented as follows:

[0069] ;

[0070] in, For the first The sample size of data held by each client For all The sum of local client data at each stage. For hyperparameters, For unnormalized weights, The weights are normalized to [0,1]. = The difference between the local face forgery detection model and the average model. Due to model parameter differences, Norm.

[0071] As a preferred technical solution, the adversarial perturbation training module is used to train adversarial perturbations based on the global face forgery detection model converged in this stage and real face samples, and to refresh the perturbation pool, specifically as follows:

[0072] ;

[0073] in, For symbolic functions, for learning rate, For the global face forgery detection model trained in stage t, Represents a real image Countering disturbances Forged substitute samples obtained by pixel-level addition Countering disturbances Iterate continuously, adding it to the adversarial disturbance pool P, and then sending it to the client at the start of the next phase.

[0074] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0075] (1) Based on the federated incremental learning collaborative training of data without leaving the domain, the present invention realizes the federated incremental learning collaborative update, which solves the technical problems that multi-terminal face data is difficult to share centrally due to privacy and compliance restrictions and that the model is difficult to continuously adapt to the new forgery distribution due to the continuous emergence of forgery types. It can realize multi-party collaborative modeling without uploading the original face data and make the model continuously updated and quickly absorb new forgery knowledge with the task stage, thereby maintaining the long-term online detection capability of the model.

[0076] (2) This invention is based on the generation and distribution of adversarial perturbation pools, and synthesizes fake substitute samples on the client side to participate in training. It retains the historical fake features without examples, which solves the technical problems of storage overhead and potential privacy risks caused by relying on the long-term storage and playback of historical samples in existing incremental face fake detection. By simulating the distribution of historical fakes with fake substitute samples, the dependence on historical samples and the burden of end-side storage are significantly reduced, thereby reducing the risk of privacy leakage and improving system security.

[0077] (3) The present invention adopts a cross-stage knowledge retention mechanism that combines feature knowledge distillation and soft label distillation. By using dual-channel knowledge retention to alleviate forgetting, it solves the problem of the decline in old forgery recognition ability and decision boundary drift caused by catastrophic forgetting of the model during continuous learning. It achieves the technical effect of the model maintaining stable detection performance of existing forgery types and maintaining stable discrimination boundary while learning new forgery types, thereby reducing overfitting and performance fluctuation.

[0078] (4) The present invention adopts a difference-aware aggregation strategy, which improves the adaptation to new forgery and global stability through robust aggregation. The weight of the client comprehensively considers its sample size and the difference between the local model and the mean model. It solves the problem that global model aggregation is prone to bias and performance fluctuations are easily diluted by a small amount of new forgery information under non-independent and identically distributed and unbalanced data conditions. While protecting the contribution of the mainstream distribution, it improves the effectiveness of client updates with a small amount of new forgery data, and enhances the training stability of the global model and its adaptability to new forgery types. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the overall implementation architecture of the face forgery detection method based on federated incremental learning of the present invention;

[0080] Figure 2 This is a flowchart illustrating the face forgery detection method based on federated incremental learning of the present invention. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0082] Example 1

[0083] like Figure 1 , Figure 2 As shown, this embodiment provides a face forgery detection method based on federated incremental learning. It uses federated incremental learning to continuously train on newly emerging forgery types, ultimately enabling the global model to maintain good forgery detection capabilities on both new and old data. The specific steps include:

[0084] Step 1: Construct a federated incremental learning framework;

[0085] Federated incremental learning is a distributed training paradigm that combines privacy protection with continuous learning capabilities. It consists of a central server and multiple client sets. Each client... Owning its private data This data is stored and processed locally without uploading the raw data. Under the coordination of the server, each client only uploads its local model parameters. The server then aggregates these parameters and distributes the updated global model, thus achieving collaborative modeling without data leaving the local domain. Federated incremental learning organizes the training process in the form of a phased task sequence within a distributed data flow. The unified detector is trained, and the training process includes a basic training phase. With incremental training phase The basic training phase is used to obtain a global face forgery detection model with initial detection capabilities; in the incremental training phase, as new detection tasks continue to emerge at different stages, the server updates the global model from the previous stage. The data is sent to the client as an initialization model; client i can only access local data for the current stage. Training is performed without accessing the original historical sample set. By iteratively and collaboratively updating the basic training phase and the incremental training phase, the global detection model can continuously adapt to new detection tasks without sharing original data or accessing historical original data, and mitigate catastrophic forgetting to maintain effective recognition capabilities for learned tasks.

[0086] Preferably, the federated incremental learning framework includes a central server S and a set of k clients C={1,…,k}. The local clients include multiple data centers, each data center is equipped with a local face spoofing detection model, and the server is equipped with a global face spoofing detection model.

[0087] Preferably, the local face spoofing detection model and the global face spoofing detection model have the same structure, which includes a feature extractor. and classification head ;

[0088] Step 2: Obtain the face image dataset, preprocess the face image dataset, and divide the preprocessed dataset into training set, validation set and test set;

[0089] In this embodiment, the face image data can be from the FaceForensics++ dataset, DFDC dataset, Celeb-DF dataset, WildDeepfake dataset, FaceShifter dataset, DeepFakeDetection dataset, or DeepperForensics-1.0 dataset. 20,000 processed images are created for each method, and face detection and alignment are performed uniformly using DSFD. After cropping, the input scale is fixed at 224×224, and then fed into the local face forgery detection model, which includes real and forged face images, to provide data support for the subsequent training of the face forgery detection model.

[0090] In this embodiment, the client can obtain face image data from the data center, which contains multiple fake datasets. Each client selects 60% of the data from all datasets in this stage, ensuring that the data distribution of each client is distributed and non-independent and identically distributed, in order to support the training of the federated incremental learning model. The datasets are distributed and sent to the face fake detection model of each client to train the model respectively.

[0091] In this embodiment, the face image data is divided into a training set, a validation set, and a test set, with a ratio of 8:1:1. The training set accounts for 80% and is used for model training; the validation set accounts for 10% and is used to monitor the training effect and prevent overfitting; and the test set accounts for 10% and is used to evaluate the final performance of the model.

[0092] Step 3: Training the local face forgery detection model;

[0093] The training process is divided into a basic training phase and an incremental training phase. The basic training phase... Used to obtain a basic face forgery detection model during the incremental training phase. We continuously introduce new types of forgery data and train the learning of the forgery features corresponding to the forgery types at that stage.

[0094] Specifically, the training process of the local face forgery detection model at each stage includes the following steps:

[0095] The 224×224 image, which has undergone face detection, alignment, and cropping preprocessing, is input into the local face spoofing model for detection. The feature extractor (pre-trained EfficientNet-B4) extracts discriminative features that can distinguish between real and fake faces.

[0096] The extracted features are input into the classifier head for training. The classifier head performs forgery detection on the preprocessed face image based on the input features, and obtains the forgery detection result, which corresponds to a binary decision to determine the authenticity of the face.

[0097] Basic training phase This is used to obtain a face forgery detection model to be incrementally learned as the base model, which includes a feature extractor for extracting image features. And a classification head that assigns detection categories to image features. In this embodiment, since face forgery detection is essentially considered a binary classification task, the cross-entropy loss function is used as the loss function for classifying true and false faces. The client uses a local dataset... The optimization is performed using cross-entropy, and its form is as follows:

[0098] ;

[0099] in, For training images, (0 represents the real class, 1 represents the fake class) are the corresponding tags;

[0100] Incremental training phase It mainly consists of the following parts: face forgery detection model, forgery substitute sample generation module, and knowledge retention module;

[0101] In the fake substitute sample generation module, the server generates the global model obtained in the previous stage. The anti-disturbance pool P is also distributed to the client. After each client receives the data, it randomly samples adversarial perturbations from the adversarial perturbation pool in each iteration. Overlay onto the local real face image The above is used to generate a dataset of forged alternative samples containing historical traces of forgery. This is used to simulate the distribution of historical forgery features and assign labels. , will forge alternative sample datasets Real human face images With fake samples Together, they are used to train a local face forgery detection model. The model needs to maintain a high confidence level in predicting "forgery" from these forged substitute samples. To achieve this, a pseudo-sample entropy loss term is added in stage t, which can be expressed as:

[0102] ;

[0103] Entropy at each stage obtained by random sampling from the adversarial perturbation pool And sum them up to get the total training entropy The total training entropy reflects both the impact of the current stage and the cumulative impact of previous stages. This loss constraint ensures that the distribution of historical forged features is maintained without replaying historical forged images.

[0104] In the knowledge retention module, soft-label knowledge distillation and feature-level knowledge distillation are performed. Specifically, soft-label knowledge distillation includes:

[0105] In the context of federated incremental face forgery detection, data on newly forged faces in the new stage is often scarce and exhibits significant distributional shifts. Direct training with hard labels can easily lead to forgetting of historical forgery distributions. Soft-label knowledge distillation uses the soft probabilities provided by the previous stage's global face forgery detection model as the learning objective, explicitly conveying the teacher's uncertainty and relative confidence relationship regarding the "true / false" boundary. This allows for the maintenance of old knowledge while learning new forgeries. The loss calculation formula is as follows:

[0106] ;

[0107] in, Indicates the student model on the sample The probability of it being identified as counterfeit. Represents the sigmoid function; Indicates the teacher model on the sample The probability of it being identified as counterfeit; , These represent the logit outputs of the student and teacher models, respectively. , These represent the temperatures on the student and teacher sides, respectively, and are used to smooth the logit before performing the sigmoid function. This represents the sample set used for distillation. Indicates the number of samples in this batch;

[0108] Feature-level knowledge distillation specifically includes:

[0109] As training parameters are continuously updated, the distribution consistency of forged surrogate samples may be disrupted. Furthermore, the introduction of new forgery types may alter the distribution of real faces, thus affecting the generation of forged surrogate samples. Therefore, the feature extractor from the previous stage is used... Performing feature knowledge distillation, maintaining consistency between fake surrogate samples and real samples, helps preserve knowledge from old tasks.

[0110] Distillation of feature knowledge from forged surrogate samples:

[0111] ;

[0112] Distillation of feature knowledge from real samples:

[0113] ;

[0114] in, The feature extractor (student) represents the current stage. This represents the feature extractor (teacher) from the previous stage. This represents a set of forged substitute samples; This represents the set of real samples at the current stage.

[0115] In summary, at this stage When a new face spoofing attack occurs, the overall loss consists of the sum of the basic training loss and the knowledge distillation term mentioned above:

[0116] ;

[0117] in, , The hyperparameter used to measure the weight of the regularization term, This represents the true / false classification loss function. Represents the total training entropy. This represents the knowledge distillation loss of soft-labels. The feature knowledge distillation loss represents the real sample. This represents the feature knowledge distillation loss of forged surrogate samples;

[0118] After the local face forgery detection model is trained, the model parameters are uploaded to the global server for aggregation.

[0119] Step 4: Update the global face forgery detection model using a difference-aware aggregation strategy;

[0120] To address the data heterogeneity among clients in federated incremental face forgery detection, a difference-aware aggregation strategy is designed. This strategy retains the contribution of mainstream data distributions while appropriately amplifying the weight of clients carrying a small number of new forgery types, thereby improving the global model's adaptability to new forgery attacks. Specifically, each data center has a local face forgery detection model, denoted as [model name missing]. In each round of communication, M samplings are performed independently; in the m-th sampling, the average model of the e selected client models is first calculated, denoted as... :

[0121] ;

[0122] Client weight Taking into account its sample size and the difference between the local model and the mean model = , No. Weight of each client Defined as:

[0123] ;

[0124] in, For the first The sample size of data held by each client For all The sum of local client data at each stage. For hyperparameters; Due to model parameter differences, Norm, For unnormalized weights, The weights are normalized to [0,1].

[0125] Perform a weighted summation on the e local models sampled in the m-th round to obtain the aggregated model weighted aggregated model under this round of sampling. :

[0126] ;

[0127] After M samplings, the final global model for this round is formed by averaging these M weighted aggregation models:

[0128] ;

[0129] The updated global model is then sent to the local client to begin the next round of training. Steps 3 and 4 are executed iteratively until the global model converges in this stage.

[0130] Step 5: Update and distribute the anti-disturbance pool on the global server;

[0131] At the end of each training phase, the global server trains adversarial perturbations based on the global model converged in that phase and public real samples, and refreshes the perturbation pool. At phase t, the server uses the model trained in phase t. and real subset samples The goal is to find perturbation vectors that can mislead real images into being mistaken for false ones. , satisfy Treating adversarial perturbations as strong discriminative features, gradient descent is used for refinement to minimize the entropy between fake surrogate samples and false labels, thus making the model more convinced that they are fake. The update is as follows:

[0132] ;

[0133] in, It is a symbolic function; for The learning rate; The global face forgery detection model trained in stage t; A forged substitute sample obtained by pixel-level addition of a real image and an adversarial perturbation is defined as... Countering disturbances Iterate continuously until Crossing the decision boundary; when the number of samples successfully modified to the target prediction exceeds a preset threshold. Stop at that time. Obtain the counter-perturbation. Then, it is added to the anti-disturbance pool P and sent to the client at the start of the next phase;

[0134] Iteratively execute steps 2 to 5 above until the global model in the final stage reaches the preset termination condition. The preset termination condition includes reaching a specified number of communication rounds or the global model converging, and using the global face forgery detection model to identify the authenticity of the face image to be tested.

[0135] This invention's federated incremental learning-based face forgery detection method achieves phased continuous updates of the detection model under multi-client collaborative conditions without sharing original face data. This allows the global model to adapt promptly to the emergence of new forgery attacks, thereby improving the ability to identify new forgery types and the efficiency of online updates. Simultaneously, this invention generates forgery substitute samples through adversarial perturbation to simulate the distribution of historical forgery features. These samples can participate in incremental training without saving historical forgery samples, significantly reducing end-side storage overhead and mitigating potential privacy risks. Furthermore, combined with a cross-stage knowledge preservation mechanism, it effectively alleviates catastrophic forgetting during incremental training, ensuring that the model maintains stable recognition capabilities for existing forgery patterns while learning new tasks.

[0136] Example 2

[0137] This embodiment provides a face forgery detection system based on federated incremental learning, which implements the face forgery detection method based on federated incremental learning in Embodiment 1 above. It can maintain the ability to identify existing forgery types while maintaining a high detection accuracy for new forgery methods, and output a clear judgment of authenticity or forgery. The system specifically includes: a federated incremental learning framework construction module, a client data receiving module, a client data sending module, a global server data receiving module, a global server data sending module, a training module, a forgery substitution sample generation module, a difference perception aggregation module, an adversarial perturbation training module, and a training control module.

[0138] The federated incremental learning framework building module is used to build the federated incremental learning framework, which includes multiple clients and a global server. Each client has a local face spoofing detection model, and the global server has a global face spoofing detection model.

[0139] The client data receiving module is used to receive preprocessed face image data and to receive parameters of the global face forgery detection model and adversarial perturbation pool from the global server. The client data sending module is used to send parameters of the local face forgery detection model.

[0140] The global server data receiving module is used to receive parameters of the local face forgery detection model uploaded from the local client and to receive public real face image data. The global server data sending module is used to send global face forgery detection model parameters and adversarial perturbation pool to each client.

[0141] The training module is in the basic training phase. Perform cross-entropy training; during the incremental training phase In this phase, real, fake, and fake substitute samples are used as training data, and the overall loss function is minimized jointly. ;

[0142] The fake substitute sample generation module in the incremental training phase In each stage, perturbations need to be randomly extracted from the adversarial perturbation pool and added pixel by pixel to the real face image data of the current period on the local end to generate fake substitute samples to participate in the training of the local face forgery detection model;

[0143] The difference perception aggregation module performs M independent subsamplings on the parameters of the local face forgery detection model uploaded in this round, weighted aggregation on the e-end each time, and then weighted averages the M results to generate the global model for this round.

[0144] At the end of each stage, the adversarial perturbation training module trains adversarial perturbations based on public real face image data and a converged global model, refreshes the adversarial perturbation pool, and distributes it to the local client for use in the next stage of training.

[0145] The training control module is used to determine whether the preset termination condition has been met. If the preset termination condition is met, the above module is stopped from being called; otherwise, the above module is called repeatedly. The preset termination condition includes the number of client communications and the convergence of the global face forgery detection model.

[0146] Face forgery detection is used to identify the authenticity of face images under test based on a trained global face forgery detection model.

[0147] In this embodiment, the client data receiving module receives preprocessed face image data and parameters of the face forgery detection model and adversarial perturbation pool from the global server. The global server data receiving module receives parameters of the face forgery detection model uploaded from the local client and receives public real face image data. The client data sending module reports the face forgery detection model parameters on its own end, and the global server data sending module sends global face forgery detection model parameters and adversarial perturbation pool to each client. The local client, through the forgery substitution sample generation module, randomly extracts perturbations from the adversarial perturbation pool in each iteration, adds them pixel-by-pixel to the current public real face image data on its own end, and generates forgery substitution samples to participate in training together with the training data of this stage. The local client updates the local model through the training module, using cross-entropy for training in the basic stage. In the incremental phase, using the current real / fake samples and fake substitute samples as training data, the overall loss function is jointly minimized. The global server, through the difference-aware aggregation module, performs M independent subsampling operations on the local face forgery detection model uploaded in this round, selecting e local clients for weighted aggregation each time, and then averaging the M aggregated models to obtain the final global model for this round of communication. The global server trains adversarial perturbations through the adversarial perturbation training module. After the global model converges at each stage, the adversarial perturbation is trained based on public real face image data and the converged global model, the adversarial perturbation pool is refreshed, and it is distributed to local clients in the new stage. The training control module is used to determine whether a preset termination condition has been met. If the preset termination condition is met, the above modules are stopped; otherwise, the above modules are called repeatedly. The preset termination condition includes reaching a specified number of communication rounds or global model convergence. In this embodiment, the adversarial perturbation is only used in the training stage to construct forged substitute samples to preserve the distribution of historical forged features, but it is not superimposed in the inference stage.

[0148] This invention introduces a federated incremental learning framework, in which each client completes training locally and only exchanges model parameters. Multiple parties collaborate and continuously learn new fake face attacks, while maintaining the ability to distinguish existing fake types. It can cover cross-domain data distributions that are not independent and identically distributed, while avoiding the privacy and compliance risks brought about by the centralized aggregation of original face data.

[0149] This invention maintains historical forgery knowledge through adversarial perturbation. The global side trains the adversarial perturbation based on a converged global model and a publicly available set of real faces, and refreshes the adversarial perturbation pool in stages. The client synthesizes forged substitute samples to simulate the distribution of historical forged features, without the need to store historical forged samples, which significantly reduces storage overhead.

[0150] This invention proposes a dual-channel knowledge distillation method to stabilize boundaries and resist forgetting. At the feature level, intermediate layer distillation based on mean square error is used, and at the soft label level, temperature-controlled Kullback-Leibler (KL) distillation is used to synergistically achieve the preservation of the historical distribution of the representation space and the calibration stability of the decision space.

[0151] This embodiment is applicable to face spoofing governance scenarios involving "edge-cloud" collaboration. The local client can be deployed on mobile terminal apps, IoT and edge devices, as well as on local servers or private clouds of enterprises or institutions. It is used to perform spoofing detection and incremental training within their respective data boundaries without uploading the original face data. Specific applications include content moderation and pre-upload detection on social platforms, real-time anti-spoofing for park access control and exam security checks, and evidence collection and identification by media and the judiciary. The server is deployed on cloud nodes, facing internet platforms, cross-institutional industry alliances, or public safety clouds. It is responsible for organizing the aggregation of federated incremental learning models, unified evaluation and version management, and coordinating rapid adaptation of each end when new tasks are detected. This continuously improves the detection capabilities against new spoofing methods and cross-domain data while ensuring privacy compliance.

[0152] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A face forgery detection method based on federated incremental learning, characterized in that, Includes the following steps: A federated incremental learning framework is constructed, consisting of multiple clients and a global server. Each client has a local face spoofing detection model, and the global server has a global face spoofing detection model. Obtain a training dataset of facial images and train a local face forgery detection model, including a basic training phase and an incremental training phase. During the basic training phase, the client performs cross-entropy training on the local face spoofing model based on the face image training dataset; During the incremental training phase, the client adds adversarial perturbations to real face samples to generate fake substitute samples, and inputs them along with real face samples and fake face samples into the local face spoofing model for model training. After the local face forgery detection model is trained, the model parameters are uploaded to the global server for aggregation. The global face forgery detection model is updated based on a difference-aware aggregation strategy. The global server trains adversarial perturbation based on the global face forgery detection model that has converged in this stage and real face samples, refreshes the perturbation pool, and sends it to the client in the next stage until the global face forgery detection model reaches the preset termination condition. The trained global face forgery detection model is used to identify the authenticity of the face image under test.

2. The face forgery detection method based on federated incremental learning according to claim 1, characterized in that, During the incremental training phase, the total loss function is constructed as follows: ； ； ； ； ； ； in, Represents the total loss function. This represents the true / false classification loss function. Represents the total training entropy. This represents the knowledge distillation loss of soft-labels. The feature knowledge distillation loss represents the real sample. This represents the feature knowledge distillation loss of the forged surrogate sample. A dataset for training face images. Image tags, Client-side local dataset, This indicates a local face spoofing detection model. This represents the entropy of each stage obtained by random sampling from the adversarial perturbation pool. The total training entropy is obtained by summing the results. , Fake substitute samples Indicates the student model on the sample The probability of it being identified as counterfeit. Indicates the teacher model on the sample The probability of it being identified as counterfeit. This represents the sample set for distillation. This represents the feature extractor for the current stage, corresponding to the student model. This represents the feature extractor from the previous stage, corresponding to the teacher model. This represents a real face sample at the current stage.

3. The face forgery detection method based on federated incremental learning according to claim 1, characterized in that, The global face forgery detection model is updated based on a difference-aware aggregation strategy, specifically including: The parameters of the local face forgery detection model uploaded in this round are sampled M times independently. In the m-th sampling, the average model of the e selected local face forgery detection models is calculated, denoted as . , represented as: ； in, Local face forgery detection model; Perform a weighted summation on the e local face forgery detection models sampled in the m-th round to obtain the aggregated model weighted aggregated model under this round of sampling. , represented as: ； in, Indicates the first The weight of each client; After M samplings, the average of the M weighted aggregation models is taken to obtain the final global face forgery detection model for this round.

4. The face forgery detection method based on federated incremental learning according to claim 3, characterized in that, No. The weight of each client is represented as follows: ； in, For the first The sample size of data held by each client For all The sum of local client data at each stage. For hyperparameters, For unnormalized weights, The weights are normalized to [0,1]. = The difference between the local face forgery detection model and the average model. Due to model parameter differences, Norm.

5. The face forgery detection method based on federated incremental learning according to claim 1, characterized in that, The global server trains adversarial perturbations based on the global face forgery detection model converged in this stage and real face samples, and refreshes the perturbation pool, specifically as follows: ； in, For symbolic functions, for learning rate, For the global face forgery detection model trained in stage t, Represents a real image Countering disturbances Forged substitute samples obtained by pixel-level addition Countering disturbances Iterate continuously, adding it to the adversarial disturbance pool P, and then sending it to the client at the start of the next phase.

6. A face forgery detection system based on federated incremental learning, characterized in that, include: The module includes a federated incremental learning framework construction module, a client data receiving module, a client data sending module, a global server data receiving module, a global server data sending module, a training module, a fake substitute sample generation module, a difference-aware aggregation module, an adversarial perturbation training module, and a training control module. The incremental learning framework building module is used to build a federated incremental learning framework, which includes multiple clients and a global server. Each client has a local face spoofing detection model, and the global server has a global face spoofing detection model. The client data receiving module is used to receive the face image training dataset and the parameters and adversarial perturbation pool of the global face forgery detection model issued by the global server. The client data sending module is used to send parameters of the local face forgery detection model; The global server data receiving module is used to receive parameters of the local face forgery detection model and receive real face samples; The global server data sending module is used to send global face forgery detection model parameters and adversarial perturbation pool to each client. The training module is used for training a local face forgery detection model, including a basic training phase and an incremental training phase. During the basic training phase, the client performs cross-entropy training on the local face spoofing model based on the face image training dataset; During the incremental training phase, the client inputs fake substitute samples, real face samples, and fake face samples into the local face spoofing model for model training; After the local face forgery detection model is trained, the model parameters are uploaded to the global server for aggregation. The fake substitute sample generation module is used to superimpose adversarial perturbations onto real face samples to generate fake substitute samples. The difference-aware aggregation module is used to update the global face forgery detection model based on the difference-aware aggregation strategy. The adversarial perturbation training module is used to train adversarial perturbations based on the global face forgery detection model converged in this stage and real face samples, refresh the perturbation pool, and send it to the client in the next stage. The training control module is used to determine whether the global face forgery detection model has reached the preset termination condition. The face forgery detection is used to identify the authenticity of the face image to be tested based on the trained global face forgery detection model.

7. The face forgery detection system based on federated incremental learning according to claim 6, characterized in that, During the incremental training phase, the total loss function is constructed as follows: ； ； ； ； ； ； in, Represents the total loss function. This represents the true / false classification loss function. Represents the total training entropy. This represents the knowledge distillation loss of soft-labels. The feature knowledge distillation loss represents the real sample. This represents the feature knowledge distillation loss of the forged surrogate sample. A dataset for training face images. Image tags, Client-side local dataset, This indicates a local face spoofing detection model. This represents the entropy of each stage obtained by random sampling from the adversarial perturbation pool. The total training entropy is obtained by summing the results. , Fake substitute samples Indicates the student model on the sample The probability of it being identified as counterfeit. Indicates the teacher model on the sample The probability of it being identified as counterfeit. This represents the sample set for distillation. This represents the feature extractor for the current stage, corresponding to the student model. This represents the feature extractor from the previous stage, corresponding to the teacher model. This represents a real face sample at the current stage.

8. The face forgery detection system based on federated incremental learning according to claim 6, characterized in that, The difference-aware aggregation module is used to update the global face forgery detection model based on the difference-aware aggregation strategy, specifically including: The parameters of the local face forgery detection model uploaded in this round are sampled M times independently. In the m-th sampling, the average model of the e selected local face forgery detection models is calculated, denoted as . , represented as: ； in, Local face forgery detection model; Perform a weighted summation on the e local face forgery detection models sampled in the m-th round to obtain the aggregated model weighted aggregated model under this round of sampling. , represented as: ； in, Indicates the first The weight of each client; After M samplings, the average of the M weighted aggregation models is taken to obtain the final global face forgery detection model for this round.

9. The face forgery detection system based on federated incremental learning according to claim 8, characterized in that, No. The weight of each client is represented as follows: ； in, For the first The sample size of data held by each client For all The sum of local client data at each stage. For hyperparameters, For unnormalized weights, The weights are normalized to [0,1]. = The difference between the local face forgery detection model and the average model. Due to model parameter differences, Norm.

10. The face forgery detection system based on federated incremental learning according to claim 1, characterized in that, The adversarial perturbation training module is used to train adversarial perturbations based on the global face forgery detection model converged in this stage and real face samples, and to refresh the perturbation pool, specifically as follows: ； in, For symbolic functions, for learning rate, For the global face forgery detection model trained in stage t, Represents a real image Countering disturbances Forged substitute samples obtained by pixel-level addition Countering disturbances Iterate continuously, adding it to the adversarial disturbance pool P, and then sending it to the client at the start of the next phase.

Images