Continuous deepfake detection method and device, electronic equipment and storage medium

Abstract

The application provides a continuous deepfake detection method and device, electronic equipment and storage medium. The continuous deepfake detection method comprises the following steps: obtaining target image data, detecting the target image data by using a deepfake detection model to obtain a fake detection result; and the deepfake detection model is obtained by using basic training, incremental analysis training and uncertainty guidance expert selection. The application has the beneficial effect of improving the detection accuracy of continuous deepfake.

Description

Technical Field

[0001] This invention relates to the fields of computer and artificial intelligence technology, and in particular to a method, apparatus, electronic device and storage medium for continuous deepfake detection. Background Technology

[0002] With the rapid development of generative artificial intelligence (AIGC), the cost of forging images and videos has significantly decreased. Deepfake technology has seen explosive growth in social media, short video platforms, and online communications. These technologies can generate highly realistic face-swapped videos in seconds, bringing innovative applications to film and television production, virtual avatars, and digital content creation. However, they are also widely used for the spread of fake news and fraudulent activities, seriously threatening the authenticity of multimedia content and public trust.

[0003] Traditional deepfake detection models rely on static training to generalize all known forgery types at once, but they are unable to cope with the distributional shift caused by the rapid iteration of forgery techniques, and cannot achieve long-term defense against deepfake attacks on multimedia data. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, electronic device, and storage medium for detecting persistent deepfakes, thereby improving the detection accuracy of persistent deepfakes.

[0005] One aspect of the present invention provides a method for continuous deepfake detection, comprising: Acquire target image data, and use a deepfake detection model to detect the target image data to obtain forgery detection results; The training of the deepfake detection model includes: The first model is obtained by training on the base dataset using standard gradient descent. The first model includes a model analytic matrix and a feature extractor. The base dataset includes labeled samples. The model analytic matrix is ​​initialized, and incremental analytic training of the first model is performed. The incremental analytic training includes training with a small dataset while the feature extractor is frozen to obtain the first classification prediction output. During incremental analytical training at each stage, the prediction residual of the model's analytical matrix is ​​obtained. The model is then optimized using standard gradient descent based on the prediction residual by a fake perception residual enhancer to obtain the second classification prediction output. The first classification prediction output and the second classification prediction output are aggregated to obtain the stage prediction category and sub-classifier; The sub-classifiers are evaluated using an uncertainty-guided expert selection method to obtain evaluation results, and the deepfake detection model is determined based on the evaluation results.

[0006] According to the aforementioned persistent deepfake detection method, a first model is obtained by performing basic training on a base dataset using standard gradient descent training, including: Through including The base dataset of samples is trained using standard gradient descent to obtain a discriminative feature extractor. The backbone network is then trained using backpropagation optimization. The sample labels are... The value indicates the authenticity of the sample, with 0 representing a fake sample and 1 representing a real sample. The sample is an image. The training of the backbone network is represented as follows: in The dimension representing the feature vector. This is the feature representation after processing by the backbone network. For feature dimensionality reduction processing, This is the feature extraction function for the backbone network. The training data for the base dataset, for Dimensions This is the weight matrix of the fully connected layers in the backbone network. The dimension of the image features output by the backbone network. This represents the output after Softmax processing.

[0007] According to the aforementioned persistent deepfake detection method, the initialization of the model parse matrix includes: Initializing the model's analytical matrix includes freezing the network parameters of the backbone network, and performing feature expansion using a randomly initialized frozen linear projection matrix. Represented as: Where the linear projection matrix is ​​frozen ,and ; Solving in high-dimensional space using closed-form ridge regression yields the following closed-form solution: in, To parse the classification matrix, the basic training representation is as follows: , Each row is a one-hot encoded vector. The regularization coefficient is used to maintain numerical stability and prevent overfitting. The auxiliary matrix during basic training. It is the identity matrix. This is the label matrix for the training data.

[0008] According to the aforementioned continuous deepfake detection method, the incremental parsing training of the first model includes: Number of times in incremental parsing training phase At that time, by obtaining the analytical classification matrix of the previous stage. Perform recursive construction and update to obtain the stage. The analytical classification matrix is The update process is represented as follows: in, This is a zero matrix, used to expand the dimension of the analytical weight matrix; Indicates that it has been re-marked as The one-hot encoded label represents the target category in the current training phase; the auxiliary matrix... It is a stage The regularized inverse of the eigenautocorrelation matrix is ​​the analytical support matrix calculated in closed-form, and the auxiliary matrix is... Its update formula is recursively as follows: in This indicates that features will be extracted. Through the frozen random projection matrix The high-dimensional representation obtained after processing with activation functions; The parsing classifier during incremental parsing training at each stage is updated using the closed-form solution, and the first classification prediction is output through the updated parsing classifier.

[0009] According to the continuous deep forgery detection method, during incremental parsing training at each stage, the prediction residual of the model's parsing matrix is ​​obtained. A forgery-aware residual enhancer is then used to optimize the prediction residual using standard gradient descent to obtain a second classification prediction output, including: The prediction residuals are determined using the residual matrix and the analytical classification matrix, where the residual matrix is: The predicted residuals are: in, Indicates the first The phase involves a frozen backbone network and a frozen expansion matrix. Extracted High-dimensional features of each sample Represents the one-hot encoded label corresponding to the recalibrated category in the current stage; residual matrix Used to capture the residual prediction of the parsing classifier for the newly added categories in the current stage. To analyze the estimated value of the classification matrix; The prediction residuals are compensated using a low-rank residual compensation matrix, which is as follows: in, and For a learnable low-rank matrix, where the hyperparameters are... Used to balance residual modeling capability and parameter efficiency is the dimension of the low-rank matrix; The input to the residual branch is generated using the Tanh activation function through a fake perceptual residual enhancer: The objective function of the fake perception residual enhancer is to optimize the loss by minimizing the cross-entropy loss between the combined output of the parsing classifier and residual compensation and the true label. in, Represents the cross-entropy loss function. The analytical features representing the current stage, This represents the residual path features after Tanh activation. It is a low-rank residual compensation matrix.

[0010] According to the continuous deepfake detection method, the first classification prediction output and the second classification prediction output are aggregated to obtain a stage prediction category and a sub-classifier, including: The first classification prediction output and the second classification prediction output are linearly aggregated to obtain the aggregated result, where the linear aggregation is as follows: The aggregation results are processed using Softmax for binary classification probability to obtain the predicted category for each stage, as well as the sub-classifier trained for each stage.

[0011] According to the aforementioned persistent deepfake detection method, the sub-classifier is evaluated using an uncertainty-guided expert selection method to obtain the evaluation results, including: Randomly discarding extended features yields the outputs of each stage's sub-classifiers. The randomly discarded extended features are: For output dimensions The analytical output and FARE compensation output for uncertainty estimation are as follows: The test samples are sampled N times using Monte Carlo Dropout, and the mean of the output of the t-th sub-classifier is calculated. With variance for: in, Indicates the first The prediction variance of each subclassifier in the two output dimensions; For all stages Calculate the prediction mean of each sub-classifier. With variance The subclassifier with the smallest total variance is selected as the final prediction source: Based on the final prediction source, the evaluation result is as follows: The prediction output of the final prediction source, Category.

[0012] Another aspect of the present invention provides a continuous deepfake detection device, comprising: The first module is used to acquire target image data, and to perform forgery detection on the target image data using a deep forgery detection model to obtain forgery detection results. The training of the deepfake detection model includes: The second module is used to perform basic training on the base dataset using standard gradient descent to obtain the first model. The first model includes a model analytic matrix and a feature extractor, wherein the base dataset includes labeled samples. The third module is used to initialize the model analytic matrix and perform incremental analytic training of the first model. The incremental analytic training includes training with a small dataset while the feature extractor is frozen to obtain the first classification prediction output. The fourth module is used to obtain the prediction residual of the model's analytical matrix during incremental analytical training at each stage, and to optimize the prediction residual using the standard gradient descent method through a fake perception residual enhancer to obtain the second classification prediction output. The fifth module is used to aggregate the first classification prediction output and the second classification prediction output to obtain the stage prediction category and sub-classifier; The sixth module is used to evaluate the sub-classifiers using an uncertainty-guided expert selection method, obtain the evaluation results, and determine the deepfake detection model based on the evaluation results.

[0013] Another aspect of the present invention provides an electronic device, including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method as described above.

[0014] This invention also discloses a computer-readable storage medium storing a program that is executed by a processor to implement the continuous deepfake detection method.

[0015] The beneficial effects of this invention are as follows: By training a dedicated forgery expert for a closed-loop solution, gradient interference and catastrophic forgetting are avoided. No data replay is required and only a single forward propagation is needed, reducing computational and storage overhead. Residual enhancement module: A lightweight forgery-aware residual enhancer module (accounting for only 0.08% of the parameters) is introduced to enhance the fine-grained forgery artifact recognition capability. Based on the uncertainty-based UGES mechanism, dynamic routing without task labels is achieved, which shows significant advantages in anti-forgetting and adaptability, and improves the detection accuracy of continuous deep forgery. Attached Figure Description

[0016] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the continuous deepfake detection method according to an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the continuous deepfake detection framework structure according to an embodiment of the present invention. Figure 2 (a) Training for a deepfake backbone network; Figure 2 (b) Realignment for deepfake analysis; Figure 2 (c) Continuous training for deepfakes.

[0018] Figure 3 This is a comparison chart of the average test accuracy of embodiments of the present invention. Figure 3 (a) represents the average test accuracy of all datasets at different dropout rates after all incremental training phases have been completed; Figure 3 (b) represents the accuracy distribution of different analysis managers on the dataset.

[0019] Figure 4 This is a visualization comparison chart of t-SNE in an embodiment of the present invention.

[0020] Figure 5This is a schematic diagram of a continuous deepfake detection device according to an embodiment of the present invention. Detailed Implementation

[0021] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" can be used interchangeably. Terms such as "first," "second," etc., are used only to distinguish technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the sequential relationship of the indicated technical features. In the following description, the consecutive reference numerals for method steps are for ease of review and understanding. Adjusting the implementation order of steps, in conjunction with the overall technical solution of the present invention and the logical relationship between the various steps, will not affect the technical effect achieved by the technical solution of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0022] refer to Figure 1 , Figure 1 This is a schematic flowchart of a continuous deepfake detection method according to an embodiment of the present invention, which includes, but is not limited to, steps S100-S600: S100: Acquire target image data, perform deepfake detection on the target image data, and obtain fake detection results.

[0023] In some embodiments, the forgery detection result is used to indicate whether the target image data is a fake image.

[0024] Training a deepfake detection model includes: In some embodiments, reference Figure 2 The schematic diagram of the continuous deepfake detection framework shown includes: (a) Backbone training: The visual encoder is trained on the base dataset using cross-entropy loss.

[0025] (b) The analytical readjustment is performed to construct a random projection matrix, followed by closed-loop analytical learning (AL) and residual fitting via FARE.

[0026] (c) For each incremental task, the analytical parameters are expanded and updated, and the FARE module is trained to fit the residuals. During inference, the Uncertainty Guided Expert Selection (UGES) module dynamically selects the most confident expert for the final prediction.

[0027] S200 uses standard gradient descent training on a base dataset to obtain the first model, which includes a model analytic matrix and a feature extractor. The base dataset includes labeled samples.

[0028] In some embodiments, analytic matrix training includes basic training, analytic matrix initialization, and incremental phase analytic training.

[0029] In some embodiments, during the basic training phase (i.e., at t=1), the model first includes... The basic dataset of each sample The above is trained using standard gradient descent to obtain a discriminative feature extractor, with the input image... Corresponding tags This indicates the authenticity of the samples. The backbone network is optimized through backpropagation. : in The dimension representing the feature vector. This is the feature representation after processing by the backbone network. For feature dimensionality reduction processing, This is the feature extraction function for the backbone network. The training data for the base dataset, for Dimensions This is the weight matrix of the fully connected layers in the backbone network. The dimension of the image features output by the backbone network. This represents the output after Softmax processing.

[0030] S300, initialize the model analytic matrix, perform incremental analytic training of the first model, which includes training with a small dataset while the feature extractor is frozen, to obtain the first classification prediction output.

[0031] In some embodiments, analytic matrix initialization includes freezing the backbone network parameters after basic training is completed, and then randomly initializing the frozen linear projection matrix. Perform feature expansion, where : Solving in high-dimensional space using closed ridge regression: The closed-form solution is: in, Each row is a one-hot encoded vector, with real samples and fake samples labeled as 0 and 1, respectively. Parameters This is the regularization coefficient, used to maintain numerical stability and prevent overfitting. Auxiliary matrix. It will be reused in subsequent incremental learning phases; It is the identity matrix. This is the label matrix for the training data.

[0032] In some embodiments, incremental stage parsing training (t>1) means that in each incremental stage, the embodiments of the present invention employ a recursive parsing training strategy to continuously expand and optimize the parsing classification matrix. Specifically, in stage... At that time, the model will receive a small dataset that includes real samples and fake samples specific to the current stage. In this embodiment of the invention, these two categories are relabeled as... They are considered as two new categories.

[0033] This indexing strategy ensures that the model learns two "new" categories at each incremental stage, effectively transforming the original binary incremental detection task into a class incremental learning problem. Through this transformation, the model can gradually absorb new knowledge in a progressive manner. This embodiment of the invention employs an analytical learning method to solve this transformed class incremental learning problem.

[0034] Based on the basic mechanism of parsing learning, the stages Analytical classification matrix By expanding the matrix of the previous stage We recursively build the update mechanism to adapt it to the current task. This update process can be formally represented as: here, Time is a zero matrix. is the dimension of the low-rank matrix, used to expand the dimension of the analytical weight matrix, where This represents the analytical weight matrix for stage t, with output dimension . The first part of the matrix The first column corresponds to the category weights of the previous stage, while the last two columns are initialized to zero to represent the new categories added in the current stage. Indicates that it has been re-marked as The one-hot encoded label corresponds to the target category of the current task.

[0035] Auxiliary matrix It is a stage The regularized inverse of the eigenautocorrelation matrix serves as the analytical support matrix for closed-form computation. This matrix is ​​updated recursively, using the following update formula: In the incremental learning process, by gradually integrating new stage data into the existing solution space, the model can both retain historical knowledge and acquire new knowledge.

[0036] It is worth noting that throughout the incremental learning process, the feature extractor and the linear expansion matrix remain frozen. This indicates that features will be extracted. Through the frozen random projection matrix The high-dimensional representation obtained after activation function.

[0037] Through this mechanism, the output dimension of the parsing classifier is gradually expanded from the basic 2-dimensional stage to the next stage. time Dimensionality. Each stage expands the output space through closed-form analytic updates, enhancing the model's expressive power. For example... Figure 2 As shown, the first... and Dimension corresponding to the first The binary classification results of the dataset.

[0038] S400 obtains the prediction residuals of the model's analytical matrix during incremental analytical training at each stage. It then optimizes the model using standard gradient descent based on the prediction residuals through a fake perception residual enhancer to obtain the second classification prediction output.

[0039] In some embodiments, to enhance the model's ability to discriminate the current stage category, this embodiment of the invention introduces a forgery-aware residual enhancer (FARE) module based on the analytical matrix.

[0040] Residuals and Modeling Objectives: Let the current stage be the k-th stage, and the analytical matrix be... The result has been obtained using a closed-form solution. The current stage's prediction residuals are: Here, Indicates the first The phase involves a frozen backbone network and a frozen expansion matrix. Extracted High-dimensional features of each sample This represents the one-hot encoded label corresponding to the relabeling category in this stage. This represents the estimated value of the analytical classification matrix.

[0041] residual matrix The prediction error of the parsing classifier for the newly added category (real / fake) in this stage is captured. To compensate for this error, this embodiment of the invention introduces a low-rank residual compensation matrix: Here, and It is a learnable low-rank matrix, where the hyperparameters are... Used to balance residual modeling capability and parameter efficiency The dimension of the low-rank matrix is ​​preferably [the dimension of the matrix]. .

[0042] To enhance the information diversity of the compensation path and optimize its objectives, the FARE module employs an activation function different from that used in the analytical path. Specifically, this is achieved by freezing the expansion matrix. For original features After reprojection, the Tanh activation function is applied to generate the input for the residual branch: The FARE module optimizes by minimizing the cross-entropy loss between the combined output of the parsing classifier and residual compensation and the true label. Its objective function is: Here, This represents the cross-entropy loss function. The analytical features representing the current stage, and This represents the residual path characteristics after Tanh activation. It is a low-rank residual compensation matrix.

[0043] This objective is optimized using the standard gradient descent method and is only applied to the residual signals of newly introduced categories in the current stage.

[0044] S500 aggregates the first classification prediction output and the second classification prediction output to obtain the stage prediction category and sub-classifier.

[0045] In some embodiments, the first classification prediction output and the second classification prediction output are linearly aggregated to obtain an aggregation result, wherein the linear aggregation is as follows: The aggregation results are used Perform binary classification probability processing to obtain the predicted category for each stage, and obtain the sub-classifier trained for each stage.

[0046] S600 uses an uncertainty-guided expert selection method to evaluate the sub-classifiers, obtains the evaluation results, and determines the deepfake detection model based on the evaluation results.

[0047] In some embodiments, as the classification matrix is ​​parsed In the incremental phase The continuous expansion eventually formed The output structure of the dimension. This process is functionally equivalent to training. Stage-specific sub-classifiers achieve knowledge reuse through partial parameter sharing.

[0048] For example, each incremental stage introduces two new output channels specifically for distinguishing real / fake faces in the current stage.

[0049] While this multi-stage parsing structure preserves some knowledge from previous tasks, it may introduce interference signals from early stages under limited data conditions, leading to performance drift. To address this issue, this invention proposes a dynamic selection mechanism during inference—Uncertainty-Guided Expert Selection (UGES). This method utilizes Monte Carlo Dropout to evaluate the predictive stability of stage-specific sub-classifiers and selects the most reliable output source based on the uncertainty score.

[0050] In some embodiments, an uncertainty estimation process is employed, including: The current test sample is Its features, after processing by the backbone network and flattening module, are represented as follows: To obtain the outputs of the sub-classifiers at each stage, this embodiment of the invention performs a random discarding operation on the extended features. The extended features after random discarding are: .

[0051] For output dimensions The analytical output and FARE compensation output for uncertainty estimation are as follows: The test samples are sampled NNN times using Monte Carlo Dropout, and the mean of the output of the t-th sub-classifier is calculated. With variance : in, Indicates the first The prediction variance of a subclassifier across two output dimensions. The sum of these two variance values ​​reflects the predictive stability of the subclassifier on the current test sample.

[0052] This invention employs a dynamic selection strategy for all stages. Calculate the prediction mean of each sub-classifier. With variance The system selects the subclassifier with the smallest total variance as the final prediction source. The evaluation results are obtained based on the final prediction source. It is given by the following formula: The prediction output of the final prediction source, Category.

[0053] During the inference phase, this method dynamically selects the most stable expert output by measuring the prediction uncertainty of all sub-classifiers. This mechanism requires no changes to the training framework, only introducing additional sampling and variance estimation steps at test time, thus combining high flexibility and practicality.

[0054] In some embodiments, referring to Algorithm 1 shown in Table 1, it is a continuous deepfake detection algorithm: .

[0055] The embodiments of the present invention are evaluated on four publicly available deepfake face datasets, as detailed below: FaceForensics++ (FF++): A widely used dataset in the field of deepfake detection, including four forgery methods (DeepFakes, Face2Face, FaceSwap, and NeuralTextures), totaling 4000 forged videos and 1000 real videos.

[0056] DFDC Preview (DFDCP): A preview dataset released for the Deepfake Detection Challenge, containing 5214 videos generated using two forgery techniques.

[0057] Deepfake Detection (DFD): Google released a dataset to supplement FF++, which includes more than 3,000 face-swapped videos.

[0058] Celeb-DF v2 (CDF2): Includes 590 real videos and 5639 fake videos of 59 public figures, generated using more advanced face-swapping technology.

[0059] The experiments in this embodiment of the invention follow a four-stage training and evaluation sequence, defined as: {S={FF++,DFDCP,DFD,CDF2}. Each model is trained sequentially on the training sets of these datasets. After each stage is completed, the model is evaluated on the test sets of all currently encountered and previously encountered datasets.

[0060] Evaluation metrics: This invention employs three metrics to evaluate the proposed method: overall accuracy (ACC), average accuracy (AA), and average forgetting rate (AF). Overall accuracy refers to the proportion of correctly predicted results across all test samples, defined as (TP+TN) / n, where TP and TN represent the number of true positives and true negatives, respectively, and n is the total number of samples. Average accuracy (AA) measures the average accuracy across all incremental tasks, defined as... ,in For the first The accuracy of each task This represents the total number of tasks.

[0061] The average forgetting rate (AF) quantifies the average performance degradation during continuous learning, and is calculated as follows: ,in Indicates task Accuracy rate during initial learning This indicates the accuracy rate after the final training phase.

[0062] In terms of implementation details, the deepfakebench tool was used for dataset loading and organization during the data preprocessing stage. The model was trained for 20 epochs on the basic task using the Adam optimizer, with an initial learning rate of... The training time decays by 50% every 5 epochs. Each subsequent incremental task requires an additional 3 epochs of training. Base training uses the full training set of the FF++ (HQ) dataset. In each incremental phase, this embodiment of the invention samples 25 fake and 25 real videos from the current incremental dataset (DFDCP, DFD, and CDF2) to construct a low-resource training set. 20 frames are randomly sampled from each training video, and 10 frames are sampled from each test video. RetinaFace is used for face alignment and cropping, and all facial images are resized to 299×299 pixels before being input into the model.

[0063] This invention systematically compares the method of this embodiment with two classic incremental learning baselines (LwF and DGR) and three deepfake detection-specific methods (DFIL, CoReD, and DMP), as shown in Table 2. For fair comparison, all methods use Xception as the backbone network.

[0064] During the basic training phase, all methods were trained on the full FF++ training set. Results showed that all models achieved high accuracy (ACC) during this phase, demonstrating their strong detection capabilities. In the incremental phase, models were updated using only a limited number of samples from DFDCP, DFD, and CDF2.

[0065] Compared to other methods, the method of this invention mitigates performance degradation during incremental updates by introducing analytical learning and the UGES expert selection mechanism. Notably, even without relying on the replay buffer, the method of this invention maintains the highest ACC on previous tasks and consistently outperforms existing replay-based methods across multiple evaluation scenarios.

[0066] In the incremental phase, only a limited number of samples from DFDCP, DFD, and CDF2 are used for model updates. Compared to other methods, the method of this embodiment effectively mitigates the performance degradation during incremental updates by introducing analytical learning and an expert selection mechanism based on UGES. Notably, even without relying on the replay buffer, the method of this embodiment maintains the highest accuracy (ACC) on previous tasks and consistently outperforms existing replay-based methods in multiple evaluation scenarios. All methods in Table 2 use the Xception backbone network to ensure fairness.

[0067] Table 2 Performance comparison of existing persistent deepfake detection methods (ACC, AA, and AF) .

[0068] To evaluate generalization ability, after incremental training on FF++ and DFDCP, the embodiments of this invention were tested with zero samples on two unseen datasets, DFD and CDF2. No fine-tuning or additional adaptation was performed. As shown in Table 3, the method of the embodiments of this invention consistently outperforms the current state-of-the-art publicly available persistent deep forgery detection model DFIL on unseen datasets, demonstrating excellent adaptability and robustness to changes in forgery style and distribution shifts.

[0069] Table 3. Comparison of generalization performance of DFIL and the method of the present invention on unseen deepfake datasets. .

[0070] To verify the effectiveness of each key component in the framework, embodiments of the present invention conduct a series of ablation experiments to evaluate the impact of FARE and UGES on the performance of persistent deepfake detection. As shown in Table 4, embodiments of the present invention compare four configurations: a baseline model with only analysis and learning (ALonly), a model with only FARE, a model with only UGES, and a complete model.

[0071] Effectiveness of the FARE module. Experimental results show that adding the FARE module consistently improves the average accuracy (AA) across multiple test datasets without using UGES. The gains are particularly significant on datasets with greater forgery diversity, such as CDF2 and DFD, indicating that FARE effectively compensates for the insufficient ability of the analysis matrix to fit the residual signals of new forgery categories.

[0072] refer to Figure 3 The chart showing the average test accuracy comparison is as follows: Figure 3 (a) represents the average test accuracy of all datasets at different dropout rates after all incremental training phases have been completed.

[0073] Specifically, when the dropout rate is 0, the system defaults to averaging the outputs of all binary classification experts, while Figure 3 (b) represents the accuracy distribution of different analysis managers on the dataset.

[0074] Effectiveness of the UGES module. To evaluate the impact of UGES's dynamic selection of the analysis head during inference, this embodiment of the invention analyzes two aspects after completing all incremental training phases: average test accuracy at different dropout rates (see...). Figure 3 (a) and the accuracy distribution of different analysis heads on various datasets ( Figure 3 (b) The results show that there are significant differences in the performance of the analysis head across the datasets, and each dataset shows a clear preference for a specific stage. Compared with the fixed head strategy, the dynamic selection strategy (labeled as mix) provided by UGES consistently achieves the highest accuracy, proving that it can adaptively select the subsystem best suited to the sample distribution.

[0075] Table 4 Ablation study on the impact of UGES and FARE on persistent deepfake detection .

[0076] also, Figure 3 (a) This shows that an appropriate dropout rate is crucial to UGES performance. Optimal performance is achieved with a dropout rate of 0.3, while excessively high dropout rates (such as 0.7) lead to unstable predictions and reduced selection reliability, highlighting UGES's significant contribution to robustness during inference.

[0077] Visualization and qualitative analysis are used to gain a deeper understanding of the model's internal decision-making mechanism and dynamic expert selection strategy. This embodiment of the invention conducts visualization analysis from two perspectives: feature distribution and significant response.

[0078] Expert Selection Preference Analysis. In this embodiment of the invention, it was observed that during inference on the DFDCP test set, the model frequently favored experts trained in the FF++ stage. To investigate this phenomenon, this embodiment of the invention sampled three types of videos: (1) videos randomly selected from FF++; (2) DFDCP videos processed by FF++ experts; and (3) DFDCP videos processed by the model's own experts. Ten frames were extracted at equal intervals from each video, and the average feature representation was calculated using the Xception backbone network and visualized using t-SNE. For details, please refer to [reference needed]. Figure 4 The t-SNE visualization comparison chart shown below. Figure 4 Visualization of FF++ and DFDCP samples in the t-SNE space. It can be observed that some DFDCP samples (green) are closer to FF++ samples (red) in the feature space, thus favoring the parsing expert of the FF++ stage during inference.

[0079] The visualization results show that many DFDCP samples (green) processed by FF++ experts are closely adjacent to FF++ samples (red) in the feature space. This indicates that the forged features of these DFDCP samples are more similar to those seen by FF++, making FF++ experts trained on large-scale datasets more discriminative on such samples. This supports the design of UGES to select experts through a confidence-aware mechanism.

[0080] Training overhead is significantly reduced during continuous deep forgery detection training, as only the FARE module and analysis matrix are updated while the backbone network remains frozen. As shown in Table 5, the method of this embodiment introduces only 0.12M trainable parameters in all incremental stages, accounting for only 0.57% of the total model size.

[0081] Table 5 Comparison of parameters between FARE and backbone network .

[0082] This invention re-examines the persistent deep forgery detection problem from an analytical perspective, revealing a unique structure distinct from traditional incremental class learning. By constructing each forgery type as an independent expert within a fixed binary label space, this invention unlocks the potential of closed-form solutions in incremental learning. The proposed framework combines analytical learning with a lightweight residual enhancement module and a confidence-aware expert selection mechanism, achieving efficient adaptation without data replay or parameter interference. Extensive experiments demonstrate that this method achieves state-of-the-art detection performance while maintaining excellent generalization capabilities in low-resource scenarios.

[0083] Figure 5This is a diagram of a continuous deepfake detection and analysis device according to an embodiment of the present invention. The device includes a first module 510, a second module 520, a third module 530, a fourth module 540, a fifth module 550, and a sixth module 560.

[0084] The system comprises the following modules: First module 510 acquires target image data and uses a deep forgery detection model to detect forgeries, obtaining detection results. Training the deep forgery detection model includes: Second module 520 performs basic training on a base dataset using standard gradient descent to obtain a first model, which includes a model analytic matrix and a feature extractor. The base dataset includes labeled samples. Third module 530 initializes the model analytic matrix and performs incremental analytic training on the first model, including training on a small dataset while the feature extractor remains frozen, obtaining a first classification prediction output. Fourth module 540 acquires the prediction residuals of the model analytic matrix during incremental analytic training at each stage, optimizes the prediction residuals using standard gradient descent with a forgery-aware residual enhancer, and obtains a second classification prediction output. Fifth module 550 aggregates the first and second classification prediction outputs to obtain stage prediction categories and sub-classifiers. Sixth module 560 evaluates the sub-classifiers using an uncertainty-guided expert selection method, obtains evaluation results, and determines the deep forgery detection model based on the evaluation results. Exemplarily, with the cooperation of the first to sixth modules in the device, the embodiment device can acquire target image data, perform detection on the target image data using a deepfake detection model, and obtain a forgery detection result; the training of the deepfake detection model includes: performing basic training on a base dataset using standard gradient descent to obtain a first model, the first model including a model analytic matrix and a feature extractor, wherein the base dataset includes labeled samples; initializing the model analytic matrix, performing incremental analytic training of the first model, the incremental analytic training including training on a small dataset while the feature extractor is kept frozen, to obtain a first classification prediction output; during incremental analytic training at each stage, obtaining the prediction residual of the model analytic matrix, optimizing it using standard gradient descent based on the prediction residual using a forgery-aware residual enhancer, to obtain a second classification prediction output; aggregating the first classification prediction output and the second classification prediction output to obtain a stage prediction category and a sub-classifier; evaluating the sub-classifier using an uncertainty-guided expert selection method to obtain an evaluation result, and determining the deepfake detection model based on the evaluation result. The beneficial effects of this invention are as follows: By training a dedicated forgery expert for a closed-loop solution, gradient interference and catastrophic forgetting are avoided. No data replay is required and only a single forward propagation is needed, reducing computational and storage overhead. Residual enhancement module: A lightweight forgery-aware residual enhancer module (accounting for only 0.08% of the parameters) is introduced to enhance the fine-grained forgery artifact recognition capability. Based on the uncertainty-based UGES mechanism, dynamic routing without task labels is achieved, which shows significant advantages in anti-forgetting and adaptability, and improves the detection accuracy of continuous deep forgery.

[0085] This invention also provides an electronic device, which includes a processor and a memory; The memory stores the program; The processor executes a program to perform the aforementioned persistent deepfake detection method; the electronic device has the function of carrying and running the software system for persistent deepfake detection provided in the embodiments of the present invention, such as a personal computer, minicomputer, mainframe, workstation, network or distributed computing environment, standalone or integrated computer platform, or communicating with charged particle tools or other imaging devices, etc.

[0086] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the continuous deepfake detection method described above.

[0087] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0088] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned persistent deepfake detection method.

[0089] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0090] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, a computer-readable medium can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0092] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0093] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0094] In the description of this specification, references to the terms "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0095] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0096] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for continuous deepfake detection, characterized in that, include: Acquire target image data, and use a deepfake detection model to detect the target image data to obtain forgery detection results; The training of the deepfake detection model includes: The first model is obtained by training on the base dataset using standard gradient descent. The first model includes a model analytic matrix and a feature extractor. The base dataset includes labeled samples. The model analytic matrix is ​​initialized, and incremental analytic training of the first model is performed. The incremental analytic training includes training with a small dataset while the feature extractor is frozen to obtain the first classification prediction output. During incremental analytical training at each stage, the prediction residual of the model's analytical matrix is ​​obtained. The model is then optimized using standard gradient descent based on the prediction residual by a fake perception residual enhancer to obtain the second classification prediction output. The first classification prediction output and the second classification prediction output are aggregated to obtain the stage prediction category and sub-classifier; The sub-classifiers are evaluated using an uncertainty-guided expert selection method to obtain evaluation results, and the deepfake detection model is determined based on the evaluation results. The first model is obtained by performing basic training on the basic dataset using standard gradient descent training, including: Through including The base dataset of samples is trained using standard gradient descent to obtain a discriminative feature extractor. The backbone network is then trained using backpropagation optimization. The sample labels are... The value indicates the authenticity of the sample, with 0 representing a fake sample and 1 representing a real sample. The sample is an image. The training of the backbone network is represented as follows: in The dimension representing the feature vector. This is the feature representation after processing by the backbone network. For feature dimensionality reduction processing, This is the feature extraction function for the backbone network. The training data for the base dataset, for Dimensions This is the weight matrix of the fully connected layers in the backbone network. The dimension of the image features output by the backbone network. This represents the output after Softmax processing; During incremental analytical training at each stage, the prediction residual of the model's analytical matrix is ​​obtained. This residual is then optimized using standard gradient descent via a fake perception residual enhancer to obtain the second classification prediction output, including: The prediction residuals are determined using the residual matrix and the analytical classification matrix, where the residual matrix is: The predicted residuals are: in, Indicates the first The phase involves a frozen backbone network and a frozen expansion matrix. Extracted High-dimensional features of each sample Represents the one-hot encoded label corresponding to the recalibrated category in the current stage; residual matrix Used to capture the residual prediction of the parsing classifier for the newly added categories in the current stage. To analyze the estimated value of the classification matrix; The prediction residuals are compensated using a low-rank residual compensation matrix, which is as follows: in, and For a learnable low-rank matrix, where the hyperparameters are... Used to balance residual modeling capability and parameter efficiency is the dimension of the low-rank matrix; The input to the residual branch is generated using the Tanh activation function through a fake perceptual residual enhancer: The objective function of the fake perception residual enhancer is to optimize the loss by minimizing the cross-entropy loss between the combined output of the parsing classifier and residual compensation and the true label. in, Represents the cross-entropy loss function. The analytical features representing the current stage, This represents the residual path features after Tanh activation. This is a low-rank residual compensation matrix; The step of aggregating the first classification prediction output and the second classification prediction output to obtain the stage prediction category and sub-classifier includes: The first classification prediction output and the second classification prediction output are linearly aggregated to obtain the aggregated result, where the linear aggregation is as follows: The aggregation results are used Perform binary classification probability processing to obtain the stage prediction category and the sub-classifier trained for each stage; The sub-classifiers are evaluated using an uncertainty-guided expert selection method, and the evaluation results include: Randomly discarding extended features yields the outputs of each stage's sub-classifiers. The randomly discarded extended features are: For output dimensions The analytical output and FARE compensation output for uncertainty estimation are as follows: The test samples are sampled N times using Monte Carlo Dropout, and the mean of the output of the t-th sub-classifier is calculated. With variance for: in, Indicates the first The prediction variance of each subclassifier in the two output dimensions; For all stages Calculate the prediction mean of each sub-classifier. With variance The subclassifier with the smallest total variance is selected as the final prediction source: Based on the final prediction source, the evaluation result is as follows: The prediction output of the final prediction source, Category.

2. The continuous deepfake detection method according to claim 1, characterized in that, The initialization of the model analytical matrix includes: Initializing the model's analytical matrix includes freezing the network parameters of the backbone network, and performing feature expansion using a randomly initialized frozen linear projection matrix. Represented as: Where the linear projection matrix is ​​frozen ,and ; Solving in high-dimensional space using closed-form ridge regression yields the following closed-form solution: in, To parse the classification matrix, the basic training representation is as follows: , Each row is a one-hot encoded vector. γ The regularization coefficient is used to maintain numerical stability and prevent overfitting. The auxiliary matrix during basic training. I It is the identity matrix. This is the label matrix for the training data.

3. The continuous deepfake detection method according to claim 2, characterized in that, The incremental analytical training of the first model includes: Number of times in incremental parsing training phase At that time, by obtaining the analytical classification matrix of the previous stage. Perform recursive construction and update to obtain the stage. The analytical classification matrix is The update process is represented as follows: in, This is a zero matrix, used to expand the dimension of the analytical weight matrix; Indicates that it has been re-marked as The one-hot encoded label represents the target category in the current training phase; the auxiliary matrix... It is a stage The regularized inverse of the eigenautocorrelation matrix is ​​the analytical support matrix calculated in closed-form, and the auxiliary matrix is... Its update formula is recursively as follows: in This indicates that features will be extracted. Through the frozen random projection matrix The high-dimensional representation obtained after processing with activation functions; The parsing classifier during incremental parsing training at each stage is updated using the closed-form solution, and the first classification prediction is output through the updated parsing classifier.

4. A continuous deepfake detection device, characterized in that, include: The first module is used to acquire target image data, and to perform forgery detection on the target image data using a deep forgery detection model to obtain forgery detection results. The training of the deepfake detection model includes: The second module is used to perform basic training on the base dataset using standard gradient descent to obtain the first model. The first model includes a model analytic matrix and a feature extractor, wherein the base dataset includes labeled samples. The third module is used to initialize the model analytic matrix and perform incremental analytic training of the first model. The incremental analytic training includes training with a small dataset while the feature extractor is frozen to obtain the first classification prediction output. The fourth module is used to obtain the prediction residual of the model's analytical matrix during incremental analytical training at each stage, and to optimize the prediction residual using the standard gradient descent method through a fake perception residual enhancer to obtain the second classification prediction output. The fifth module is used to aggregate the first classification prediction output and the second classification prediction output to obtain the stage prediction category and sub-classifier; The sixth module is used to evaluate the sub-classifiers using an uncertainty-guided expert selection method, obtain the evaluation results, and determine the deepfake detection model based on the evaluation results; The first model is obtained by performing basic training on the basic dataset using standard gradient descent training, including: Through including The base dataset of samples is trained using standard gradient descent to obtain a discriminative feature extractor. The backbone network is then trained using backpropagation optimization. The sample labels are... The value indicates the authenticity of the sample, with 0 representing a fake sample and 1 representing a real sample. The sample is an image. The training of the backbone network is represented as follows: in The dimension representing the feature vector. This is the feature representation after processing by the backbone network. For feature dimensionality reduction processing, This is the feature extraction function for the backbone network. The training data for the base dataset, for Dimensions This is the weight matrix of the fully connected layers in the backbone network. The dimension of the image features output by the backbone network. This represents the output after Softmax processing; During incremental analytical training at each stage, the prediction residual of the model's analytical matrix is ​​obtained. This residual is then optimized using standard gradient descent via a fake perception residual enhancer to obtain the second classification prediction output, including: The prediction residuals are determined using the residual matrix and the analytical classification matrix, where the residual matrix is: The predicted residuals are: in, Indicates the first The phase involves a frozen backbone network and a frozen expansion matrix. Extracted High-dimensional features of each sample Represents the one-hot encoded label corresponding to the recalibrated category in the current stage; residual matrix Used to capture the residual prediction of the parsing classifier for the newly added categories in the current stage. To analyze the estimated value of the classification matrix; The prediction residuals are compensated using a low-rank residual compensation matrix, which is as follows: in, and For a learnable low-rank matrix, where the hyperparameters are... Used to balance residual modeling capability and parameter efficiency is the dimension of the low-rank matrix; The input to the residual branch is generated using the Tanh activation function through a fake perceptual residual enhancer: The objective function of the fake perception residual enhancer is to optimize the loss by minimizing the cross-entropy loss between the combined output of the parsing classifier and residual compensation and the true label. in, Represents the cross-entropy loss function. The analytical features representing the current stage, This represents the residual path features after Tanh activation. This is a low-rank residual compensation matrix; The step of aggregating the first classification prediction output and the second classification prediction output to obtain the stage prediction category and sub-classifier includes: The first classification prediction output and the second classification prediction output are linearly aggregated to obtain the aggregated result, where the linear aggregation is as follows: The aggregation results are used Perform binary classification probability processing to obtain the stage prediction category and the sub-classifier trained for each stage; The sub-classifiers are evaluated using an uncertainty-guided expert selection method, and the evaluation results include: Randomly discarding extended features yields the outputs of each stage's sub-classifiers. The randomly discarded extended features are: For output dimensions The analytical output and FARE compensation output for uncertainty estimation are as follows: The test samples are sampled N times using Monte Carlo Dropout, and the mean of the output of the t-th sub-classifier is calculated. With variance for: in, Indicates the first The prediction variance of each subclassifier in the two output dimensions; For all stages Calculate the prediction mean of each sub-classifier. With variance The subclassifier with the smallest total variance is selected as the final prediction source: Based on the final prediction source, the evaluation result is as follows: The prediction output of the final prediction source, Category.

5. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the continuous deepfake detection method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the continuous deepfake detection method as described in any one of claims 1-3.

Images