A real environment depth video forgery tracing method

Abstract

The application discloses a real environment depth video forgery tracing method, aiming at the problems of uneven confidence distribution of known and unknown category samples, high pseudo-label noise and uncertain number of unknown categories in the real environment video frame depth forgery tracing scene, the application evaluates the pseudo-label quality, combines the prediction confidence, category interval information, enhanced consistency and prototype stability information to generate sample quality scores; based on the sample prediction belonging to the partition, the asymmetric dynamic threshold of the known and unknown category partitions is established, the pseudo-label correction and screening of the unlabeled samples are carried out; the quality perception weighting of the consistency training process is carried out in combination with the training stage information; and the dynamic prototype update is carried out based on the prototype usage rate and the prototype stability. The application can reduce the risk of pseudo-label noise propagation, improve the stability of model training, and enhance the attribution ability to unknown depth forgery categories and the adaptability in the real environment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, digital video frame forensics and deep learning, specifically to a method for tracing the source of deep forgeries in real-world environments, and more particularly to a method for tracing the source of deep forgeries in real-world environments based on asymmetric pseudo-label correction and dynamic prototype updating. Background Technology

[0002] With the rapid iteration of deep generation, image editing and multimodal technologies, the quality of Deepfake audio and video has been greatly improved. It is close to real content in terms of visual realism, local texture consistency and semantic coherence, which brings serious challenges to network content governance, media authenticity verification, judicial evidence collection and platform security review.

[0003] Current deepfake detection methods largely focus on "binary classification of true and false," while deepfake attribution can further identify the generation method and source of the forged content, offering greater application value in accountability and evidence collection. Most existing attribution methods are based on the closed-set assumption, which assumes that all forgery categories are known during training, and the model only needs to classify within a predefined range. However, in real-world open environments, novel forgery methods constantly emerge, leading to numerous erroneous attribution attempts.

[0004] To address this scenario, existing solutions often employ a training approach of "a small number of labeled samples + a large number of unlabeled samples," leveraging mechanisms such as pseudo-labels, self-supervised consistency training, and prototype clustering to enhance the model's ability to utilize unknown samples. However, existing deepfake source tracing methods for real-world environments generally suffer from the following problems when utilizing unlabeled samples: First, there is a confidence bias between known and unknown categories. In the early stages of model training, known category features are prioritized, leading to generally high prediction confidence for known category samples and generally low confidence for unknown category samples. This severe imbalance in the distribution of the two classes directly lowers the quality of pseudo-label generation. Second, a single, uniform confidence threshold is extremely unfriendly to unknown categories. An excessively high threshold filters out a large number of low-confidence unknown category samples, resulting in insufficient learning of unknown categories. Conversely, an excessively low threshold introduces a large number of unknown category samples. Noisy pseudo-labels disrupt training stability; third, the reliability measurement of pseudo-labels is singular, relying solely on the highest class probability as the selection criterion, failing to cover key information such as feature consistency before and after sample augmentation and clarity of class boundaries, frequently resulting in misjudgments of "high confidence on the surface but unstable features" or "low confidence but reliable structure," significantly reducing the accuracy of pseudo-label selection; fourth, the prototype mechanism has poor adaptability when the number of unknown categories is uncertain. Most prototype-based methods require a fixed number of prototypes to be set in advance, but the number of unknown pseudo-categories in real-world scenarios is completely unpredictable. Insufficient prototypes can lead to incorrect aggregation of different unknown categories, while excessive prototypes can cause redundant prototypes, prototype drift, and training oscillations. Existing pruning schemes rely solely on a single usage rate indicator, lacking joint constraints on prototype stability, resulting in very limited effectiveness.

[0005] In summary, existing methods for tracing deepfakes in real-world environments generally suffer from high pseudo-label noise, insufficient utilization of unknown category samples, poor training stability, and weak adaptability to the number of unknown categories. Therefore, there is an urgent need for a real-world deep video forgery tracing method based on asymmetric pseudo-label correction and dynamic prototype updating to improve the model's ability to effectively utilize unlabeled samples, especially those of unknown categories, and enhance its attribution stability and practicality under real-world conditions. Summary of the Invention

[0006] The purpose of this invention is to provide a method for tracing the source of deep video forgeries in real-world environments. By constructing a multi-index pseudo-label quality scoring system, an asymmetric dynamic screening mechanism for known and unknown partitions, a quality-aware phased consistency training mechanism, and a dynamic prototype update mechanism, this method achieves robust utilization of unlabeled samples, improves the model's attribution ability for unknown forgery categories, and thus solves the problems raised in the background art.

[0007] To achieve the above objectives, this invention provides a method for tracing the source of forgeries in real-world deep video. This method is based on asymmetric pseudo-label correction and dynamic prototype updating, and includes the following steps:

[0008] Step S1: Obtain training data for processing deepfakes in real-world environments;

[0009] We obtain labeled and unlabeled deepfake sample sets, and use them as model inputs after preprocessing. The labeled sample set is used to supervise the model's learning of known fake categories, containing samples of known fake categories and their corresponding labels. The unlabeled sample set is used for model training in real-world scenarios, containing both samples of known and unknown categories.

[0010] Step S2: Construct a feature extraction network and a prototype classification model;

[0011] A feature extraction network for deepfake tracing is constructed to extract feature representations of input samples. Simultaneously, a prototype classification module is built to output category prediction results based on the similarity between sample features and prototype vectors. The prototype classification module includes: a known category prototype set, representing known fake categories with supervised labels available during training; and a candidate unknown category prototype set, carrying samples with potential unknown categories in real-world environments. The candidate unknown category prototype set can be redundantly configured, i.e., setting a number of candidate prototypes far exceeding the actual number of unknown categories in the early stages of training, providing adjustable space for subsequent dynamic prototype updates.

[0012] Step S3: Construction and prediction of dual views for unlabeled samples;

[0013] For unlabeled samples, weak augmentation views and strong augmentation views are constructed respectively. These two types of views are then input into the feature extraction network and the prototype classification module, respectively, to obtain the corresponding weak augmentation branch prediction results and strong augmentation branch pre-prediction results. Specifically, the weak augmentation view is used to generate candidate prediction results while preserving as much of the original forgery traces and semantic structure of the samples as possible; the strong augmentation view is used to apply stronger perturbations to the samples to construct consistency training constraints and evaluate the stability of sample predictions.

[0014] Step S4: Calculate the quality score of the pseudo-label;

[0015] For each unlabeled sample, a pseudo-label quality score is calculated based on the prediction results and feature information obtained in step S3. The pseudo-label quality score is constructed based on the following information, specifically:

[0016] (1) Prediction confidence information: used to reflect the strength of the model’s direct judgment on the current predicted category; (2) Enhancement consistency information: used to reflect the consistency of the prediction results of unlabeled samples under weak and strong enhancement views; (3) Category interval information: used to reflect the degree of distinction between the first candidate category and the second candidate category, thereby characterizing whether the sample is close to the decision boundary; (4) Prototype stability information: used to reflect the stability of the change of the current prototype of the sample within the training window, thereby judging whether the category structure corresponding to the sample is reliable.

[0017] Step S5: Perform known partition and unknown partition partitioning, and generate corresponding dynamic thresholds for the partitions;

[0018] Based on the prototype set to which the current prediction results of the unlabeled samples belong, the unlabeled samples are divided into known partition samples and unknown partition samples; the pseudo-label quality score distribution of known partition samples and unknown partition samples are statistically analyzed, and known partition dynamic threshold and unknown partition dynamic threshold are generated.

[0019] Step S6: Perform quality-perceived asymmetric pseudo-label correction;

[0020] Based on the partition to which the unlabeled sample belongs, the corresponding partition dynamic threshold, and the pseudo-label quality score, asymmetric pseudo-label screening and correction are performed on the unlabeled sample; for unlabeled samples that meet the corresponding partition dynamic threshold conditions, pseudo-labels are generated and participate in pseudo-label supervised training.

[0021] Step S7: Perform quality perception phased consistency training;

[0022] Based on training phase information, pseudo-label quality scores of unlabeled samples, and the partition to which the samples belong, the prediction distribution consistency loss of unlabeled samples under weak and strong enhancement branches is dynamically weighted, thereby achieving adaptive adjustment of the participation intensity of unlabeled samples under different training phases.

[0023] Step S8: Perform dynamic prototype update;

[0024] Based on the usage and drift of candidate unknown category prototypes during training, dynamic updates are performed on the candidate unknown category prototypes to reduce the interference of redundant and unstable prototypes on the training process; wherein, the dynamic prototype update is to merge candidate unknown category prototypes with low usage and high drift into the active candidate unknown category prototype with the highest cosine similarity.

[0025] Step S9: Jointly optimize model parameters and output attribution results;

[0026] The model parameters are updated by combining the supervised loss of labeled samples, the pseudo-labeled supervised loss of unlabeled samples, and the consistency loss of unlabeled samples. After training, the model is input with the deep fake sample to be tested, and the corresponding fake source attribution result is output.

[0027] Furthermore, the pseudo-label quality score in step S4 is jointly determined by the following four factors: the prediction confidence corresponding to the weak enhancement branch prediction result; the enhancement consistency between the weak enhancement branch prediction result and the strong enhancement branch prediction result; the class interval between the first candidate class and the second candidate class in the weak enhancement branch prediction result; and the stability of the current prototype to which the sample belongs in adjacent training rounds. Specifically, the pseudo-label quality score... Determine using the following formula: ;

[0028] in, Indicates the first The prediction confidence of an unlabeled sample. Indicates the first Enhanced consistency of unlabeled samples Indicates the first The class interval of each unlabeled sample. Indicates the first Prototype stability of unlabeled samples Let be the weight coefficient, and satisfy... .

[0029] Furthermore, in step S5, the partitioning method for unlabeled samples is as follows: when the current predicted category of an unlabeled sample falls into the known category prototype set, the unlabeled sample is divided into known partition samples; when the current predicted category of an unlabeled sample falls into the candidate unknown category prototype set, the unlabeled sample is divided into unknown partition samples.

[0030] Furthermore, in step S5, the known partition dynamic threshold and the unknown partition dynamic threshold are determined based on the mean and standard deviation of the corresponding partition pseudo-label quality score set, respectively, as follows: ;

[0031] in, Indicates the first Known partition dynamic thresholds during round training. Indicates the first Unknown partition dynamic threshold during round training and These represent the mean quality scores of known partition pseudo-labels and the mean quality scores of unknown partition pseudo-labels, respectively. and These represent the standard deviations of the quality scores for known and unknown partition pseudo-labels, respectively. This is the preset threshold coefficient.

[0032] Furthermore, in step S6, the method for determining whether an unlabeled sample enters the pseudo-label supervised training branch is as follows:

[0033] in, Indicates the first Whether an unlabeled sample enters the pseudo-label supervised training branch. Indicates the first The partition label to which each unlabeled sample belongs. Indicates the first The pseudo-label quality score of an unlabeled sample. Indicates the first Known partition dynamic thresholds in round training, Indicates the first Dynamic threshold for unknown partitions in round training.

[0034] Furthermore, in step S6, for unlabeled samples entering the pseudo-label supervised training branch, the predicted category of the weak enhancement branch is used as the pseudo-label, and the pseudo-label supervised loss is weighted according to the pseudo-label quality score. The pseudo-label supervised loss is:

[0035] in, This represents the total number of unlabeled samples. Indicates the first The pseudo-label of an unlabeled sample. Indicates the first The predicted probability of a pseudo-label class for an unlabeled sample under the strong enhancement branch.

[0036] Furthermore, in step S7, consistency training weights are determined based on the training phase coefficients, the pseudo-label quality score of the unlabeled samples, and the partition to which the unlabeled samples belong. These consistency training weights satisfy the following:

[0037] in, Indicates the first In the first round of training Consistent training weights for unlabeled samples. Represents the coefficients during the training phase. Indicates the first The pseudo-label quality score of an unlabeled sample. Indicates the first The partition label to which each unlabeled sample belongs. This represents the unknown partition enhancement coefficient.

[0038] Furthermore, the consistency loss in step S7 is determined as follows:

[0039] in, Indicates consistency loss. Indicates the first The predicted probability distribution of unlabeled samples under the weak enhancement branch Indicates the first The predicted probability distribution of an unlabeled sample under the strong enhancement branch. This represents the Kullback-Leibler divergence.

[0040] Furthermore, in step S8, dynamic prototype updates are performed based on the usage rate of candidate unknown category prototypes in the current training round and the drift amount between adjacent training rounds. When the usage rate of a candidate unknown category prototype is lower than the average usage rate of all candidate unknown category prototypes and its drift amount is higher than the average drift amount of all candidate unknown category prototypes, the candidate unknown category prototype is determined to be a low-usage, high-drift prototype and merged into the active candidate unknown category prototype with the highest cosine similarity. The update method for candidate unknown category prototypes is as follows:

[0041] in, This represents the prototype of the unknown category to be merged. The active candidate unknown category prototype is represented by the one with the highest cosine similarity to the candidate unknown category prototype to be merged.

[0042] Furthermore, in step S9, the total loss function is constructed as follows:

[0043] in, This indicates supervised loss with labeled samples. This represents the pseudo-label supervision loss for unlabeled samples. This represents the consistency loss of unlabeled samples. and This is the loss weighting coefficient.

[0044] Furthermore, when the deepfake sample is a video sample (i.e., labeled deepfake samples are labeled deepfake video frame samples, and unlabeled deepfake samples are unlabeled deepfake video frame samples), the video sample is first extracted to obtain a video frame sequence, and the above method is performed on each video frame respectively; in the inference stage, the predicted probability distribution corresponding to each video frame is temporally aggregated to obtain the video-level deepfake attribution result.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. More comprehensive and accurate pseudo-label assessment: Through a quality scoring mechanism that integrates multiple indicators, it breaks through the limitations of a single confidence level and achieves a full-dimensional judgment on the reliability of unlabeled samples.

[0047] 2. Significantly improved sample utilization efficiency: The asymmetric dynamic threshold screening with known / unknown partitions is adopted to avoid over-filtering of unknown samples with a uniform threshold, thus solving the problem of imbalance in the screening of the two types of samples.

[0048] 3. Significantly enhanced training stability: Introducing quality scores into the dynamic weighting of the dual training branches suppresses the propagation of false label noise and reduces disturbances in the early stages of training.

[0049] 4. Enhanced adaptability to complex scenarios: Relying on a dynamic prototype update mechanism, it adaptively manages various prototypes, adapting to real-world scenarios with unknown categories and uncertain numbers, resulting in stronger robustness.

[0050] 5. Outstanding value in practical applications: Customized design for deepfake tracing in real-world environments, it can directly support business scenarios such as media authenticity verification, forgery source tracking, and digital evidence collection, and has good engineering deployability.

[0051] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart illustrating the overall process of a real-world deep video forgery tracing method according to the present invention.

[0054] Figure 2 This is a flowchart of the pseudo-label quality scoring process in this invention;

[0055] Figure 3 This is a flowchart of the dynamic prototype update process in this invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent transformations or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the protection scope of the present invention.

[0057] This invention provides a method for tracing the source of deep video forgeries in real-world environments, based on asymmetric pseudo-label correction and dynamic prototype updating. This method integrates four types of information: prediction confidence, consistency between strong and weak augmented dual-view predictions, feature interval of the top two predicted prototype categories, and historical update stability of the corresponding category's prototypes. A unified pseudo-label quality score is generated through weighted fusion to comprehensively characterize the usability of unlabeled forged samples. Then, based on the prototype set to which the current prediction result of the unlabeled sample belongs, it is automatically divided into known forged category partitions and unknown forged category partitions. The pseudo-label quality score distributions of the two categories are statistically analyzed to generate independent dynamic filtering thresholds adapted to their respective distribution characteristics, avoiding unreasonable filtering of unknown category samples by a globally uniform threshold. Simultaneously, a quality-aware consistency training branch is constructed by combining the model convergence stage information corresponding to the training epochs. In the early stages of training, only high-scoring samples are allowed to participate in supervised learning to reduce pseudo-label noise interference when the model is unstable. In the later stages of training, the quality-aware consistency training branch is gradually increased. This method expands the participation scope of medium-quality samples and unknown category partition samples to balance training stability and the adaptability of unknown categories. Finally, it adopts a strategy of "pre-setting a redundant prototype pool and adaptive iteration during training" for candidate unknown category prototypes. The usage rate and feature stability of each prototype are statistically analyzed in real time. Low-usage redundant prototypes are merged and recycled, unstable prototypes with drifting features are reset, and potential low-sample prototypes with sufficient sample support are retained and strengthened. This enables the prototype structure to be dynamically adjusted according to the distribution of forgery categories. This method breaks through the limitations of a single confidence measure, improves the accuracy of quality judgment of unlabeled forgery samples and the sample utilization rate of unknown forgery categories. It can adaptively generate new deep forgery sources without fixing the total number of prototypes in advance. It can be directly deployed on content review platforms to support various practical businesses such as media authenticity verification, forgery source tracking, and digital forensics.

[0058] Example 1

[0059] like Figures 1 to 3 This embodiment provides a real-world deep video forgery tracing method based on asymmetric pseudo-label correction and dynamic prototype update. This method is designed for application scenarios where only known forgery category labels are known during the training phase, but unknown forgery categories exist during the testing phase. By introducing pseudo-label quality scoring, partitioned asymmetric screening, consistency training, and dynamic prototype update mechanisms, robust utilization of unlabeled samples is achieved. The method specifically includes the following steps:

[0060] First, a labeled sample set is constructed. and unlabeled sample set The labeled sample set contains only samples of known forgery categories and their corresponding labels, while the unlabeled sample set contains both samples of known forgery categories and samples of unknown forgery categories. Specifically, labeled deepfake samples are labeled deepfake video frame samples, and unlabeled deepfake samples are unlabeled deepfake video frame samples; let the labeled sample set be represented as: , ;

[0061] The unlabeled sample set is represented as: ;

[0062] in, This indicates a fake image. express The corresponding forgery method (i.e., the corresponding label). Given the number of known forgery categories, This represents the number of labeled samples. This represents the number of unlabeled samples.

[0063] Subsequently, face region cropping, size normalization, and pixel standardization are performed sequentially on all input video frames to obtain model input in a unified format.

[0064] Next, a feature extraction network is constructed. And a prototype classification network. For any input video frame The deep feature representation is obtained using a feature extraction network: .in The feature dimension is denoted as . The prototype classification model consists of a set of known category prototypes and a set of candidate unknown category prototypes. Let the total number of prototypes be: Among them, the former Each prototype corresponds to a known category, and then... Each prototype corresponds to a candidate unknown category.

[0065] For sample features Calculate its similarity to each prototype: And the predicted probability is obtained through softmax: ,in This is the temperature coefficient. The sample predicted category is defined as: For unlabeled samples Construct enhanced views respectively and weak enhancement view The weakly augmented view is used to generate candidate pseudo-labels, while the strongly augmented view is used to construct the consistency training branch. Inputting the two types of views into the feature extraction network and the prototype classification model respectively yields: .

[0066] We define the weak enhancement branch prediction category as: To improve the accuracy of pseudo-label selection for unlabeled samples, this embodiment constructs a pseudo-label quality score for each unlabeled sample. The quality score is determined by prediction confidence, enhanced consistency, class margin, and prototype stability. Prediction confidence is defined as follows: The enhanced consistency is obtained through the Jensen-Shannon divergence between the prediction distributions of the weak and strong enhancement branches: Let... ,but Define the enhanced consistency item as The category margin is defined as the difference between the highest and second-highest probability values ​​in the weakly enhanced branch: Prototype stability is determined by the drift of the sample's current prototype across adjacent training rounds. Let the sample's current prototype be... Then its drift is: Prototype stability is defined as: The final sample quality score is defined as follows: ;in, Let be the weight coefficient, and satisfy... .

[0067] After obtaining the quality score, the unlabeled samples are divided into known partition samples and unknown partition samples based on the prototype set to which the currently predicted category belongs. The partition labeling function is defined as follows: ;in, This indicates that the sample belongs to a known partition. This indicates that the sample belongs to an unknown partition. The quality score sets for known and unknown partitions are calculated separately for the current round: Calculate its mean and standard deviation: ; And generate known partition dynamic thresholds and unknown partition dynamic thresholds respectively: and ;in, Indicates the first Known partition dynamic thresholds during round training. Indicates the first Unknown partition dynamic threshold during round training and These represent the mean quality scores of known partition pseudo-labels and the mean quality scores of unknown partition pseudo-labels, respectively. and These represent the standard deviations of the quality scores for known and unknown partition pseudo-labels, respectively. This is the preset threshold coefficient.

[0068] Next, quality-aware asymmetric pseudo-label correction is performed based on the partition and threshold. The selection variables for whether a sample enters the pseudo-label supervision branch are defined as follows:

[0069] To satisfy The samples will be used to predict the category of the weakly enhanced branch. As its pseudo-label: .in, Indicates the first Whether an unlabeled sample enters the pseudo-label supervised training branch. Indicates the first The partition label to which each unlabeled sample belongs. Indicates the first The pseudo-label quality score of an unlabeled sample. Indicates the first Known partition dynamic thresholds in round training, Indicates the first Dynamic threshold for unknown partitions in round training.

[0070] For unlabeled samples entering the pseudo-label supervised training branch, the predicted class of the weakly enhanced branch is used as the pseudo-label, and the pseudo-label supervised loss is weighted according to the pseudo-label quality score; the pseudo-label supervised loss is constructed based on the above pseudo-labels:

[0071] in, This represents the total number of unlabeled samples. Indicates the first The pseudo-label of an unlabeled sample. Indicates the first The predicted probability of a pseudo-label class for an unlabeled sample under the strong enhancement branch.

[0072] Meanwhile, to improve the model's robustness to perturbations, this embodiment also performs quality-aware phased consistency training on unlabeled samples. Let the total number of training rounds be T, and the current round number be t, then the phase coefficient is defined as... The consistency training weights are defined as follows: ;in, Indicates the first In the first round of training Consistent training weights for unlabeled samples. Represents the coefficients during the training phase. Indicates the first The pseudo-label quality score of an unlabeled sample. Indicates the first The partition label to which each unlabeled sample belongs. Enhancement coefficients for unknown partitions. Construct a consistency loss based on sample-level weights:

[0073] in, Indicates consistency loss. Indicates the first The predicted probability distribution of unlabeled samples under the weak enhancement branch Indicates the first The predicted probability distribution of an unlabeled sample under the strong enhancement branch. This represents the Kullback-Leibler divergence.

[0074] To adapt to real-world scenarios where the number of unknown categories cannot be predetermined, this embodiment dynamically updates the candidate unknown prototypes. Specifically, the first... The usage rate of the unknown prototypes in the current round is: The drift amount between adjacent rounds is defined as: Calculate the average utilization and average drift of all unknown prototypes: and When an unknown prototype satisfies: When this occurs, it is identified as a low-use, high-drift prototype and merged into the active unknown prototype with the highest cosine similarity. Let the target prototype index be: Then the target prototype is updated as follows: Finally, the total loss function is constructed:

[0075] in, This indicates supervised loss with labeled samples. This represents the pseudo-label supervision loss for unlabeled samples. This represents the consistency loss of unlabeled samples. and These are the loss weight coefficients. The labeled supervised loss is defined as: .

[0076] Finally, the parameters of the feature extraction network and the prototype parameters are jointly optimized using the total loss function until training is complete. After training, the depth-spoofed video frames to be tested are input into the model, and the attribution results are output: ,when Output the corresponding known forgery category; when Output the cluster number of the corresponding unknown forgery category.

[0077] Example 2

[0078] This embodiment retains the quality-aware asymmetric pseudo-label correction mechanism and the quality-aware phased consistency training mechanism, but does not perform the dynamic prototype update step.

[0079] Specifically, in this embodiment, a label sample set is also constructed first. and unlabeled sample set The weak and strong augmentation branch prediction results for unlabeled samples are obtained through a feature extraction network and a prototype classification model. Subsequently, the pseudo-label quality score is calculated in the same manner as in Example 1. The predicted categories are divided into known and unknown partitions based on their prototype range.

[0080] Then, calculate the dynamic threshold for each known partition. and unknown partition dynamic threshold Based on the quality score and partition threshold, it determines whether unlabeled samples enter the pseudo-label supervision branch, thereby constructing the pseudo-label supervision loss. .

[0081] Meanwhile, this embodiment also uses stage coefficients. and consistent training weights Perform consistency training on unlabeled samples and construct a consistency loss. .

[0082] Unlike Example 1, the prototype set in this example maintains a constant total size during training, and prototype merging is not performed based on usage rate and drift. The total loss function during training is still defined as:

[0083] This embodiment is suitable for real-world deepfake attribution scenarios where the number of unknown categories does not change significantly. Because it does not perform dynamic prototype updates, this embodiment has a simpler structure, a more stable training process, and is easier to deploy in resource-constrained environments.

[0084] Example 3

[0085] This embodiment extends the method of Embodiment 1 to video-level deepfake tracing scenarios.

[0086] In this embodiment, the input is no longer a single video frame, but the video to be tested. First, video frames are extracted from the input video at fixed time intervals to form a frame sequence:

[0087] in, This represents the number of video frames extracted.

[0088] For each video frame, the same preprocessing, feature extraction, dual-view prediction, pseudo-label quality score calculation, known / unknown partition determination, asymmetric pseudo-label correction, consistency training, and dynamic prototype update processes as in Example 1 are performed to obtain the attribution result for each frame. and the corresponding prediction probability vector .

[0089] During the inference phase, the prediction results for all frames in the video are time-series aggregated. This embodiment uses the average probability aggregation method, defining the video-level probability as: .

[0090] The final video-level attribution result is defined as follows: ;when When the output video belongs to the corresponding known forgery category; when At that time, the output video belongs to the corresponding unknown forgery category cluster.

[0091] This embodiment extends the video frame-level real-world deepfake source tracing method to video frame-level processing and performs temporal aggregation, improving the model's attribution stability in video scenarios. It is suitable for application scenarios such as short video forgery source identification, forgery video detection and source tracing.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for tracing the source of forged deep video in a real environment, characterized in that, This method, based on asymmetric pseudo-label correction and dynamic prototype update, includes the following steps: First, the video samples are frame-extracted to obtain a video frame sequence; among them, labeled depth-spoofed samples are labeled depth-spoofed video frame samples, and unlabeled depth-spoofed samples are unlabeled depth-spoofed video frame samples; and then the following steps S1 to S9 are performed on each video frame respectively: S1. Obtain and preprocess labeled and unlabeled deepfake sample sets; S2. Construct a feature extraction network and a prototype classification model to predict the category based on feature similarity. S3. Generate weak and strong augmented views for unlabeled samples, and input them into the feature extraction network and the prototype classification model respectively to obtain the prediction results of the corresponding two branches. S4. Based on the prediction results obtained in S3 and the prototype state corresponding to the sample, calculate the pseudo-label quality score of the unlabeled sample; wherein, the pseudo-label quality score is determined by the following formula: ； in, Indicates the first The prediction confidence of an unlabeled sample. Indicates the first Enhanced consistency of unlabeled samples Indicates the first The class interval of each unlabeled sample. Indicates the first Prototype stability of unlabeled samples Let be the weight coefficient, and satisfy... ; S5. Based on the prototype set to which the current predicted category of the unlabeled sample belongs, divide the unlabeled sample into known partition samples and unknown partition samples, and statistically analyze the pseudo-label quality score distribution of the two types of partition samples respectively, and generate known partition dynamic threshold and unknown partition dynamic threshold. S6. Using the pseudo-label quality score obtained in S4 and the partition to which the unlabeled sample belongs and the corresponding partition dynamic threshold obtained in S5, construct the pseudo-label supervision branch; assign pseudo-labels to unlabeled samples that meet the corresponding partition dynamic threshold conditions, and weight the pseudo-label supervision loss according to the pseudo-label quality score. S7. Using the pseudo-label quality score obtained in S4 and the sample partitioning results obtained in S5, a consistency training branch is constructed by combining the training phase information. The consistency loss of the predicted distribution of unlabeled samples under the weak enhancement branch and the strong enhancement branch is dynamically weighted. S8. Perform dynamic prototype updates based on the usage rate and drift of candidate unknown category prototypes during training. Specifically, perform dynamic prototype updates based on the usage rate of candidate unknown category prototypes in the current training round and the drift between adjacent training rounds. When the usage rate of a candidate unknown category prototype is lower than the average usage rate of all candidate unknown category prototypes and its drift is higher than the average drift of all candidate unknown category prototypes, the candidate unknown category prototype is identified as a low-usage, high-drift prototype and merged into the active candidate unknown category prototype with the highest cosine similarity. The update method for candidate unknown category prototypes is as follows: in, This represents the prototype of the unknown category to be merged. The active candidate unknown category prototype that has the highest cosine similarity to the candidate unknown category prototype to be merged; S9. The model parameters are updated by combining the supervised loss of labeled samples, the pseudo-labeled supervision loss of unlabeled samples, and the consistency loss of unlabeled samples. During the inference phase, the predicted probability distributions corresponding to each video frame are aggregated in time to obtain the video-level deep forgery attribution results.

2. The method according to claim 1, characterized in that, In step S5, the partitioning method for unlabeled samples is as follows: when the current predicted category of an unlabeled sample falls into the known category prototype set, the unlabeled sample is divided into known partition samples. When the current predicted category of an unlabeled sample falls into the candidate unknown category prototype set, the unlabeled sample is classified as an unknown partition sample.

3. The method according to claim 1, characterized in that, In step S5, the known partition dynamic threshold and the unknown partition dynamic threshold are determined based on the mean and standard deviation of the corresponding partition pseudo-label quality score set, respectively, as follows: ； in, Indicates the first Known partition dynamic thresholds during round training. Indicates the first Unknown partition dynamic threshold during round training and These represent the mean quality scores of known partition pseudo-labels and the mean quality scores of unknown partition pseudo-labels, respectively. and These represent the standard deviations of the quality scores for known and unknown partition pseudo-labels, respectively. This is the preset threshold coefficient.

4. The method according to claim 3, characterized in that, In step S6, the method for determining whether an unlabeled sample enters the pseudo-label supervised training branch is as follows: in, Indicates the first Whether an unlabeled sample enters the pseudo-label supervised training branch. Indicates the first The partition label to which each unlabeled sample belongs. Indicates the first The pseudo-label quality score of an unlabeled sample.

5. The method according to claim 4, characterized in that, In step S6, for unlabeled samples entering the pseudo-label supervised training branch, the predicted class of the weak enhancement branch is used as the pseudo-label, and the pseudo-label supervised loss is weighted according to the pseudo-label quality score. The pseudo-label supervised loss is: in, This represents the total number of unlabeled samples. Indicates the first Pseudo-labels for unlabeled samples, Indicates the first The predicted probability of a pseudo-label class for an unlabeled sample under the strong enhancement branch.

6. The method according to claim 1, characterized in that, In step S7, consistency training weights are determined based on the training phase coefficients, the pseudo-label quality scores of unlabeled samples, and the partition to which the unlabeled samples belong. These consistency training weights satisfy the following: in, Indicates the first In the first round of training Consistent training weights for unlabeled samples. Represents the coefficients during the training phase. Indicates the first Pseudo-label quality score of an unlabeled sample. Indicates the first The partition label to which each unlabeled sample belongs. This represents the unknown partition enhancement coefficient.

7. The method according to claim 6, characterized in that, The consistency loss in step S7 is determined as follows: in, This represents the total number of unlabeled samples. Indicates consistency loss. Indicates the first The predicted probability distribution of unlabeled samples under the weak enhancement branch Indicates the first The predicted probability distribution of an unlabeled sample under the strong enhancement branch. This represents the Kullback-Leibler divergence.

8. The method according to claim 1, characterized in that, In step S9, construct the total loss function. for: in, This indicates supervised loss with labeled samples. This represents the pseudo-label supervision loss for unlabeled samples. This represents the consistency loss of unlabeled samples. and This is the loss weighting coefficient.

Images