Self-adaptive detection system and method during false news video testing

Abstract

The invention relates to a self-adaptive detection system and method during false news video testing, and relates to the technical field of false news video detection.The system comprises an entropy selection retrieval module, a stable anchor point guide alignment module and a target domain perception self-training module.The method introduces an innovative design of entropy selection retrieval and stable anchor point guide alignment; the accurate self-adaption of the unstable video instance in the target domain is realized; according to the method, the adaptive detection of an unstable video is enhanced by using a stable (low entropy) video instance in a target domain, and the dynamic adaptation problem of false news video detection under emerging events and topics is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of fake news video detection technology, specifically to an adaptive detection system and method for testing fake news videos. Background Technology

[0002] With the rise of video-sharing platforms such as YouTube and TikTok, video has become one of the main channels for obtaining news information. However, the nature of these open platforms also facilitates the spread of false information by malicious actors. They use videos to spread misleading rumors or false content, posing a serious threat to viewer safety and eroding public trust. Therefore, developing effective fake news video detection technology is of significant practical importance.

[0003] Early methods for detecting fake news videos primarily relied on single-modal feature extraction and classification, such as text analysis based on keyword matching or image classification based on visual features. These methods performed reasonably well with simple content, but struggled with complex multimodal content because they failed to effectively utilize complementary information between different modalities.

[0004] With the development of deep learning technology, researchers have begun to explore applying multimodal fusion techniques to video detection tasks. Existing fake news video detection methods typically model multimodal information such as visual, textual, and audio elements in videos to reveal deceptive cues. For example, the SV-FEND method captures cross-modal relevance to select informational features and leverages comments for detection. However, these methods often implicitly assume that the distribution of news topics is consistent during the training and testing phases. When encountering out-of-distribution samples related to emerging events, these models trained on stale data often fail.

[0005] Fake news videos evolve rapidly and are often associated with breaking news events, leading to significant topic-level distribution shifts between historical training videos and new instances. For example, fake news videos about the COVID-19 pandemic typically contain medical misinformation and false health advice, while fake news videos about the Russia-Ukraine conflict spread false battlefield reports and geopolitical narratives—significant differences in both form and content. This topic-level distribution shift results in severe discrepancies in feature representations, making it difficult for existing methods to effectively address them.

[0006] To address the data distribution offset problem, a simple solution is to label and fine-tune these new videos. However, this contradicts the goal of detecting fake news at the earliest stage, rather than relying on post-hoc analysis after harm has occurred. Unsupervised domain adaptation (UDA) methods appear promising because they utilize training data (source domain) and emerging unlabeled instances (target domain) to learn domain-invariant representations for generalization. However, UDA requires access to the source data for adaptation, which is often impractical in fake news video detection due to significant storage overhead and privacy concerns.

[0007] Test-Time Adaptation (TTA) offers a novel approach to addressing this problem. Existing TTA methods primarily rely on Entropy Minimization (EM) and its variants as self-supervised strategies, linking high-confidence (low-entropy) predictions with higher prediction accuracy, and achieving adaptation by iteratively reducing the entropy of unseen test data. Despite the success of these methods, applying them to test-time fake news video detection presents two fundamental challenges:

[0008] First, there are significant differences in feature representations between the source and target domains. Most TTA methods assume that slight distributional shifts in the source data (e.g., synthesis defects such as blurring or noise) can be handled by implicit alignment representations. However, in fake news video detection, topic-level shifts lead to severe representation gaps, rendering indirect alignment methods insufficient.

[0009] Secondly, the category distribution is imbalanced and fluctuating. The varying public attention to different news events leads fake news creators to produce a large number of fake videos on trending topics while neglecting others, resulting in a highly skewed and variable ratio of real to fake news videos across different events. In fake news video detection datasets such as FakeSV, FakeTT, and FVC, 85.5%, 79.0%, and 98.4% of news events, respectively, exhibit extreme imbalance, with highly skewed category distributions (majority:minority > 4:1) for news videos associated with such events. However, most TTA methods only work well when the prior category distribution (i.e., the relative proportions of different categories) in the target domain is stable and matches the source domain. Therefore, the imbalanced and constantly changing label distribution in each newly arrived cluster of news videos associated with different events poses a significant challenge to existing TTA methods.

[0010] Furthermore, existing technologies suffer from the following problems: First, they cannot handle large distribution shifts; second, they are highly dependent on the stability of prior category distributions; and third, they lack effective cross-event knowledge transfer mechanisms. These problems severely limit the effectiveness of existing methods in practical applications, especially when faced with rapidly evolving fake news videos.

[0011] In summary, current fake news video detection technologies still lack effective solutions to the adaptation problem during testing, particularly in handling significant differences in feature representations and dynamically changing category distributions. This provides ample room for research and application prospects for new technical solutions. Summary of the Invention

[0012] The purpose of this invention is to address the problems of insufficient test-time adaptive capability, inadequate utilization of cross-domain knowledge, and poor model generalization in existing fake news video detection methods. This invention proposes a test-time adaptive detection system for fake news videos. By introducing an entropy selection retrieval mechanism, a stable anchor-guided alignment module, and a target domain-aware self-training paradigm, this invention can adapt to emerging events and topics in real time without accessing source domain data or target domain labels, thereby improving the accuracy and robustness of detection.

[0013] Another objective of this invention is to provide a test-time adaptive detection method for fake news videos. This method, within a test-time adaptive framework, introduces innovative designs such as entropy-selective retrieval and stable anchor-guided alignment, achieving accurate adaptation to unstable video instances in the target domain and avoiding the failure problem of traditional methods when facing topic-level distribution shifts.

[0014] The technical solution of the present invention is as follows:

[0015] An adaptive detection method for testing fake news videos includes the following steps;

[0016] S1: Initialize the pre-trained multimodal source model and a memory bank for storing the most recent samples;

[0017] S2: In data stream scenarios, newly arriving news video batches are continuously written to the memory bank in a FIFO manner to retain the latest cross-event context information;

[0018] S3: Based on the semantic similarity of the three modalities of vision, text and audio, retrieve semantically relevant reference instances from the memory for the currently queried video;

[0019] S4: Introduce prediction entropy as a stability criterion to filter out unreliable reference instances and retain only low-entropy, stable and semantically relevant reference instances.

[0020] S5: Explicitly model the sequence features of the query video in each modality as a probability distribution;

[0021] S6: Calculate the modal feature distribution of the reference instance obtained in step S4, and aggregate the modal feature distribution of the reference instance to form a "stable anchor point distribution" representing the semantic structure of the source domain;

[0022] S7: By minimizing the difference between the probability distribution of the query video and the stable anchor distribution, explicit cross-modal distribution alignment guided by stable anchors is achieved, pulling the target domain representation back into the source domain semantic space;

[0023] S8: Use reference examples to perform weighted fusion of multimodal source model predictions to generate high-quality pseudo-labels with target domain awareness;

[0024] S9: Combine distribution alignment loss, pseudo-label self-training loss, and entropy minimization objective to perform online optimization of multimodal source models.

[0025] Furthermore, step S3 specifically includes the following:

[0026] Retrieve the video from the memory for the currently queried video. A semantically relevant reference example:

[0027] ,

[0028] in, This indicates that cosine similarity is used to measure semantic relevance within a modality, and Top-K() means to return the top K reference instances ranked by the median of the (). , , They represent the i-th reference instance in the memory bank, respectively. Visual, textual, and audio modal information; taking the visual modality as an example:

[0029] ,

[0030] in, It is the basic visual model used for multimodal retrieval, where T represents the matrix transpose operation.

[0031] Furthermore, step S4 specifically includes the following:

[0032] Semantically relevant reference examples Stable reference instances are further filtered out through an entropy selection process:

[0033] ,

[0034] in This represents a set of low-entropy, stable, and semantically relevant reference instances for the queried video. It is a predefined entropy threshold. Indicates sample Predicted entropy:

[0035] ,

[0036] here Representation source model The resulting probability distribution Indicates the number of categories. yes For the first The probability of a class being predicted.

[0037] Furthermore, step S5 specifically includes the following:

[0038] For searching videos Its modal features are modeled as a multivariate Gaussian distribution:

[0039] ,

[0040] in, Indicates the modal type, and These are the core statistical parameters describing its location and diffusion. Represents a distribution, This refers to the representation of modalities:

[0041] ,

[0042] here It is modal The representation of which It is the sequence length. It is the feature dimension, and towards Add a small identity matrix .

[0043] Furthermore, step S6 specifically includes the following:

[0044] Calculation and Query Video Related reference example collection The modal feature distributions are obtained and aggregated to obtain a stable anchor point distribution:

[0045] ,

[0046] The distribution statistics parameter is defined as follows:

[0047] ,

[0048] in, and This is a reference example. Parameters, stable anchor point distribution Constructed from low-entropy and semantically similar reference instances, it is stable and closely approximates the source domain distribution, while also being consistent with the distribution of the query videos. They share similar semantics.

[0049] Furthermore, step S7 specifically includes the following:

[0050] Based on a stable anchor point distribution, the query video is explicitly linked by minimizing the KL divergence. The characteristic distribution is aligned with its stable anchor point distribution, and its representation is pulled toward the source domain:

[0051] ,

[0052] two Gaussian distribution and The closed-form form of the KL divergence between them is:

[0053] ,

[0054] in, Let p and q be the covariance matrices. , The mean vectors of distributions p and q, express The inverse matrix.

[0055] Furthermore, step S8 specifically includes the following:

[0056] Based on low-entropy, stable, and semantically relevant reference instances, for each query video Assigning pseudo-labels :

[0057] ,

[0058] in and These are multimodal source models For video search and its reference examples In category The predicted probability, and It is a weighting factor. It is a video search. and its reference examples The sum of similarity scores across all modalities, the target domain-aware pseudo-label. This is derived through pseudo-label enhancement using low-entropy, stable, and semantically relevant reference instances.

[0059] Furthermore, step S9 specifically includes the following:

[0060] Define the self-training objective function as: ;

[0061] During testing, the source model was optimized using the following comprehensive loss function. : ;

[0062] in, It is the entropy minimization objective widely used in TTA. and It is the balance ratio.

[0063] A system for an adaptive detection method during testing of fake news videos, characterized in that it includes:

[0064] The entropy selection retrieval module is used to provide stable and semantically relevant reference instances for each query video;

[0065] The Stable Anchor Point Guided Alignment Module is used to explicitly reduce the substantial representation gap between the source and target domains caused by topic-level distribution offsets.

[0066] The target domain-aware self-training module assigns target domain-aware pseudo-labels to queries under the guidance of reference examples and guides the source model to adapt to new category distributions online.

[0067] Furthermore, the entropy selection retrieval module includes two cooperating sub-modules: a memory bank management sub-module and a semantic retrieval sub-module.

[0068] The memory management submodule uses a first-in-first-out strategy to maintain a dynamically updated memory, storing the most recently arrived news video instances and their multimodal features;

[0069] The semantic retrieval submodule retrieves the Top-K most relevant reference instances based on the cross-modal semantic similarity between the query video and the samples in the memory bank. Then, it selects the reference instances whose predicted entropy is lower than the dynamic threshold through an entropy filtering mechanism, thus providing high-quality reference data for the target video.

[0070] The stable anchor point guided alignment module includes a distributed modeling submodule, an anchor point construction submodule, and an explicit alignment submodule;

[0071] The distribution modeling submodule uses the modal encoder of the source model to encode the features of the query video and the retrieved reference instance, and models the feature representation of each modality as a multivariate Gaussian distribution, and describes the location and spread of the features by calculating the mean vector and covariance matrix;

[0072] The anchor point construction submodule constructs a stable anchor point distribution by aggregating the modal feature distributions of multiple reference instances. This anchor point distribution is stable and close to the source domain distribution, while sharing similar semantics with the queried video.

[0073] The explicit alignment submodule achieves distribution-level explicit alignment by minimizing the KL divergence between the query video feature distribution and the stable anchor point distribution, thereby pulling the representation of the query video toward the source domain.

[0074] Compared with the prior art, the beneficial effects of the present invention are:

[0075] 1. By introducing an entropy selection retrieval mechanism, a stable anchor point guided alignment module, and a target domain-aware self-training paradigm, this invention can adapt to emerging events and topics in real time without accessing source domain data or target domain labels, thereby improving the accuracy and robustness of detection.

[0076] 2. An innovative design of entropy selection retrieval and stable anchor point guided alignment is introduced, which realizes accurate adaptation to unstable video instances in the target domain and avoids the failure problem of traditional methods when facing topic-level distribution shifts.

[0077] 3. By integrating a retrieval-guided adaptive paradigm into the fake news video detection task, stable (source domain-proximate) videos in the target domain are used to guide semantically relevant but unstable instances for robust adaptation. Furthermore, an entropy-based selection retrieval mechanism provides stable (low-entropy) and semantically relevant reference instances for each query video to support robust adaptation. Through this mechanism, the detection model can utilize stable target domain instances as source domain proxies, achieving effective test-time adaptation even when source data is inaccessible. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of the overall framework of an adaptive detection method for fake news videos based on retrieval enhancement during testing. Detailed Implementation

[0079] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] Terminology Explanation

[0081] Test-Time Adaptation (TTA) is a technique that enables online adaptation of a pre-trained model during the testing phase. It aims to allow a model pre-trained in the source domain to adapt to a target domain with a shifted distribution, assuming unavailable access to the target data's labels and original source data. Unlike traditional offline training, TTA emphasizes real-time adaptation at the testing moment, which is crucial in real-world applications where the model faces changes in data distribution after deployment. In fake news video detection, TTA allows the detection model to adapt promptly to new videos related to emerging events and topics without relabeling data or accessing historical training data, meeting the practical need for detecting fake news in the earliest stages.

[0082] Entropy Minimization (EM) is a fundamental concept in information theory applied to machine learning. Its core idea is to improve prediction confidence by minimizing the entropy of the model's predictions. Prediction entropy measures the uncertainty of the model's output; a lower entropy value indicates greater confidence in the prediction. In test-time adaptation, entropy minimization is widely used as a self-supervised strategy. By iteratively reducing the prediction entropy on unseen test data, it encourages the model to produce higher-confidence predictions, thus implicitly adapting the model to the target domain. However, entropy minimization is primarily suitable for scenarios with slight distribution shifts. When facing topic-level distribution shifts in fake news video detection, this implicit alignment method is often insufficient.

[0083] Pseudo-labels are an important concept in self-supervised and semi-supervised learning, referring to labels predicted and generated by the model itself to guide further training. In test-time adaptation, since the target domain data lacks true labels, pseudo-labels provide training signals for the model. The quality of pseudo-labels directly affects the self-training effect; therefore, effective strategies are needed to improve their accuracy and informativeness. The target domain-aware pseudo-labels proposed in this invention are generated under the guidance of stable reference instances, combining predictions from the query video itself and predictions from semantically related stable reference videos. This improves the reliability of the pseudo-labels and enables them to accurately reflect the category distribution of the current news event cluster.

[0084] Example 1

[0085] This invention provides an adaptive detection system for testing fake news videos, comprising:

[0086] The entropy selection retrieval module provides stable and semantically relevant reference instances for each query video. This module aims to maintain a dynamically updated reference memory and filter the query video based on semantic relevance and entropy value to retrieve the most relevant low-entropy reference instances. This entropy selection retrieval ensures that the retrieved reference videos are both highly semantically relevant to the target video and possess stable predictive properties, thus providing reliable guidance for subsequent adaptive processing.

[0087] A stable anchor-guided alignment module is used to explicitly reduce substantial representational discrepancies between the source and target domains caused by topic-level distribution offsets. This module aligns unstable news video representations with the anchor distributions of their stable reference instances by modeling modal features as multivariate Gaussian distributions and utilizing a KL divergence-driven distribution-level alignment mechanism. This explicit alignment method handles severe cross-domain representational differences more effectively than traditional implicit alignment methods.

[0088] The target domain-aware self-training module enables the source model to quickly adapt to constantly changing target category distributions. This module assigns target domain-aware pseudo-labels to query videos, guided by stable reference instances. These pseudo-labels accurately reflect the label distribution of the news video (query video) clusters associated with the corresponding breaking news event, and guide the source model to adapt to new category distributions online.

[0089] The aforementioned adaptive detection system for fake news videos based on retrieval enhancement includes an entropy selection retrieval module comprising two cooperating sub-modules: a memory bank management sub-module and a semantic retrieval sub-module.

[0090] The memory management submodule adopts a first-in-first-out (FIFO) strategy to maintain a dynamically updated memory, storing the most recently arrived news video instances and their multimodal features, ensuring efficiency while taking into account that the reference value of early news videos for subsequent videos is limited.

[0091] The semantic retrieval submodule retrieves the Top-K most relevant candidate instances based on the cross-modal semantic similarity between the query video and the samples in the memory bank. Then, it selects stable reference instances whose predicted entropy is lower than the dynamic threshold through an entropy filtering mechanism, providing high-quality reference data for the target video.

[0092] The aforementioned adaptive detection system for fake news videos based on retrieval enhancement includes a stable anchor-guided alignment module comprising a distributed modeling submodule, an anchor-construction submodule, and an explicit alignment submodule.

[0093] The distribution modeling submodule uses the modal encoder of the source model to encode features of the video and the retrieved reference instances, and models the feature representation of each modality as a multivariate Gaussian distribution, describing the location and spread of features by calculating the mean vector and covariance matrix;

[0094] The anchor point construction submodule constructs a stable anchor point distribution by aggregating the modal feature distributions of multiple stable reference instances. This anchor point distribution is stable and close to the source domain distribution, while also sharing similar semantics with the queried video.

[0095] The explicit alignment submodule achieves distribution-level explicit alignment by minimizing the KL divergence between the query video feature distribution and the stable anchor point distribution, thereby pulling the representation of the query video toward the source domain.

[0096] Example 2

[0097] This invention proposes an adaptive detection method for testing fake news videos, which specifically includes the following steps:

[0098] S1: Initialize the source model and memory. For the pre-trained source model... ,in , and These represent visual, text, and audio encoders, respectively. Represents the modality fusion and prediction network. Initialize the memory. Used to store the most recent An example of a news video.

[0099] S2: Memory update for each arriving batch of news videos. Each video (see example) Includes visual modalities (video frame), text modality (Video title and on-screen text) and audio modal (Audio transcription), N b Representing the number of videos in a news video batch, newly arriving videos are first added to the memory bank using a FIFO (First-In, First-Out) method. .

[0100] S3: Retrieval based on cross-modal semantic similarity for query videos. Retrieve its from memory bank A semantically relevant video reference example:

[0101] ,

[0102] in, Cosine similarity is used to measure semantic relevance within a modality. Top-K() means to return the K samples ranked by the median of the (). , , They represent the i-th candidate reference instance in the memory bank, respectively. Visual, textual, and audio modal information; taking the visual modality as an example:

[0103] ,

[0104] here It is an off-the-shelf visual foundation model (such as ViT) used for multimodal retrieval, where T represents the matrix transpose operation.

[0105] S4: Entropy-based selection filtering of unstable reference videos. Semantically relevant reference instances. Stable reference instances are further filtered out through an entropy selection process:

[0106] ,

[0107] in Indicates querying videos A set of low-entropy, stable, and semantically relevant reference instances. It is a predefined entropy threshold. Indicates sample Predicted entropy:

[0108] ,

[0109] here Representation source model The resulting probability distribution Indicates the number of categories (in fake news video detection) ), yes For the first The probability of a class being predicted.

[0110] S5: Perform distribution modeling for query videos and reference examples, for query videos Its modal features are modeled as a multivariate Gaussian distribution:

[0111] ,

[0112] in Indicates the modal type, and These are the core statistical parameters describing its location (mean vector) and diffusion (covariance matrix). Represents a distribution, This refers to the representation of modalities:

[0113] ,

[0114] here It is modal The representation of which It is the sequence length (tokens / frames). It is the feature dimension. For stability, a small identity matrix is ​​added to the estimated covariance matrix. The identity matrix is ​​an initialization matrix with all 1s on the main diagonal and 0s in the remaining positions. It can be added to the covariance matrix using matrix addition.

[0115] S6: Construct a stable anchor point distribution. Calculate and query videos. Related reference example collection The modal feature distributions are obtained and aggregated to obtain the anchor point distribution:

[0116] ,

[0117] The distribution statistics parameter is defined as follows:

[0118] ,

[0119] here and This is a reference example. The parameters. Stable anchor point distribution. Constructed from low-entropy and semantically similar reference instances, it is stable and closely approximates the source domain distribution, while also being consistent with the distribution of the query videos. They share similar semantics.

[0120] S7: Perform explicit alignment guided by stable anchor points. Based on the stable anchor point distribution, explicitly align the news video by minimizing the KL divergence. Aligning the characteristic distribution with its stable reference distribution, we pull its representation toward the source domain:

[0121] ,

[0122] two Gaussian distribution and The closed-form form of the KL divergence between them is: ;

[0123] in, Let p and q be the covariance matrices. , The mean vectors of distributions p and q, express The inverse matrix.

[0124] S8: Generate target-domain-aware pseudo-labels. Guided by low-entropy, stable, and semantically relevant reference instances, generate pseudo-labels for each news video. Assigning pseudo-labels :

[0125] ,

[0126] in and These are the source models News video and its reference examples In category The predicted probability, and It is a weighting factor. It's a news video. and reference examples The sum of similarity scores across all modalities. Target domain-aware pseudo-labels. The pseudo-labels are derived from stable (low-entropy) and semantically relevant reference videos, thus accurately reflecting the category distribution of the news event cluster to which the query video belongs.

[0127] S9: Perform self-training optimization. Define the self-training objective function as:

[0128] ,

[0129] During testing, the source model was optimized using the following comprehensive loss function. :

[0130] ,

[0131] in It is the entropy minimization objective widely used in TTA. and This is the balance ratio. Based on previous work, only the modal encoder and normalization layer are learnable to ensure efficient and stable adaptation. An online TTA setup is employed, using the source model... First, optimize the news videos in the current batch, and then generate the final prediction for that batch in real time.

[0132] In steps S3-S4, the entropy selection retrieval mechanism is designed based on an important observation: the source-trained model, without adaptation, produces low predictive entropy for some videos within the same event in the target domain, while producing high entropy for others. Low-entropy videos often share common patterns with source samples (e.g., similar editing techniques or narrative styles), causing the source model to produce representations closer to the source domain and more accurate detections. By retrieving semantically relevant videos and filtering out stable low-entropy instances, these stable target instances can act as source domain proxies, aiding robust adaptation when source data is inaccessible.

[0133] In steps S5-S7, stable anchor point-guided alignment is one of the core innovations of this invention. Traditional TTA methods typically employ an entropy minimization-based strategy to implicitly align cross-domain representations, but this indirect approach struggles to handle the substantial representation gaps caused by topic-level distribution shifts in fake news video detection. This invention models modal features as multivariate Gaussian distributions and utilizes KL divergence for explicit alignment at the distribution level, thus more effectively reducing the representation gap between the source and target domains. The construction of the anchor point distribution aggregates statistical information from multiple stable reference instances, ensuring the stability and effectiveness of the alignment.

[0134] In steps S8-S9, the target domain-aware self-training paradigm is a key innovation of this invention in addressing the challenges of imbalanced and dynamically changing category distributions. The varying popularity and public attention of different news events lead to a severe imbalance, constant change, and unpredictability in the category distribution of newly arriving video clusters related to different breaking events. Traditional methods only work well when the label distribution in the target domain is stable and consistent with the source domain. This invention generates pseudo-labels guided by stable reference instances. These pseudo-labels accurately reflect the category distribution of the current event cluster, enabling the model to quickly adapt to imbalanced and changing category distributions, achieving real-time online detection.

[0135] Experimental results

[0136] 1. Experimental Setup

[0137] 1.1 Dataset

[0138] To verify the effectiveness of this invention, three publicly available fake news video detection datasets, differing in event time range, language, or platform, were selected for experimental evaluation: the FakeTT dataset, spanning from May 2019 to March 2024, containing 1991 video samples (1172 fake, 819 real), covering 286 events, using English, from the TikTok platform; the FakeSV dataset, spanning from October 2017 to February 2022, containing 3624 video samples (1810 fake, 1814 real), covering 738 events, using Chinese, from the Douyin platform; and the FVC dataset, spanning from January 2016 to January 2018, containing 2764 video samples (1633 fake, 1131 real), covering 305 events, using English, from the YouTube platform.

[0139] These three datasets do not overlap in event time range and differ across languages ​​and platforms. No two datasets have overlapping events, exhibiting significant covariate shifts. This selection effectively simulates topic-level distribution shifts in the evolution of fake news videos, providing a realistic and challenging evaluation scenario for test-time adaptation. By setting one dataset as the source domain and the other two as the target domains, six evaluation tasks are constructed (e.g., FakeTT→FakeSV).

[0140] 1.2 Evaluation Indicators

[0141] To comprehensively evaluate the performance of this invention, following the conventions of previous work on fake news video detection, accuracy (Accuracy, Acc), macro-F1 score (M-F1), and macro-recall (MR) are used as the main evaluation metrics. Accuracy reflects the overall classification correctness of the model, macro-F1 score comprehensively considers precision and recall, and is particularly suitable for class imbalance situations, while macro-recall measures the model's ability to identify all classes. The combined use of these metrics can comprehensively evaluate the model's performance in the fake news video detection task.

[0142] 1.3 Experiment Setup Instructions

[0143] The most popular online adaptive setting is adopted, which means that evaluation is performed immediately after each batch of data arrives. To comprehensively evaluate the method's performance, the sampling method is defined as follows:

[0144] Random batch sampling: Each batch is randomly selected from the target domain, and the batch size is set to 128. Under this setting, the class distribution of each batch is relatively balanced and almost unchanged, representing a relatively mild evaluation scenario.

[0145] 1.4 Benchmark Method

[0146] To evaluate the performance of this invention (RADAR), it was compared with eleven representative benchmark models, which can be divided into two groups:

[0147] (1) Traditional Fake News Video Detection Methods: This group includes methods trained on the source dataset but without adaptation, including FANVM (Multimodal Fake News Video Detection), SV-FEND (Social Video Fake Information Detection), FakRec (Fake Video Detection Based on Creation Process), ExMRD (Cross-Modal Relation Decomposition Detection), and REAL (Prototype-Aligned Authenticity Detection). These methods are designed for the fake news video detection task but assume that the topic distribution is consistent during the training and testing phases.

[0148] (2) Test-Time Adaptive Methods: This group of methods can adapt to distribution shifts during testing, including Source (source model, no adaptation), Tent (TTA method based on entropy minimization), SAR (field TTA method combining batch-independent normalization optimization and parameter recovery), READ (TTA method for multimodal applications), ABPEM (attention-based permutation entropy minimization method), and SuMi (smoothly adaptive multimodal TTA method). These methods represent the latest advancements in the TTA field, but are mainly designed for scenarios with mild distribution shifts.

[0149] 1.5 Implementation Details

[0150] The source model uses a feedforward network (FFN) as the modality encoder, a two-layer Transformer as the modality fusion module, and a multilayer perceptron (MLP) as the classifier. Video frames are used for visual modality information, video titles and screen text for text modality information, and audio transcription for audio modality information. Pre-trained ViT and BERT are used for feature extraction and multimodal retrieval. The adaptive process uses an initial learning rate of 0.0001 within a single epoch. Entropy thresholding is used. Set as ,in Number of categories. Pseudo-label coefficient. and All are set to 0.5. and The default setting is 1. For random batch sampling, the batch size is 128; for event-level batch sampling, the batch size is set to the average number of news videos related to the same event to better simulate real-world conditions.

[0151] 2. Main Experimental Results

[0152] 2.1 Performance Comparison under Random Batch Sampling

[0153] Under the same experimental settings, RADAR was compared with all 11 benchmark models across all six evaluation tasks. The table below shows the full results under the randomized batch sampling setting, with the best results in each column highlighted in bold:

[0154] method FakeTT→FakeSV FakeTT→FVC FakeSV→FakeTT Acc M-F1 MR Acc M-F1 MR Acc M-F1 MR FANVM 51.43 42.21 51.39 55.03 55.01 57.27 57.48 45.13 50.65 SV-FEND 53.53 50.36 53.56 58.50 56.96 56.94 63.00 60.06 60.07 FakRec 62.50 62.24 62.49 60.24 58.83 58.82 61.09 55.74 60.45 ExMRD 50.52 34.77 50.47 61.25 58.02 59.13 61.85 46.70 54.08 REAL 54.75 51.19 54.78 58.61 52.12 53.69 60.04 54.52 55.60 Source 62.47 59.77 62.44 56.55 55.94 56.27 59.69 40.21 51.16 Tent 64.74 64.49 64.74 60.53 56.65 56.97 60.49 46.34 53.04 SAR 65.56 64.95 65.58 61.83 59.57 59.50 60.04 45.53 52.53 READ 64.02 63.83 64.03 56.40 56.04 56.61 62.90 62.55 63.17 SuMi 58.83 55.93 58.86 60.60 56.23 56.74 59.09 59.09 60.99 RADAR 71.61 71.60 71.61 64.44 63.83 64.21 65.71 64.49 64.46

[0155] method FakeSV→FVC FVC→FakeSV FVC→FakeTT Acc M-F1 MR Acc M-F1 MR Acc M-F1 MR FANVM 53.26 44.80 47.72 50.91 49.35 50.89 47.69 46.88 46.98 SV-FEND 61.00 45.02 52.95 57.34 57.07 57.33 56.33 56.32 58.09 FakRec 63.02 52.85 56.45 60.18 59.10 60.16 61.95 61.87 63.16 ExMRD 63.42 51.76 56.35 50.19 33.81 50.14 56.88 56.71 60.54 REAL 60.96 45.54 53.05 57.59 57.12 57.58 60.19 59.64 60.01 Source 63.93 49.57 56.09 59.00 53.68 58.96 61.85 61.85 63.86 Tent 61.40 46.31 53.54 63.49 61.81 63.52 61.70 61.69 63.55 SAR 61.29 45.96 53.38 64.46 63.64 64.48 63.05 63.02 64.59 READ 63.89 63.36 63.80 64.62 63.52 64.64 63.40 63.25 64.28 SuMi 60.96 60.90 62.27 55.38 47.54 55.42 63.55 57.72 58.81 RADAR 69.68 68.05 67.82 68.87 68.36 68.89 66.52 65.70 65.86

[0156] Based on the above experimental results, the following important observations can be drawn:

[0157] Observation 1: Limitations of traditional fake news video detection methods and existing TTA methods. Traditional methods assume consistent distributions between training and testing data, thus failing to address significant distribution shifts caused by the evolution of fake news videos. These methods exhibit relatively low performance across all evaluation tasks, demonstrating their failure in test-time adaptive scenarios. TTA methods demonstrate some effectiveness in adapting to the target domain with distribution shifts, showing a relatively consistent performance improvement compared to the source model. However, these methods employing entropy minimization strategies still struggle to address the significant representation gaps caused by topic-level distribution shifts in fake news video detection.

[0158] Observation 2: RADAR's significant superiority. RADAR significantly outperforms all competing benchmarks across all evaluation tasks, achieving an average improvement of 6.55% in macro F1 scores. This significant performance improvement stems from the proposed novel retrieval-guided adaptive paradigm. Specifically, this paradigm effectively narrows the cross-domain gap by explicitly aligning the representations of unstable news video instances with semantically similar and stable reference videos. Furthermore, the self-training paradigm further enhances the robustness of fake news video detection during test-time adaptation by leveraging highly informative pseudo-labels derived from stable reference video augmentations.

[0159] 2.2. Ablation Experiment

[0160] To explore the role of each component in RADAR and conduct a comprehensive ablation study, the results under a randomized batch sampling setting are shown in the table below:

[0161] Module variants FTT→FSV Acc FTT→FSV M-F1 FTT→FVC Acc FTT→FVC M-F1 Search No similarity search 67.52 67.37 63.71 62.92 Search Entropy-free choice 65.98 65.98 59.01 58.15 Alignment MSE Alignment 66.53 66.27 64.04 63.38 Alignment Unaligned 64.49 64.47 63.35 62.62 Self-training No reference, self-annotated 68.57 68.24 51.01 50.84 Self-training No self-training 69.62 69.58 58.83 57.99 RADAR Complete method 71.61 71.60 64.44 63.83

[0162] Validation of the entropy selection retrieval mechanism: Two variants were designed: (1) "No similarity retrieval" based solely on entropy selection of reference instances; and (2) "No entropy selection" based solely on similarity selection of reference instances. Both variants suffered significant performance degradation because aligning representations with irrelevant or unstable (high entropy) reference videos led to unreliable adaptation and potential model collapse. This demonstrates the necessity of considering both semantic relevance and prediction stability.

[0163] Validation of the effectiveness of the stable anchor-guided alignment module: Two variants are introduced: (1) “MSE alignment” replaces the distribution-level alignment with a simple L2 norm minimization between the features of the query video and the reference instance; (2) “No alignment” completely removes the alignment module. MSE alignment causes a significant performance drop because it performs a simple point-to-point alignment, ignoring the underlying semantic structure of the representation and forcing rigid feature matching, resulting in feature distortion. Removing the alignment module also leads to suboptimal performance because the significant cross-domain representation differences caused by topic-level shifts in fake news video detection require more explicit remedies.

[0164] Validation of the effectiveness of the target domain-aware self-training paradigm: Two variants are proposed: (1) "No-reference self-labeling" pseudo-labels are generated from the query video without any reference enhancement; (2) "No self-training" eliminates self-training. Both variants lead to performance degradation because pseudo-labels derived under the guidance of stable reference videos are more informative and reliable, better reflecting the category distribution of the current event cluster, thus achieving more effective adaptation.

[0165] 3. Experimental Conclusions

[0166] Based on the above experimental results, the following conclusions can be drawn:

[0167] First, RADAR achieved the best performance across all evaluation tasks, with an average macro F1 improvement of 6.55% in the random batch sampling setting and an average macro F1 improvement of 5.45% in the more challenging event-level batch sampling setting. This demonstrates the effectiveness and robustness of RADAR across different scenarios.

[0168] Second, ablation experiments fully validated the necessity and effectiveness of each module of RADAR. The entropy selection retrieval mechanism ensured the quality of reference instances, the stable anchor-guided alignment module effectively reduced cross-domain representation gaps, and the target domain-aware self-training paradigm enabled the model to adapt to dynamically changing class distributions. The three modules worked together to achieve superior test-time adaptive performance.

[0169] These experimental results fully verify the effectiveness and superiority of the retrieval-enhanced test-time adaptive detection method for fake news videos proposed in this invention, providing an innovative and efficient technical solution for solving the test-time adaptive problem in practical applications. This invention is not only significant in academic research but also provides strong technical support for the practical deployment of fake news video detection systems, and is expected to play an important role in ensuring network information security and maintaining social stability.

[0170] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive detection method for testing fake news videos, characterized in that: Includes the following steps; S1: Initialize the pre-trained multimodal source model and a memory bank for storing the most recent samples; S2: In data stream scenarios, newly arriving news video batches are continuously written to the memory bank in a FIFO manner to retain the latest cross-event context information; S3: Based on the semantic similarity of the three modalities of vision, text and audio, retrieve semantically relevant reference instances from the memory for the currently queried video; S4: Introduce prediction entropy as a stability criterion to filter out unreliable reference instances and retain only low-entropy, stable and semantically relevant reference instances. S5: Explicitly model the sequence features of the query video in each modality as a probability distribution; S6: Calculate the modal feature distribution of the reference instance obtained in step S4, and aggregate the modal feature distribution of the reference instance to form a "stable anchor point distribution" representing the semantic structure of the source domain; S7: By minimizing the difference between the probability distribution of the query video and the stable anchor distribution, explicit cross-modal distribution alignment guided by stable anchors is achieved, pulling the target domain representation back into the source domain semantic space; S8: Use reference examples to perform weighted fusion of multimodal source model predictions to generate high-quality pseudo-labels with target domain awareness; S9: Combine distribution alignment loss, pseudo-label self-training loss, and entropy minimization objective to perform online optimization of multimodal source models.

2. The adaptive detection method for testing fake news videos according to claim 1, characterized in that: Step S3 specifically includes the following: Retrieve the video from the memory for the currently queried video. A semantically relevant reference example: ， in, This indicates that cosine similarity is used to measure semantic relevance within a modality, and Top-K() means to return the top K reference instances ranked by the median of the (). , , They represent the i-th reference instance in the memory bank, respectively. Visual, textual, and audio modal information; taking the visual modality as an example: ， in, It is the basic visual model used for multimodal retrieval, where T represents the matrix transpose operation.

3. The adaptive detection method for testing fake news videos according to claim 2, characterized in that: Step S4 specifically includes the following: Semantically relevant reference examples Stable reference instances are further filtered out through an entropy selection process: ， in This represents a set of low-entropy, stable, and semantically relevant reference instances for the queried video. It is a predefined entropy threshold. Indicates sample Predicted entropy: ， here Representation source model The resulting probability distribution Indicates the number of categories. yes For the The probability of a class being predicted.

4. The adaptive detection method for testing fake news videos according to claim 3, characterized in that: Step S5 specifically includes the following: For searching videos Its modal features are modeled as a multivariate Gaussian distribution: ， in, Indicates the modal type, and These are the core statistical parameters describing its location and diffusion. Represents a distribution, This refers to the representation of modalities: ， here It is modal The representation of which It is the sequence length. It is the feature dimension, and towards Add a small identity matrix .

5. The adaptive detection method for testing fake news videos according to claim 4, characterized in that: Step S6 specifically includes the following: Calculation and Query Video Related reference example collection The modal feature distributions are obtained and aggregated to obtain a stable anchor point distribution: ， The distribution statistics parameter is defined as follows: ， in, and This is a reference example. Parameters, stable anchor point distribution Constructed from low-entropy and semantically similar reference instances, it is stable and closely approximates the source domain distribution, while also being consistent with the distribution of the query videos. They share similar semantics.

6. The adaptive detection method for testing fake news videos according to claim 5, characterized in that: Step S7 specifically includes the following: Based on a stable anchor point distribution, the query video is explicitly linked by minimizing the KL divergence. The characteristic distribution is aligned with its stable anchor point distribution, and its representation is pulled toward the source domain: ， two Gaussian distribution and The closed-form form of the KL divergence between them is: ， in, Let p and q be the covariance matrices. , The mean vectors of distributions p and q, express The inverse matrix.

7. The adaptive detection method for testing fake news videos according to claim 4, characterized in that: Step S8 specifically includes the following: Based on low-entropy, stable, and semantically relevant reference instances, for each query video Assigning pseudo-labels : ， in and These are multimodal source models For video search and its reference examples In category The predicted probability, and It is a weighting factor. It is a video search. and its reference examples The sum of similarity scores across all modalities, the target domain-aware pseudo-label. This is derived through pseudo-label enhancement using low-entropy, stable, and semantically relevant reference instances.

8. The adaptive detection method for testing fake news videos according to claim 4, characterized in that: Step S9 specifically includes the following: Define the self-training objective function as: ; During testing, the source model was optimized using the following comprehensive loss function. : ; in, It is the entropy minimization objective widely used in TTA. and It is the balance ratio.

9. A system for an adaptive detection method during testing of fake news videos according to any one of claims 1-8, characterized in that: include: The entropy selection retrieval module is used to provide stable and semantically relevant reference instances for each query video; The Stable Anchor Point Guided Alignment Module is used to explicitly reduce the substantial representation gap between the source and target domains caused by topic-level distribution offsets. The target domain-aware self-training module assigns target domain-aware pseudo-labels to queries under the guidance of reference examples and guides the source model to adapt to new category distributions online.

10. The adaptive detection system for testing fake news videos according to claim 9, characterized in that: The entropy selection retrieval module includes two cooperating sub-modules: a memory bank management sub-module and a semantic retrieval sub-module. The memory management submodule uses a first-in-first-out strategy to maintain a dynamically updated memory, storing the most recently arrived news video instances and their multimodal features; The semantic retrieval submodule retrieves the Top-K most relevant reference instances based on the cross-modal semantic similarity between the query video and the samples in the memory bank. Then, it selects the reference instances whose predicted entropy is lower than the dynamic threshold through an entropy filtering mechanism, thus providing high-quality reference data for the target video. The stable anchor point guided alignment module includes a distributed modeling submodule, an anchor point construction submodule, and an explicit alignment submodule; The distribution modeling submodule uses the modal encoder of the source model to encode the features of the query video and the retrieved reference instance, and models the feature representation of each modality as a multivariate Gaussian distribution, and describes the location and spread of the features by calculating the mean vector and covariance matrix; The anchor point construction submodule constructs a stable anchor point distribution by aggregating the modal feature distributions of multiple reference instances. This anchor point distribution is stable and close to the source domain distribution, while sharing similar semantics with the queried video. The explicit alignment submodule achieves distribution-level explicit alignment by minimizing the KL divergence between the query video feature distribution and the stable anchor point distribution, thereby pulling the representation of the query video toward the source domain.

Images