Method and system for detecting cross-sample fake news videos

Abstract

The application provides a method for detecting cross-sample false news videos, comprising: obtaining a news video and a refutation video of a target event, extracting single-sample features of the news video as first features, and extracting single-sample features of the refutation video as second features; constructing an event graph with the first features as nodes and importance between the nodes as edges; obtaining node features of the nodes by information aggregation, classifying the node features into true and false, selecting a news video corresponding to a node feature classified as true as a to-be-detected video, and selecting a first false video classified as false; selecting a second false video in the to-be-detected video by detecting factual information conflicts between the second features and the first features of the to-be-detected video, and taking the first false video and the second false video as false news videos. The application also provides a system for detecting cross-sample false news videos, and a data processing device for detecting cross-sample false news videos.

Description

Technical Field

[0001] This invention relates to the field of news credibility authentication technology, and in particular to a method and system for detecting fake news videos across samples. Background Technology

[0002] In recent years, short video platforms such as Douyin and Kuaishou have spawned a large number of fake news videos. Compared with traditional text-based and image-based fake news, video-based fake news is more attractive and persuasive, making fake news video detection an emerging research area in multimodal fake news detection tasks. Fake news videos generally consist of a news headline and a video. The goal of the fake news video detection task is to provide a binary classification judgment of true or false for the input news video.

[0003] Most existing work focuses on how to fully utilize multimodal information within a single sample for classification. As one of the most representative works, Qi et al. proposed SV-FEND (a short video fake news detection model), which currently boasts the best detection performance. Figure 1 As shown, this model extracts multimodal features from titles and subtitles, audio, keyframes, video clips, comments, and user profiles. It uses two cross-modal Transformers to model the association between text, audio, and keyframe features, and finally uses a Transformer to fuse all features for classification.

[0004] Although existing methods have made relatively full use of multimodal information in single samples, the detection clues presented in a single video are often not obvious due to the elaborate manipulation by forgers. This limits the performance of existing single-sample-based detection methods. For example, some fake news videos only modify news elements such as time and location in the title of genuine news videos, and detectors based on single-sample multimodal content struggle to successfully detect such fake news videos. Summary of the Invention

[0005] To achieve effective automatic detection of fake news videos, this invention addresses the technical problem of limited single-sample clues in existing technologies by proposing a cross-sample fake news video detection method. The method includes: acquiring news videos and debunking videos of the target event; extracting single-sample features from the news video as a first feature and extracting single-sample features from the debunking video as a second feature; constructing an event graph with the first feature as nodes and the importance representations between nodes as edges; acquiring node features through information aggregation; performing true / false binary classification on the node features; selecting news videos corresponding to node features classified as true as test videos and news videos corresponding to node features classified as false as first fake videos; and selecting second fake videos from the test videos by detecting factual information conflicts between the second feature and the first feature of the test video. The first and second fake videos are then considered as the fake news videos of the target event.

[0006] The method for detecting fake news videos across samples according to the present invention includes a step of detecting factual information conflict between the second feature and the first feature of the video to be tested, comprising: taking the text feature S in the second feature... D And the text feature S in the first feature C Generate feature pairs D S C > By modeling using the BERT model, text conflict features x are obtained. t =BERT([CLS]S D [SEP] S C [SEP]); add temporal location encoding f to the keyframe feature set in the second feature. tem and classification tags Obtain the keyframe feature set after processing the debunking video. Add temporal location encoding (FTEM) and classification tags to the keyframe feature set in the first feature. Obtain the keyframe feature set of the video under test after processing. Enhanced with stacked self-attention and cross-attention modules and And and Visual consistency features are obtained by concatenating vectors. x through self-attention layer t and x v Dynamic fusion is performed, and the fused features are binary classified to obtain the probability that the test video is the second fake video. Then predict the probability that the news video is a fake news video. To predict the probability that the news video is the first fake video. ​

[0007] The method for detecting fake news videos across samples according to the present invention, wherein the event graph is organized in the form of a graph attention neural network. Side e ij Obtained through the attention mechanism, representing node v j The first feature of node v i The importance of the first feature.

[0008] The method for detecting fake news videos across samples as described in this invention, wherein node v i Node features e ij =LeakyReLU(a T [Wv i Wv j ]), For v i The set of neighboring nodes, α i，j For v i With v j The weights between them are a and W, which are trainable parameters. [·,·] represents the concatenation operation, and σ is a non-linear operation.

[0009] The method for detecting fake news videos across samples as described in this invention includes the analysis of node features. Loss function for true / false binary classification This indicates the authenticity of the news video.

[0010] The cross-sample fake news video detection method of the present invention uses the SV-FEND single-sample detector to extract the first feature and the second feature.

[0011] This invention also proposes a cross-sample fake news video detection system, comprising: a feature extraction module, used to acquire news videos and debunking videos of a target event, extracting single-sample features of the news video as a first feature, and extracting single-sample features of the debunking video as a second feature; a graph aggregation module, used to construct an event graph with the first feature as a node and the importance representation between the nodes as edges; acquiring node features of the node through information aggregation, and selecting news videos corresponding to node features classified as true as test videos and news videos corresponding to node features classified as false as first fake videos by performing true / false binary classification on the node features; and a sample correction module, used to select second fake videos in the test videos by detecting factual information conflicts between the second feature and the first feature of the test video, and using the first fake video and the second fake video as fake news videos of the target event.

[0012] The cross-sample fake news video detection system of the present invention includes a sample correction module comprising: a text conflict detection module, used to acquire text conflict features; and to modify the text feature S in the second feature... D And the text feature S in the first feature C Generate feature pairs D S C > By modeling using the BERT model, text conflict features x are obtained. t =BERT([CLS]S D [SEP]S C [SEP]); Visual consistency evaluation module, used to obtain visual consistency features; add temporal location encoding f to the keyframe feature set in the second feature. tem and classification tags Obtain the keyframe feature set after processing the debunking video. Add time location encoding f to the keyframe feature set in the first feature tem and classification tags Obtain the keyframe feature set of the video under test after processing. Enhanced with stacked self-attention and cross-attention modules and And and Visual consistency features are obtained by concatenating vectors. Attention fusion and classification modules are used to classify x through a self-attention layer. t and x v Dynamic fusion is performed, and the fused features are binary classified to obtain the probability that the test video is the second fake video. Then predict the probability that the news video is a fake news video. To predict the probability that the news video is the first fake video.

[0013] The present invention also proposes a computer-readable storage medium storing computer-executable instructions, characterized in that, when the computer-executable instructions are executed, the detection of cross-sample fake news videos as described above is realized.

[0014] The present invention also proposes a data processing apparatus, including a computer-readable storage medium as described above, wherein when the processor of the data processing apparatus retrieves and executes computer-executable instructions in the computer-readable storage medium, it realizes the detection of fake news videos across samples. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the single-sample detection model SV-FEND. ​

[0016] Figure 2 This is the cross-sample detection model of the present invention.

[0017] Figure 3 This invention is a debunking relationship reasoning model.

[0018] Figure 4 This is a schematic diagram of the data processing device of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] While researching fake news videos, the inventors discovered that the shortcomings of existing technologies stemmed from insufficient multimodal cues from single samples. Their research revealed that in real-world scenarios, when a news event occurs, multiple related videos from different perspectives are often released, including fake news videos, genuine news videos, and related debunking videos. These news videos may contain complementary or conflicting information, helping to verify each other's authenticity. Furthermore, debunking videos are widely available on short video platforms, providing direct assistance in detecting fake news.

[0021] The goal of this invention is to take a set of videos related to the same event as input and provide a binary classification judgment of "true" or "false" for each video. The cross-sample detection model proposed in this invention includes three stages: single-sample feature extraction, multi-sample graph aggregation, and debunking sample correction. Figure 2 As shown.

[0022] 1. Single-sample feature extraction

[0023] In this stage, any existing single-sample detector (such as SV-FEND) can be used for feature extraction. First, the model is pre-trained under supervision. Then, the model parameters are frozen, and for the input news video set, the corresponding multimodal feature representation F is extracted before the classification layer. base For subsequent operations.

[0024] 2. Multi-sample graph aggregation

[0025] 2.1 Graph Construction

[0026] Given a feature set of related news videos under the same event E This invention organizes it in the form of a graph attention neural network. Representation diagram, express In the context of nodes, ε represents the edge between nodes. Each node... The edge e represents the feature corresponding to a news video generated by a single-sample detector. ij The importance of the features of node j to the features of node i is represented by the attention mechanism.

[0027] 2.2 Feature Aggregation and Classification

[0028] To aggregate neighbor information, this invention constructs an event graph. We then use an attention mechanism to update the node representation and classify the nodes. Specifically, given a node v... i and its neighbors v i and its neighbors The weights α between i,j It is expressed as:

[0029] e ij =LeakyReLU(a T [Wv i ,Wv j ])

[0030]

[0031] a and W are trainable parameters, and [·,·] denotes the concatenation operation. Then, v i Feature representation Updates are achieved through information aggregation:

[0032]

[0033] σ represents a non-linear operation. To avoid excessive smoothing of node features, this invention employs only two graph attention layers. Ultimately, node v... i Feature representation The input news video is fed into a binary classifier for classification, and the output is the probability of predicting it as fake. The model is optimized using binary cross-entropy loss:

[0034]

[0035] y∈{0,1} represents the true label.

[0036] 3. Correction of debunking samples

[0037] The debunking correction module utilizes the conflict of factual information between the debunking video and the corresponding fake news video to detect fake news videos that were "missed" by the first two stages. To achieve this effect, this invention proposes a novel multimodal reasoning task to identify this relationship, namely, debunking relation inference. The task is defined as follows: given a debunking video and candidate videos belonging to the same event, the goal of debunking relation inference (DRI) is to determine whether the debunking video can refute the candidate video.

[0038] The purpose of this module is to correct false negative predictions generated by the model in the previous stages. Specifically, for a given event, this invention considers videos predicted as true in the aforementioned stages as candidate video sets. For each candidate video This invention compares it with any debunking video from the same event. They are fed into the DRI model together. Then, the predicted probabilities from the graph aggregation module are... The predicted probability of the DRI model Combined to assess the authenticity of candidate videos (final prediction probability p) i Verification will be performed.

[0039]

[0040]

[0041] To achieve the purpose of debunking relational reasoning, this invention designs its model according to the following three principles:

[0042] Detect conflicts between debunking videos and candidate video news text.

[0043] Detect the consistency between video segments in a given video pair. For example, if a debunking video refutes a fake news video that mistakenly uses an "old" video segment from a previous event, the present invention needs to determine whether the candidate video also contains that "old" video segment.

[0044] Dynamically integrate textual and visual evidence to mitigate the impact of visual inconsistencies in news events where visual evidence is not critical (such as "the United Nations declares Chinese as an international language").

[0045] Based on the above principles, this invention designs a new multimodal reasoning model (such as...). Figure 3 As shown in the figure, it includes three stages: text conflict detection, visual consistency assessment, and attention fusion and classification.

[0046] Text Conflict Detection: Inspired by natural language reasoning tasks, this invention detects text conflicts through consistency between sentence pairs. Specifically, given a debunking video, this invention extracts the subtitle text from the video and concatenates it with the video title to form a word sequence S. D = [w1, ..., w m ], where w i S represents the i-th word in the combined sentence. Similarly, the news text in the candidate news video is represented as S. C = [w1, ..., w n Then, the present invention will pair sentences. D S C The tokens are fed into BERT to model intra- and inter-sentence correlations. Each token is appended with a learnable type embedding vector indicating that it belongs to S. D Or S C Finally, the present invention yielded the following text conflict features:

[0047] x t =BERT([CLS]S D [SEP]S C [SEP])

[0048] Visual Consistency Assessment: To match video segments, this invention utilizes an EfficientNet model pre-trained on an image duplication detection dataset to obtain visual representations. Specifically, this invention evaluates the keyframe features F of a given debunking video. D Keyframe features F of candidate videos C They are respectively represented as

[0049]

[0050]

[0051] This invention employs fixed sine and cosine time position encoding f tem It is added to the initial features and a learnable classification label f ​​is assigned. [CLS] It is added as a global feature to the feature sequence. (Debunking video F) D And candidate video F C The processed frame features are represented as follows:

[0052]

[0053]

[0054] ​Similar to text conflict detection, this invention needs to consider intra-video and inter-video correlations. Therefore, this invention employs stacked self-attention and cross-attention modules to enhance the initial features, where the query vector in the cross-attention module comes from another video. Finally, the visual consistency feature is obtained by concatenating the classification label vectors corresponding to the debunking video and the candidate video.

[0055]

[0056] Attention fusion and classification: Given text conflict features x t Visual consistency features x v This invention uses a self-attention layer to dynamically fuse these features to discover important information and eliminate irrelevant information. The fused features are then fed into a binary classifier to obtain the probability that a debunking video refutes a candidate video in the above formula.

[0057] Figure 4 This is a schematic diagram of the data processing apparatus of the present invention. Figure 4 As shown, embodiments of the present invention also provide a computer-readable storage medium and a data processing apparatus. The computer-readable storage medium of the present invention stores computer-executable instructions, which, when executed by the processor of the data processing apparatus, enable the detection of cross-sample fake news videos. Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware (e.g., processor, FPGA, ASIC, etc.), and the program can be stored in a readable storage medium, such as a read-only memory, a disk, or an optical disk. All or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiments can be implemented in hardware, for example, by using integrated circuits to implement its corresponding function, or it can be implemented as a software functional module, for example, by a processor executing a program / instruction stored in memory to implement its corresponding function. Embodiments of the present invention are not limited to any particular combination of hardware and software.

[0058] The fake news video detection method of this invention employs a cross-sample detection model, utilizing three stages: single-sample feature extraction, multi-sample graph aggregation, and debunking sample correction. This fully leverages multimodal clues, multi-sample associations, and the corrective effect of debunking samples within single samples. The debunking relationship inference model, including text conflict detection, visual consistency verification, and dynamic fusion, effectively detects fake news videos matching debunking videos. Compared to existing technologies, this method achieves a significant performance improvement without requiring additional data. This invention can be integrated with any single-sample detection model, and on publicly available benchmark datasets, it helps six existing single-sample detection models achieve a detection accuracy improvement of 4.67–8.34 percentage points.

[0059] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims

1. A method for detecting fake news videos across samples, characterized in that, include: Obtain news videos and debunking videos of the target event, extract the single-sample features of the news video as the first feature, and extract the single-sample features of the debunking video as the second feature; Construct an event graph with the first feature as nodes and the importance representation between the nodes as edges; obtain the node features of the node by information aggregation; perform true and false binary classification on the node features; select the news videos corresponding to the node features that are classified as true as the test videos, and the news videos corresponding to the node features that are classified as false as the first false videos. By detecting factual information conflicts between the second feature and the first feature of the test video, the second fake video in the test video is selected, including: selecting the text feature of the second feature. and the text features in the first feature Generate feature pairs Text conflict features are obtained by modeling using the BERT model. Add time location encoding to the keyframe feature set in the second feature; and classification tags The keyframe feature set of the debunking video was obtained. Add time location encoding to the keyframe feature set in the first feature; and classification tags The keyframe feature set of the video under test is obtained after processing. Enhanced by using stacked self-attention and cross-attention modules and and will and Visual consistency features are obtained by concatenating vectors. ; through self-attention layer and Dynamic fusion is performed, and the fused features are binary classified to obtain the probability that the test video is the second fake video. Then predict the probability that the news video is a fake news video. , To predict the probability that the news video is the first fake video; the first fake video and the second fake video are fake news videos of the target event.

2. The method for detecting fake news videos across samples as described in claim 1, characterized in that, Organize the event graph in the form of a graph attention neural network. ,side Obtained through an attention mechanism, representing the node. The first feature of the node The importance of the first feature.

3. The method for detecting fake news videos across samples as described in claim 2, characterized in that, node Node features : in, , , for The set of neighboring nodes, , for and The weights between them , For trainable parameters, This indicates a splicing operation. This is a non-linear operation.

4. The method for detecting fake news videos across samples as described in claim 3, characterized in that, To analyze node features Loss function for true / false binary classification ,in This indicates the authenticity of the news video.

5. The method for detecting fake news videos across samples as described in claim 1, characterized in that, The first feature and the second feature are extracted using the SV-FEND single-sample detector.

6. A cross-sample fake news video detection system, characterized in that, include: The feature extraction module is used to acquire news videos and debunking videos of the target event, extract the single sample features of the news video as the first feature, and extract the single sample features of the debunking video as the second feature. The graph aggregation module is used to construct an event graph with the first feature as nodes and the importance representation between the nodes as edges; the node features of the node are obtained by information aggregation; the node features are classified as true or false by performing true / false binary classification on the node features; the news videos corresponding to the node features classified as true are selected as the test videos, and the news videos corresponding to the node features classified as false are selected as the first false videos. The sample correction module is used to select a second fake video from the test video by detecting factual information conflicts between the second feature and the first feature of the test video, and to identify the first fake video and the second fake video as fake news videos of the target event; it includes: a text conflict detection module, an attention fusion and classification module, and an attention fusion and classification module, wherein, The text conflict detection module is used to obtain text conflict features; the text features in the second feature are then used to obtain text conflict features. and the text features in the first feature Generate feature pairs Text conflict features are obtained by modeling using the BERT model. ; The visual consistency evaluation module is used to obtain visual consistency features; temporal location encoding is added to the keyframe feature set in this second feature. and classification tags The keyframe feature set of the debunking video was obtained. Add time location encoding to the keyframe feature set in the first feature; and classification tags The keyframe feature set of the video under test is obtained after processing. Enhanced by using stacked self-attention and cross-attention modules and and will and Visual consistency features are obtained by concatenating vectors. ; The attention fusion and classification module is used to perform self-attention layer operations. and Dynamic fusion is performed, and the fused features are binary classified to obtain the probability that the test video is the second fake video. Then predict the probability that the news video is a fake news video. , To predict the probability that the news video is the first fake video.

7. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, the detection of cross-sample fake news videos as described in any one of claims 1 to 5 is achieved.

8. A data processing apparatus comprising the computer-readable storage medium as claimed in claim 7, wherein when a processor of the data processing apparatus retrieves and executes computer-executable instructions in the computer-readable storage medium, it enables the detection of cross-sample fake news videos.

Images