A method, device and equipment for detecting news containing misleading information

Abstract

The application relates to the technical field of network space security information detection, and discloses a misleading information-containing news detection method, device and equipment, the method comprising the following steps: acquiring a public real social media data set, the social media data set has multi-modal news; a cross-modal content correlation network model is constructed and trained to obtain a trained misleading information-containing news detection model; inputting to-be-detected news data into the trained misleading information-containing news detection model, and predicting whether the to-be-detected news contains misleading information through the model. The application uses a new cross-modal content correlation network to enhance multi-modal misleading information-containing news detection, realizes extraction of fine-grained text and visual content correlation, overcomes the limitation of using scalar similarity reasoning in the previous misleading information-containing multi-modal news detection technical scheme, and improves the accuracy of misleading information-containing news detection.

Description

Technical Field

[0001] This application relates to the field of cyberspace security information detection technology, and in particular to a method, apparatus and equipment for detecting news containing misleading information. Background Technology

[0002] In many dialogue systems, including social media platforms, chatbots, and online forums, user interactions and information exchanges often become channels for the dissemination of news containing misleading information. This misleading news can not only mislead users and influence their decisions, but also negatively impact the platform's reputation and user experience.

[0003] Multimodal news containing misleading information often exhibits inconsistencies between text and images (pictures). Therefore, some studies utilize scalar similarity measures to quantify differences between modalities, such as SAFE (Similarity-Aware Multi-Modal Fake News Detection) and EM-FEND (Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues). SAFE uses a pre-trained tool to convert news images into text and then calculates a modified cosine similarity with the news text, which is used as an auxiliary loss function and learned co-with the classification task. EM-FEND extracts text from images to supplement textual content and leverages public APIs to extract representations of specific entities, such as people and landmarks, from both text and images. Enhancement is achieved through a common attention Transformer running across text, textual entities, visual entities, and visual features, while simultaneously concatenating the cosine similarity scalars between text and visual entity pairs to account for entity consistency. These studies hypothesize that news articles with different text and images are more likely to contain misleading information than news articles with consistent text and images. However, while this holds true for some samples, there are instances of news articles containing loosely related images that do not contain misleading information. Nevertheless, intermodal similarity still represents a relationship that can facilitate the fusion of heterogeneous modalities. For example, MCAN (Multimodal Fusion with Co-Attention Networks for Fake News Detection) and Att-RNN (Recurrent neural network with an attention mechanism) utilize attention mechanisms. MCAN uses stacked co-attention blocks to fuse images with textual features in both spatial and frequency domains; the Att-RNN end-to-end rumor detection model automatically fuses extracted multimodal features from images and text using a Long Short-Term Memory (LSTM) network and an attention mechanism to improve the efficiency of detecting news containing misleading information. These studies essentially measure relevance by calculating the cosine similarity between text and image vectors, in order to further integrate different unimodal features and assign greater weight to the parts of text and images with high similarity. Therefore, calculating intermodal similarity is necessary.However, attention-based methods essentially only focus on highly similar parts, ignoring the intermodal inconsistencies mentioned above. Furthermore, heterogeneous pre-training tools for different tasks also limit fusion. In addition, similarity modeling based on global features is often coarse because news samples containing misleading information come from different topics, and news images have a lot of noise in the background; therefore, extracting fine-grained multimodal elements is crucial. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the existing technology, the purpose of this application is to provide a method, apparatus and equipment for detecting news containing misleading information.

[0005] To achieve the above objectives, this application provides a method for detecting news containing misleading information, employing the following technical solution:

[0006] A method for detecting misleading news includes the following steps:

[0007] S1: Obtain a public, real-world social media dataset containing multimodal news content;

[0008] S2: Construct and train a cross-modal content relevance network model to obtain a trained news detection model containing misleading information;

[0009] S3: Input the news data to be detected into the trained news detection model containing misleading information, and use the news detection model containing misleading information to predict whether the news data to be detected contains misleading information.

[0010] Preferably, S2 includes:

[0011] S21: Feature extraction of news text and news images, that is, obtaining text and visual elements and obtaining highly generalizable representations for similarity calculation;

[0012] S22: Cross-modal enhancement, which enhances the shared information between text and visual elements to facilitate similarity calculation;

[0013] S23: Cross-modal correlation, which means obtaining the similarity matrix between text and visual elements and learning the cross-modal correlation features therein;

[0014] S24: Cross-modal fusion, which is to fuse enhanced multimodal features and cross-modal correlation features to obtain a unified representation;

[0015] S25: Classifier, which uses the learned unified representation to predict whether a news story contains misleading information.

[0016] Preferably, news text feature extraction includes: for news text T = {w1, w2, ..., w...} N}, where w represents a word, and N is the text length, i.e., the number of words, is used to extract word embedding features from news tweets using a pre-trained CLIP text encoder, resulting in a word embedding sequence. in

[0017] Preferably, the feature extraction of news images includes: for news image I, using a pre-trained Faster-R-CNN for object detection, and cropping M-1 image slices c based on the output bounding boxes. i This process yields the target slice sequence of news images; and, by inserting the original image at the starting position to preserve overall information, the final target slice sequence O = {1, c1, c2, ..., c...} is obtained. M-1 Then, the target slice sequence is encoded using the CLIP image encoder to obtain the target slice feature sequence. in

[0018] Preferably, S22 includes:

[0019] A cascaded, stacked common attention Transformer is used to enhance the sharing of information between modalities; the structure of the feedforward layer remains the same as the original Transformer, as follows:

[0020]

[0021] Q i =QW i Q K i =KW i K V i =VW i V

[0022]

[0023] in It is a splicing symbol. It is by embedding words into sequence E T Or the image target slice sequence feature E I It is created by integrating elements into a matrix, where L is the length of the multimodal element sequence. As the projection is a trainable parameter matrix, d h =d model / h;

[0024] A fully connected feedforward network consists of two linear transformation layers and a ReLU activation function.

[0025] FFN(X) = max(0, XW1+b1)W2+b2

[0026] in The output of the first residual connection and layer normalization is W1 and W2, which are trainable parameter matrices, and b1 and b2 are bias terms. These cascaded common attention Transformer blocks are stacked T times to output the enhanced word embeddings. and image slice features It retains the same dimensions as the unenhanced features.

[0027] Preferably, S23 includes:

[0028] For enhanced text element features and enhanced visual element features The cosine similarity is obtained by calculating their L2-regularized inner product:

[0029]

[0030]

[0031] Where the cosine similarity matrix S = {s ij} N×M Filters for convolution operations Applied to a window s with a length of h words and a width of N target slices i:i+h-1 To generate a new feature,

[0032] c i =max(0, w·s) i:i+h-1 +b)

[0033] Where b is the bias term, and w continues to slide along the word sequence growth direction with a step size of 1 until N-h+1 features c are generated. i This forms a feature map for the filter. One output channel,

[0034] c = [c1, c2, ..., c N-h+1 ]

[0035] Feature map Perform global average pooling on each feature map, then concatenate them to form cross-modal correlation features.

[0036]

[0037]

[0038] Among them, cross-modal correlation features k is the number of filters of different sizes, and o is the number of output channels for each size of filter;

[0039] Preferably, S24 includes:

[0040] In order to integrate multimodal features and cross-modal correlation features f c Fusion for classification, word sequence features and image target slice features It will be globally averaged pooled, and f c Low-dimensional representations are output through a multilayer perceptron.

[0041] f C =MLP(f c )

[0042] in Each MLP module ends with a discard layer of equal probability; then they are concatenated, and the final representation is output through the MLP.

[0043]

[0044] Preferably, S25 includes:

[0045] After obtaining the final representation x, it is input into a linear transformation layer, and the output is the probability of whether the prediction contains misleading information through the Softmax activation function.

[0046]

[0047] in y0 represents the probability that the prediction is news without misleading information, and y1 represents the probability that the prediction is news containing misleading information; W x It is a trainable parameter matrix, b x It is the bias term; the loss function is defined as minimizing the cross-entropy loss value.

[0048] L(Θ)=-ylog(y1)-(1-y)log(1-y1)

[0049] Where y is the true label, 1 represents news containing misleading information, 0 represents news not containing misleading information, and Θ is all trainable parameters of the model.

[0050] On the other hand, this application also provides a device for detecting news containing misleading information, comprising:

[0051] The acquisition unit is used to acquire a public, real social media dataset, which contains multimodal news content;

[0052] The training unit is used to train a news detection model containing misleading information based on the news content, and to obtain a trained news detection model containing misleading information.

[0053] The prediction unit is used to input the news data to be detected into the trained news detection model containing misleading information, and to predict whether the news data to be detected contains misleading information.

[0054] In another aspect, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the news detection method containing misleading information as described above.

[0055] Compared with existing technologies, this application has the following advantages: The method of this application enhances the detection of misleading news in multimodal contexts by using a novel cross-modal content relevance network, thereby achieving fine-grained extraction of textual and visual content relevance instead of coarse global features. This overcomes the limitations of previous multimodal news detection technologies that use scalar similarity reasoning, and is suitable for training on news classification containing misleading information without requiring additional information or auxiliary tasks, thus improving the accuracy of news detection containing misleading information. Attached Figure Description

[0056] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0057] Figure 1 One of the flowcharts for the news detection method containing misleading information provided in this application;

[0058] Figure 2 The second flowchart of the news detection method containing misleading information provided in this application;

[0059] Figure 3 A structural diagram of the cross-modal enhancement module in the news detection method containing misleading information provided in this application;

[0060] Figure 4 A structural diagram of the cross-modal fusion module in the news detection method containing misleading information provided in this application;

[0061] Figure 5 A structural block diagram of a news detection device containing misleading information provided in this application;

[0062] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0063] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0064] Figure 1 The flowchart of the news detection method containing misleading information provided in this application is as follows: Figure 1 As shown, a method for detecting news containing misleading information includes:

[0065] S1: Obtain a public, real-world social media dataset (news dataset) containing multimodal news content, including news text T and news images I;

[0066] For public, real-world social media datasets, duplicate images are removed from the dataset, and the training and testing data are split to ensure that event themes do not overlap, thus eliminating the influence of event themes. Since this application focuses on detecting both news text and images, samples containing only images or text are removed.

[0067] S2: Construct and train a cross-modal content relevance network model to obtain a trained news detection model containing misleading information. Figure 2 The second flowchart of the news detection method containing misleading information provided in this application includes the following steps for training a cross-modal content relevance network model based on the news content:

[0068] S21: Text and image slice feature extraction, i.e., acquiring text and visual elements and obtaining highly generalized representations for similarity calculation. This acquisition method includes the following steps:

[0069] S211: Text feature extraction, for news tweet text T = {w1, w2, ..., w...} N}, where w represents a word, and N is the text length, i.e., the number of words, is used to extract word embedding features from news tweets using a pre-trained CLIP text encoder, resulting in a word embedding sequence. in S212: Visual feature extraction. For news image I, a pre-trained Faster-R-CNN is used for object detection. Based on the output bounding boxes, M-1 image slices c are cropped. i This process yields the target slice sequence of news images; and, by inserting the original image at the starting position to preserve overall information, the final target slice sequence O = {1, c1, c2, ..., c...} is obtained. M-1Note that to adapt the extracted target region to the news detection task containing misleading information, specific categories, such as semantically irrelevant or small-sized categories like "pants," "eye," "collar," "forehead," and "tie," were manually removed. This is because the purpose of this application is to extract salient region slices of the image and remove the background, while these categories have virtually no corresponding counterparts in the tweet text and would introduce noise into the model. Then, the CLIP image encoder is used to encode the target slice sequence to obtain the target slice feature sequence. in For samples in the dataset containing multiple images, the slices of the multiple images are concatenated into a single sequence.

[0070] S22: Cross-modal enhancement, which enhances the shared information between text and visual elements to facilitate similarity calculation. The module structure is as follows: Figure 3 As shown.

[0071] Specifically, firstly, image slice features are used to enhance word embeddings, that is, word embedding E T As a query, the image slice feature E I As keys and values, the latter undergoes a weighted summation guided by the former. In the residual connection part after the multi-head attention mechanism, this application retains features as the query modality. The output of the multi-head attention mechanism and the query, after residual connection and layer normalization, will serve as keys and values ​​for another common attention Transformer, which in turn enhances the image slice features, rather than being input into two common attention Transformers in parallel. This is where this application differs from other methods that apply common attention Transformers. In fact, at the beginning, the word embedding E T (Query) is first projected onto the image slice feature E I The embedding space of (keys and values) then continues to guide E in turn. I This process will continue to repeat T times. In short, this application uses cascaded, stacked common attention Transformers to enhance intermodal information sharing. The structure of the feedforward layer remains the same as the original Transformer. It is as follows:

[0072]

[0073] Q i =QW i Q K i =KW i K V i =VW i V

[0074]

[0075] in It is a splicing symbol. It is by using text word features E T Or the image target slice sequence feature E I It is created by integrating elements into a matrix, where L is the length of the multimodal element sequence. As the projection is a trainable parameter matrix, d model =512, this application sets h=8, d h =d model / h=64; The fully connected feedforward network consists of two linear transformation layers and a ReLU activation function.

[0076] FFN(X) = max(0, XW1+b1)W2+b2

[0077] in The output of the first residual connection and layer normalization is W1 and W2, which are trainable parameter matrices, and b1 and b2 are bias terms. These cascaded common attention Transformer blocks are stacked T times to output the enhanced word embeddings. and image slice features It maintains the same dimension as the unenhanced feature, and this application sets T=8.

[0078] S23: Cross-modal correlation, which means obtaining the similarity matrix between text and visual elements and learning the cross-modal correlation features within it.

[0079] By calculating the cosine similarity matrix of the mutually enhanced multimodal element features, and performing 1-D convolution on it along the word sequence growth direction, cross-modal correlation features are modeled. The filter movement step size is regarded as the sliding window size. This application considers the similarity between individual words and word groups in the sliding window and all image target slices. The feature maps obtained under filters of multiple sizes and multiple channels are averaged in the spatial dimension. This will capture the average level of similarity between word groups and all visual elements. Then, they are concatenated to obtain cross-modal correlation features.

[0080] Specifically, for enhanced text element features and enhanced visual element features The cosine similarity is obtained by calculating their inner product after L2 regularization.

[0081]

[0082]

[0083] Where the cosine similarity matrix S = {s ij} N×M Filters for convolution operations Applied to a window s with a length of h words and a width of N target slices i:i+h-1 To generate a new feature, there is

[0084] c i =max(0, w·s) i:i+h-1 +b)

[0085] Where b is the bias term, and w continues to slide along the word sequence growth direction with a step size of 1 until N-h+1 features c are generated. i This forms a feature map for the filter. One output channel,

[0086] c = [c1, c2, ..., c N-h+1 ]

[0087] Feature map Perform global average pooling on each feature map, then concatenate them to form cross-modal correlation features.

[0088]

[0089]

[0090] Among them, cross-modal correlation features k is the number of filters of different sizes, and o is the number of output channels for each size of filter.

[0091] f c This method measures fine-grained similarity between textual and visual elements. Filters of different sizes move along the word sequence growth direction on the similarity matrix, helping to capture the relationship between word context and salient region slices in images. Leveraging object detection results with background noise removal and CLIP's multimodal understanding capabilities, the cross-modal correlation modeling method proposed in this application can extract more refined and easily learned similarities.

[0092] The above-described method extracts fine-grained multimodal elements, especially visual elements, from news text and visual content. By simultaneously considering cosine similarity between micro and macro levels and employing 1-D convolution, it directly captures fine-grained, learnable relevance patterns that are beneficial for judging whether news contains misleading information. It considers learnable cross-modal content relevance, which overcomes the limitations of previous multimodal news detection techniques that use scalar similarity inference. It is suitable for training on the classification of news containing misleading information without requiring additional information or auxiliary tasks, and can effectively improve the accuracy of multimodal news detection containing misleading information.

[0093] S24: Cross-modal fusion, which involves fusing enhanced multimodal features and cross-modal correlation features to obtain a unified representation. The module structure is as follows: Figure 4 As shown;

[0094] Until now, multimodal features All are serialized and have high dimensionality because sufficient information needs to be retained when calculating similarity; in order to integrate multimodal features and cross-modal correlation features f c Fusion for classification, word sequence features and image target slice features It will be globally averaged pooled, and f c Low-dimensional representations are output through a multilayer perceptron.

[0095] f C =MLP(f c )

[0096] in Each MLP module ends with a drop-out layer of equal probability. These layers are then concatenated, and the final representation is output through the MLP.

[0097]

[0098] S25: Classifier, which uses the learned unified representation to predict whether the news is false.

[0099] After obtaining the final representation x, it is input into a linear transformation layer, and the output is the probability of whether the prediction contains misleading information through the Softmax activation function.

[0100]

[0101] in y0 represents the probability that the predicted news is free of misleading information, and y1 represents the probability that the predicted news is free of misleading information. W x It is a trainable parameter matrix, b x This is the bias term. The loss function is defined as minimizing the cross-entropy loss value.

[0102] L(Θ)=-ylog(y1)-(1-y)log(1-y1)

[0103] Where y is the true label, 1 represents news containing misleading information, 0 represents news not containing misleading information, and Θ is all trainable parameters of the model.

[0104] S3: Input the news data to be detected into the trained news detection model containing misleading information, and use the model to predict whether the news data to be detected contains misleading information.

[0105] Specifically, in a preferred embodiment of this application, the news detection method containing misleading information disclosed in this application is based on CLIP, attention mechanism and convolutional neural network. It designs a deep learning model to extract, align, compare and fuse multimodal content features, which enhances the expressive power of news representation vectors and overcomes the problem of low detection accuracy caused by the heterogeneous semantic gap between modalities brought about by pre-trained models for different tasks. It realizes the accurate identification of multimodal news containing misleading information in social networks.

[0106] The following verification is based on the embodiments of this application.

[0107] Table 1 presents the statistical information for the Weibo and Twitter datasets. The Weibo dataset, released by Jin et al., is a Chinese-language dataset. It collected posts from May 2012 to January 2016. News containing misleading information (fake news) was collected from Weibo's official debunking system, while news without misleading information (real news) was obtained from tweets by the official media Xinhua News Agency. The Twitter dataset, released for a multimedia usage verification task, is primarily English-language. Relevant tweets were searched on Twitter based on keywords from 17 events, and manually annotated based on online resources.

[0108] Table 1

[0109]

[0110] Commonly used metrics are employed to evaluate model performance, including accuracy, precision, recall, and F1 score. Figure 5 This is a comparison chart of the accuracy, precision, recall, and F1 score of the method (C3N) in this application with other methods for detecting whether a news item contains misleading information; Figure 5This paper presents detailed results on two real-world datasets, comparing the accuracy, precision, recall, and F1 score of the method provided in this application with those of seven other methods for detecting news containing and not containing misleading information. Table 2 shows the results. Among these, SAFE (Similarity-Aware Multi-Modal Fake News Detection), EANN (Event adversarial neural networks), MVAE (Multimodal Variational Autoencoder for Fake News Detection), Spotfake (A Multi-modal Framework for Fake News Detection), MCAN (Multimodal Fusion with Co-Attention Networks for Fake News Detection), CAFE (Cross-modal Ambiguity Learning for Multimodal Fake News Detection), and MRML (Multimodal Rumor Detection by Deep Metric Learning) are all authoritative and widely recognized methods in the field of news detection containing misleading information, while C3N is the method proposed in this application. As shown in Table 2, the model outperforms all baselines on both datasets, achieving accuracies of 94.9% and 81.9%, respectively. It can be seen that models focusing on cross-modal semantic interaction perform better; for example, MCAN uses a stacked common attention Transformer to aggregate textual, spatial, and frequency features. However, its interaction is based on global feature expansion, ignoring the role of fine-grained multimodal content. CAFE outperforms SAFE by using adaptive feature fusion through cross-modal ambiguity, but still, like other methods, only uses global features. This application achieves best performance by using enhanced cross-modal content relevance. Overall, compared to all baselines, the proposed news detection method based on cross-modal content relevance for misleading information performs best on both datasets.

[0111] Table 2 compares the accuracy, precision, recall, and F1 score of the method in this application with other methods for detecting whether news contains misleading information.

[0112] Table 2

[0113]

[0114] The news detection method for misleading information based on cross-modal content relevance provided in this application extracts salient image slices to form fine-grained visual content, reducing noise from non-salient information and facilitating cross-modal semantic interaction. By applying CLIP and using a cross-modal enhancement module to deeply integrate text and visual content, the heterogeneity and semantic gap of multimodal features are reduced. Moreover, by performing 1-D convolution on the similarity matrix of multimodal content, it can capture fine-grained, learnable, and complex intermodal relevance patterns, thereby improving the accuracy of the model in detecting news containing misleading information.

[0115] The following describes the news detection device containing misleading information provided in this application. The news detection device containing misleading information and the news detection method containing misleading information can be referred to each other accordingly.

[0116] In one embodiment, a detection device is provided. Figure 5 The structural block diagram of the news detection device containing misleading information provided in this application is as follows: Figure 5 As shown, it includes: an acquisition unit 501, used to acquire a public real social media dataset, the public real social media dataset having multimodal news content, including news text T and news images I;

[0117] Training unit 502 is used to train a news detection model containing misleading information based on the news content, and obtain a trained news detection model containing misleading information.

[0118] The prediction unit 503 is used to input the news data to be detected into the trained news detection model containing misleading information, and to predict whether the news data contains misleading information. Furthermore, it can be integrated with a dialogue system to detect and prevent the spread of news containing misleading information in real time during dialogue.

[0119] The detection device is capable of implementing the detection method of this application.

[0120] In one embodiment, an electronic device is provided. Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a news detection method containing misleading information, the method including:

[0121] S1: Obtain a public, real-world social media dataset containing multimodal news content, including news text T and news images I;

[0122] S2: Construct and train a cross-modal content relevance network model to obtain a trained news detection model containing misleading information;

[0123] S3: Input the news data to be detected into the trained news detection model containing misleading information, and use the model to predict whether the news data to be detected contains misleading information.

[0124] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] The units described for the separation of components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, or of course, by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

Claims

1. A method of detecting news containing misleading information, characterized by, The method comprises the following steps: S1: obtaining a public real social media dataset, the public real social media dataset having multi-modal news content; S2: constructing a cross-modal content correlation network model and training to obtain a trained misleading information-containing news detection model, the S2 comprising: S21: news text and news picture feature extraction, that is, obtaining text and visual elements and obtaining a high generalization representation for similarity calculation; S22: cross-modal enhancement, that is, enhancing shared information between text and visual elements to facilitate similarity calculation, the S22 comprising: adopting a form of cascaded, stacked common attention Transformer to enhance shared information between modalities; a structure of a feedforward layer remains the same as that of an original Transformer, and the following is included: ， wherein is a concatenation symbol, is created by concatenating the word embedding sequence or the image target slice sequence feature into a matrix, using the picture slice features to augment the word embeddings, i.e. the word embeddings are queried, the picture slice features are used as keys and values, the latter being weighted summed under the guidance of the former, the word embeddings are first projected into the embedding space of the picture slice features and then the other way around continues to guide the adaptation, such a flow will continue for T times, is the length of the sequence of multi-modal elements, is a trainable parameter matrix as projection, ; a fully connected feedforward network is composed of two linear transformations and a ReLU activation function, ， wherein , W1, W2 are trainable parameter matrices, b1, b2 are bias terms; such a concatenated co-attention Transformer block is stacked T times, outputting an enhanced word embedding and picture slice features , which maintain the same dimension as the unenhanced features; S23: cross-modal correlation, that is, obtaining a similarity matrix of text and visual elements and learning cross-modal correlation features therein, the S23 comprising: For enhanced post-textual element features and enhanced post-visual element features , cosine similarities are computed by taking the regularized inner products of them ​ ， where the cosine similarity matrix ; filter of convolution operation applied to a window of h word length, N target slice width to produce a new feature, ， wherein is a bias term, Continuing the sliding by steps of 1 in the direction of the word sequence growth until N−h+1 features are produced form a feature map for one output channel of the filter ​ ， feature map Global average pooling is performed on each feature map, and the cross-modal correlation features are concatenated. ， wherein the cross-modal correlation features k is the number of different sizes of filters, and o is the number of output channels for each size of filter. S24: cross-modal fusion, that is, fusing enhanced multi-modal features and cross-modal correlation features to obtain a unified representation, the S24 comprising: In order to fuse the multi-modal features and cross-modal correlation features for classification, word sequence features and picture object slice features will be globally average-pooled, and output low-dimensional representations respectively by multi-layer perceptron ， wherein Each MLP module is followed by a dropout layer with equal probability, and the outputs of the MLP modules are concatenated and passed through a final MLP to produce the final representation. ， S25: a classifier, that is, using the learned unified representation to predict whether the news is misleading information-containing news; S3: inputting to-be-detected news data into the trained misleading information-containing news detection model, and predicting whether the to-be-detected news data contains misleading information through the misleading information-containing news detection model.

2. The misinformation-containing news detection method of claim 1, wherein, The news text feature extraction includes: for the news text , representative words, N is the length of the text, that is, the number of words, word embedding features of the news tweet are extracted using a pre-trained CLIP text encoder, and a word embedding sequence , is obtained.

3. The misinformation-containing news detection method of claim 1, wherein, The news picture feature extraction comprises: for a news picture I, using a pre-trained Faster-R-CNN to perform target detection, and cutting M-1 picture slices according to the output boundary box to obtain a target slice sequence of the news picture; and inserting the original picture at a starting position to retain overall information to obtain a final target slice sequence; then using a picture encoder of CLIP to encode the target slice sequence to obtain a target slice feature sequence, wherein . . . .

4. The misinformation-containing news detection method of claim 1, wherein, The S25 comprises: obtaining a final representation After that, it is input into a linear transformation layer, and the probability of predicting whether it contains misleading information is output through a Softmax activation function. ， wherein , denotes the probability that the prediction is a misleading information news-free, denotes the probability that the prediction is a misleading information news-containing; is a trainable parameter matrix, is a bias term; the loss function is defined to minimize the cross-entropy loss value, ， wherein is the true label, 1 indicates news containing misleading information, and 0 indicates news not containing misleading information, are all the trainable parameters of the model.

5. A device for detecting news containing misleading information for implementing the method according to any one of claims 1 to 4, characterized in that, comprising: an acquisition unit configured to acquire a public real social media dataset, the social media dataset having multi-modal news content; a training unit configured to train a misleading information-containing news detection model according to the news content to obtain a trained misleading information-containing news detection model; a prediction unit configured to input to-be-detected news data into the trained misleading information-containing news detection model and predict whether the to-be-detected news data contains misleading information through the model.

6. An electronic device, comprising: A computer program product comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the misleading information-containing news detection method according to any one of claims 1-4 when executing the program.

Images