False news detection method

Abstract

The invention relates to the technical field of information detection, in particular to a false news detection method, which comprises the step of carrying out detection through a false news detection model which comprises an input layer, a language feature extraction layer, a text feature extraction layer, a picture feature extraction layer, a feature fusion layer and a detection layer. Aiming at the problem that text and picture features are difficult to extract by an existing model, language features are extracted through LIWC, text features are extracted through BERT and BiGRU, the expression ability of the model for texts is improved, deep visual features of pictures are extracted through ResNet50, picture sequence information is processed through BiGRU, visual and language information is integrated through BLIP, and picture content is comprehensively understood; in order to solve the problem that a multi-modal feature fusion method is difficult to fully learn a relationship between modals, a cross-modal attention mechanism is utilized to learn complementary information among language features, text features and picture features, and effective fusion of multi-modal data is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of information detection, and in particular to a false news detection method. Background Art

[0002] Fake news refers to stories that contain false information, misleading content, or are completely fictitious, and are often spread as real news. With the booming development of the Internet and social media, the proliferation of fake news has become a serious problem. If fake news cannot be effectively identified and controlled in the initial stage, it will mislead the public. Therefore, it is particularly important to develop an efficient fake news detection method. However, existing fake news detection methods have the problem that existing models are difficult to extract features of text and images; there is also the problem that multimodal feature fusion methods often use simple splicing operations, which makes it difficult to fully learn the relationship between modalities. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a method for detecting fake news. In view of the problem that the existing model is difficult to extract the features of text and pictures, the present invention extracts the language features of the text through the LIWC analyzer, and uses the BERT model and the BiGRU model to extract the text features, thereby improving the model's ability to express text features. In addition, the present invention uses the ResNet50 model to extract the deep visual features of the picture, the BiGRU model to process the sequence information of the picture, and the BLIP model to integrate the visual and language information to more comprehensively understand the content of the picture; in view of the problem that the multimodal feature fusion method often uses a simple splicing operation and is difficult to fully learn the relationship between modalities, the present invention uses the cross-modal attention mechanism to learn the complementary information between language features, text features and picture features, and promotes the effective fusion between multimodal data, thereby improving the model's ability to process and understand information.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides a method for detecting fake news, the method comprising the following steps:

[0005] S1: Constructing a fake news detection model, the fake news detection model comprising: an input layer, a language feature extraction layer, a text feature extraction layer, an image feature extraction layer, a feature fusion layer and a detection layer;

[0006] S2: The input layer receives news and decomposes it into text and pictures;

[0007] S3: The language feature extraction layer extracts language features from the text to obtain a language latent state vector. The text feature extraction layer extracts features from the text to obtain a text latent state vector. The image feature extraction layer extracts features from the image to obtain an image latent state vector.

[0008] S4: Put the language latent state vector, text latent state vector and image latent state vector into the feature fusion layer, use the cross-modal attention mechanism to learn the fusion of the language latent state vector, text latent state vector and image latent state vector, and obtain the output vector of the feature fusion layer;

[0009] S5: The output vector of the feature fusion layer is input into the detection layer to detect the authenticity of the news.

[0010] The language feature extraction layer uses the LIWC analyzer to extract the language features of the text, including emotions, behaviors, positive tone, negative tone, affirmative tone, interrogative tone, causal relationship, network language, and meaningless auxiliary words, and normalizes them to obtain the language hidden state vector .

[0011] The process of extracting features from text in the text feature extraction layer includes the following steps:

[0012] D1: The text feature extraction layer uses the BERT model to extract text features from the text and obtain the BERT hidden state vector. The specific formula is as follows:

[0013] ;

[0014] in, Represents the input text, Represents the attention mechanism operation of the BERT model, Represents the BERT hidden state vector;

[0015] D2: Furthermore, the text feature extraction layer uses the BiGRU model to further extract time information from the BERT hidden state vector and convert the BERT hidden state vector into a text hidden state vector. The specific formula is as follows:

[0016] ;

[0017] in, Represents the operation of the BiGRU model, Represents the text hidden state vector.

[0018] The process of extracting features from an image by the image feature extraction layer includes the following steps:

[0019] K1: The image feature extraction layer uses a 1024-dimensional fully connected layer to process the image and obtain the image encoding;

[0020] K2: Furthermore, the image feature extraction layer uses the ResNet50 model to extract features from the image encoding and obtain the ResNet50 hidden state vector. The specific formula is as follows:

[0021] ;

[0022] in, Indicates the picture encoding. express Model operation, Represents the ResNet50 hidden state vector;

[0023] K3: Furthermore, the image feature extraction layer uses the BiGRU model to extract features from the ResNet50 hidden state vector and converts the ResNet50 hidden state vector into the BiGRU hidden state vector. The specific formula is as follows:

[0024] ;

[0025] in, Represents the operation of the BiGRU model, Represents the BiGRU hidden state vector;

[0026] K4: Further, the image feature extraction layer uses the BLIP model combined with the text to extract the image features and obtain the BLIP hidden state vector. The specific formula is as follows:

[0027] ;

[0028] in, Represents the operation of the BLIP model, represents the BLIP hidden state vector;

[0029] K5: Furthermore, the image feature extraction layer performs linear transformation on the BiGRU hidden state vector and the BLIP hidden state vector through a fully connected layer, and then the results of the linear transformation are fused through the connection layer to obtain the image hidden state vector. The specific formula is as follows:

[0030] ;

[0031] in, represents the hidden state vector of the image, represents the linear operation of the fully connected layer, Represents the connection operation of the connection layer.

[0032] In step S4, the specific contents are as follows:

[0033] The feature fusion layer generates the query vector corresponding to the language hidden state vector through linear transformation , key vector Sum value vector .

[0034] The feature fusion layer generates the query vector corresponding to the text hidden state vector through linear transformation , key vector Sum value vector .

[0035] The feature fusion layer generates the query vector corresponding to the hidden state vector of the image through linear transformation , key vector Sum value vector .

[0036] Furthermore, the feature fusion layer constructs a language multimodal attention module, specifically using the MultimodalTransformer model The cross-modal attention layer learns the language hidden state vector and uses the query vector , key vector Sum value vector Calculate the cross-modal attention, the specific formula is as follows:

[0037] ;

[0038] in, represents the calculation result of the cross-modal attention on the language hidden state vector, represents the softmax function, Represents the dimension of the language hidden state vector.

[0039] Furthermore, the feature fusion layer constructs a text multimodal attention module, which specifically uses the MultimodalTransformer model The cross-modal attention layer learns the text hidden state vector and uses the query vector , key vector Sum value vector Calculate the cross-modal attention, the specific formula is as follows:

[0040] ;

[0041] in, represents the calculation result of the cross-modal attention on the text hidden state vector, Represents the dimension of the text hidden state vector.

[0042] Furthermore, the feature fusion layer constructs a multimodal attention module for images, specifically using the The cross-modal attention layer learns the hidden state vector of the image and uses the query vector , key vector Sum value vector Calculate the cross-modal attention, the specific formula is as follows:

[0043] ;

[0044] in, represents the calculation result of the cross-modal attention on the hidden state vector of the image, Represents the dimension of the hidden state vector of the image.

[0045] Furthermore, the dimension of the language hidden state vector, the dimension of the text hidden state vector, and the dimension of the image hidden state vector are equal.

[0046] Furthermore, in the language multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated, and finally a fully connected layer is used for linear transformation to obtain the language output vector. The specific formula is as follows:

[0047] ;

[0048] in, represents the language output vector, Represents the first The computation result of the cross-modal attention layer, and , Represents the weight matrix in the language multimodal attention module.

[0049] Furthermore, in the text multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated, and finally a fully connected layer is used for linear transformation to obtain the text output vector. The specific formula is as follows:

[0050] ;

[0051] in, represents the text output vector, Represents the first The computation result of the cross-modal attention layer, and , Represents the weight matrix in the text multimodal attention module.

[0052] Furthermore, in the image multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated, and finally a linear transformation is performed through a fully connected layer to obtain the image output vector. The specific formula is as follows:

[0053] ;

[0054] in, Represents the image output vector, Represents the first image in the multimodal attention module The computation result of the cross-modal attention layer, and , Represents the weight matrix in the image multimodal attention module.

[0055] Furthermore, the feature fusion layer connects and calculates the language output vector, the text output vector, and the image output vector to obtain the feature fusion layer output vector. The specific formula is as follows:

[0056] ;

[0057] in, Represents the feature fusion layer output vector.

[0058] In step S5, the detection layer includes a binary classifier, which detects the authenticity of the news based on the output vector of the feature fusion layer and uses binary cross entropy as the loss function. The specific formula is as follows:

[0059] ;

[0060] in, Represents the calculation result of the loss function, represents the number of samples, represents the true value of each sample, Represents the predicted value of the binary classifier for each sample.

[0061] The beneficial results achieved by the present invention using the above scheme are as follows:

[0062] (1) In view of the problem that existing models are difficult to extract the features of text and images, the present invention extracts the language features of text through the LIWC analyzer, and uses the BERT model and the BiGRU model to extract text features, thereby improving the model's ability to express text features; in addition, the present invention uses the ResNet50 model to extract the deep visual features of the image, the BiGRU model to process the sequence information of the image, and the BLIP model to integrate visual and language information, so as to more comprehensively understand the image content.

[0063] (2) In view of the problem that multimodal feature fusion methods often use simple splicing operations and find it difficult to fully learn the relationship between modalities, the present invention uses a cross-modal attention mechanism to learn the complementary information between language features, text features, and image features, thereby promoting the effective fusion of multimodal data and improving the model's ability to process and understand information. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a diagram of the fake news detection model framework.

[0065] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] Example 1, see Figure 1 , Figure 1 : is a framework diagram of a fake news detection model. In this embodiment, the method specifically includes the following steps:

[0068] S1: Constructing a fake news detection model, the fake news detection model comprising: an input layer, a language feature extraction layer, a text feature extraction layer, an image feature extraction layer, a feature fusion layer and a detection layer;

[0069] S2: The input layer receives news and decomposes it into text and pictures;

[0070] S3: The language feature extraction layer extracts language features from the text to obtain a language latent state vector. The text feature extraction layer extracts features from the text to obtain a text latent state vector. The image feature extraction layer extracts features from the image to obtain an image latent state vector.

[0071] S4: Put the language latent state vector, text latent state vector and image latent state vector into the feature fusion layer, use the cross-modal attention mechanism to learn the fusion of the language latent state vector, text latent state vector and image latent state vector, and obtain the output vector of the feature fusion layer;

[0072] S5: The output vector of the feature fusion layer is input into the detection layer to detect the authenticity of the news.

[0073] In this embodiment, Adam is used as the optimizer, the learning rate is set to 0.0001, and the batch size is set to 24.

[0074] In this embodiment, the MCG-FNeWS dataset is selected when training the fake news detection model. The dataset covers 19,186 true news and 19,258 false news published on Weibo between May 2012 and November 2018. The dataset is divided into a training set and a test set in a ratio of 7:3, and the model is trained and tested respectively.

[0075] The development environment of this embodiment includes: using Windows 11 operating system, using Python 3.9 language, developing code on the PyCharm platform, and writing deep learning related codes with the help of PyTorch tool.

[0076] Embodiment 2: Based on the above embodiment, this embodiment realizes the processing of news by the input layer. The specific contents are as follows:

[0077] The input layer receives news and decomposes it into text and pictures. The maximum length of the text is set to 200, the excess part is truncated, and the insufficient part is filled with zeros; the dimension of the picture is set to 224×224.

[0078] Embodiment 3: Based on the above embodiment, this embodiment realizes the extraction of language features. The specific contents are as follows:

[0079] The language feature extraction layer uses the LIWC analyzer to extract the language features of the text, including emotions, behaviors, positive tone, negative tone, affirmative tone, interrogative tone, causal relationship, network language, and meaningless auxiliary words, and normalizes them to obtain the language hidden state vector .

[0080] Embodiment 4: Based on the above embodiment, this embodiment realizes the extraction of text features. The specific contents are as follows:

[0081] The process of extracting features from text in the text feature extraction layer includes the following steps:

[0082] D1: The text feature extraction layer uses the BERT model to extract text features from the text, and uses the Tokenizer of the BERT model to serialize the text to obtain the BERT hidden state vector. The specific formula is as follows:

[0083] ;

[0084] in, Represents the input text, Represents the attention mechanism operation of the BERT model, Represents the BERT hidden state vector;

[0085] D2: The text feature extraction layer uses the BiGRU model to further extract time information from the BERT hidden state vector and convert the BERT hidden state vector into a text hidden state vector. In this embodiment, the number of layers of the BiGRU model is set to 2. The specific formula is as follows:

[0086] ;

[0087] in, Represents the operation of the BiGRU model, Represents the text hidden state vector.

[0088] Embodiment 5: Based on the above embodiment, this embodiment realizes the extraction of image features. The specific contents are as follows:

[0089] The process of extracting features from an image by the image feature extraction layer includes the following steps:

[0090] K1: The image feature extraction layer uses a 1024-dimensional fully connected layer to process the image and obtain the image encoding;

[0091] K2: The image feature extraction layer uses the ResNet50 model to extract features from the image encoding and obtain the ResNet50 hidden state vector. The specific formula is as follows:

[0092] ;

[0093] in, Indicates the picture encoding. express Model operation, Represents the ResNet50 hidden state vector;

[0094] K3: The image feature extraction layer uses the BiGRU model to extract features from the ResNet50 hidden state vector and converts the ResNet50 hidden state vector into a BiGRU hidden state vector. In this embodiment, the number of layers of the BiGRU model is set to 1. The specific formula is as follows:

[0095] ;

[0096] in, Represents the operation of the BiGRU model, Represents the BiGRU hidden state vector;

[0097] K4: The image feature extraction layer uses the BLIP model combined with text to extract image features and obtain the BLIP hidden state vector. The specific formula is as follows:

[0098] ;

[0099] in, Represents the operation of the BLIP model, represents the BLIP hidden state vector;

[0100] K5: The image feature extraction layer performs linear transformation on the BiGRU hidden state vector and the BLIP hidden state vector through a fully connected layer respectively, and then realizes feature fusion of the results of the linear transformation through the connection layer to obtain the image hidden state vector. The specific formula is as follows:

[0101] ;

[0102] in, represents the hidden state vector of the image, represents the linear operation of the fully connected layer, Represents the connection operation of the connection layer.

[0103] Embodiment 6: Based on the above embodiment, this embodiment realizes the cross-fusion of multiple features. The specific contents are as follows:

[0104] The feature fusion layer generates the query vector corresponding to the language hidden state vector through linear transformation , key vector Sum value vector .

[0105] The feature fusion layer generates the query vector corresponding to the text hidden state vector through linear transformation , key vector Sum value vector .

[0106] The feature fusion layer generates the query vector corresponding to the hidden state vector of the image through linear transformation , key vector Sum value vector .

[0107] The feature fusion layer constructs a language multimodal attention module, which is specifically based on the Multimodal Transformer model. The cross-modal attention layer learns the language hidden state vector and uses the query vector , key vector Sum value vector Calculate cross-modal attention. In this embodiment Set to 12, the specific formula is as follows:

[0108] ;

[0109] in, represents the calculation result of the cross-modal attention on the language hidden state vector, represents the softmax function, Represents the dimension of the language hidden state vector.

[0110] The feature fusion layer constructs a text multimodal attention module, which is specifically based on the Multimodal Transformer model. The cross-modal attention layer learns the text hidden state vector and uses the query vector , key vector Sum value vector Calculate cross-modal attention. In this embodiment Set to 12, the specific formula is as follows:

[0111] ;

[0112] in, represents the calculation result of the cross-modal attention on the text hidden state vector, Represents the dimension of the text hidden state vector.

[0113] The feature fusion layer constructs the image multimodal attention module, which is specifically based on the Multimodal Transformer model. The cross-modal attention layer learns the hidden state vector of the image and uses the query vector , key vector Sum value vector Calculate cross-modal attention. In this embodiment Set to 12, the specific formula is as follows:

[0114] ;

[0115] in, represents the calculation result of the cross-modal attention on the hidden state vector of the image, Represents the dimension of the hidden state vector of the image.

[0116] The dimension of the language hidden state vector, the dimension of the text hidden state vector, and the dimension of the image hidden state vector are equal.

[0117] In the language multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated. Finally, a fully connected layer is used for linear transformation to obtain the language output vector. The specific formula is as follows:

[0118] ;

[0119] in, represents the language output vector, Represents the first The computation result of the cross-modal attention layer, and , Represents the weight matrix in the language multimodal attention module.

[0120] In the text multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated. Finally, a fully connected layer is used for linear transformation to obtain the text output vector. The specific formula is as follows:

[0121] ;

[0122] in, represents the text output vector, Represents the first The computation result of the cross-modal attention layer, and , Represents the weight matrix in the text multimodal attention module.

[0123] In the image multimodal attention module, each cross-modal attention layer is operated in parallel, and the calculation results of each cross-modal attention layer are connected and calculated. Finally, a linear transformation is performed through a fully connected layer to obtain the image output vector. The specific formula is as follows:

[0124] ;

[0125] in, Represents the image output vector, Represents the first image in the multimodal attention module The computation result of the cross-modal attention layer, and , Represents the weight matrix in the image multimodal attention module.

[0126] The feature fusion layer connects and calculates the language output vector, text output vector, and image output vector to obtain the feature fusion layer output vector. The specific formula is as follows:

[0127] ;

[0128] in, Represents the feature fusion layer output vector.

[0129] Embodiment 7: Based on the above embodiment, this embodiment realizes the detection of news. The specific contents are as follows:

[0130] In step S5, the detection layer includes a binary classifier, which detects the authenticity of the news based on the output vector of the feature fusion layer and uses binary cross entropy as the loss function. The specific formula is as follows:

[0131] ;

[0132] in, Represents the calculation result of the loss function, represents the number of samples, represents the true value of each sample, Represents the predicted value of the binary classifier for each sample.

Claims

1. A method for detecting fake news, characterized in that: The method comprises the following steps: S1: Constructing a fake news detection model, the fake news detection model comprising: an input layer, a language feature extraction layer, a text feature extraction layer, an image feature extraction layer, a feature fusion layer and a detection layer; S2: The input layer receives news and decomposes it into text and pictures; S3: The language feature extraction layer extracts language features from the text to obtain a language latent state vector. The text feature extraction layer extracts features from the text to obtain a text latent state vector. The image feature extraction layer extracts features from the image to obtain an image latent state vector. S4: Put the language latent state vector, text latent state vector and image latent state vector into the feature fusion layer, use the cross-modal attention mechanism to learn the fusion of the language latent state vector, text latent state vector and image latent state vector, and obtain the output vector of the feature fusion layer; S5: The output vector of the feature fusion layer is input into the detection layer to detect the authenticity of the news.

2. A false news detection method according to claim 1, characterized in that: The language feature extraction layer uses the LIWC analyzer to extract language features from the text and performs normalization processing to obtain a language latent state vector.

3. A false news detection method according to claim 1, characterized in that: The process of extracting features from text in the text feature extraction layer includes the following steps: D1: Use the BERT model to extract features from the text and obtain the BERT hidden state vector; D2: Use the BiGRU model to further extract sequence features from the BERT hidden state vector and convert the BERT hidden state vector into a text hidden state vector.

4. A false news detection method according to claim 1, characterized in that: The process of extracting features from an image by the image feature extraction layer includes the following steps: K1: Use a 1024-dimensional fully connected layer to process the image and obtain the image encoding; K2: Use the ResNet50 model to extract features from the image code and obtain the ResNet50 hidden state vector; K3: Use the BiGRU model to extract sequence features from the ResNet50 hidden state vector and convert the ResNet50 hidden state vector into a BiGRU hidden state vector; K4: Use the BLIP model combined with text to extract image features and obtain the BLIP hidden state vector; K5: Perform linear transformation on the BiGRU hidden state vector and the BLIP hidden state vector respectively, and then concatenate the results of the linear transformation to obtain the image hidden state vector.

5. A false news detection method according to claim 1, characterized in that: In step S4, the feature fusion layer generates a query vector corresponding to the language hidden state vector by linear transformation. , key vector Sum value vector ; In step S4, the feature fusion layer generates a query vector corresponding to the text hidden state vector by linear transformation , key vector Sum value vector ; In step S4, the feature fusion layer generates a query vector corresponding to the image hidden state vector by linear transformation , key vector Sum value vector ; In step S4, the feature fusion layer constructs a language multimodal attention module, specifically using the MultimodalTransformer model The cross-modal attention layer learns the language hidden state vector and uses the query vector , key vector Sum value vector Compute cross-modal attention; In step S4, the feature fusion layer constructs a text multimodal attention module, specifically using the MultimodalTransformer model The cross-modal attention layer learns the text hidden state vector and uses the query vector , key vector Sum value vector Compute cross-modal attention; In step S4, the feature fusion layer constructs a multimodal attention module for the image, specifically using the MultimodalTransformer model The cross-modal attention layer learns the hidden state vector of the image and uses the query vector , key vector Sum value vector Compute cross-modal attention.

6. A false news detection method according to claim 5, characterized in that: In step S4, the language multimodal attention module performs parallel calculations on each cross-modal attention layer, then connects the calculation results of each cross-modal attention layer, and finally obtains the language output vector through linear transformation; In step S4, each cross-modal attention layer is calculated in parallel in the text multimodal attention module, and then the calculation results of each cross-modal attention layer are connected and calculated, and finally a text output vector is obtained through linear transformation; In step S4, each cross-modal attention layer in the image multimodal attention module is operated in parallel, and then the calculation results of each cross-modal attention layer are connected and calculated, and finally the image output vector is obtained through linear transformation.

7. A false news detection method according to claim 6, characterized in that: In step S4, the feature fusion layer connects and calculates the language output vector, the text output vector and the image output vector to obtain the feature fusion layer output vector.

8. A false news detection method according to claim 1, characterized in that: The detection layer includes a binary classifier, which detects the authenticity of the news based on the output vector of the feature fusion layer.

Images