A method and device for identifying fake news

Abstract

The application discloses a false news identification method and device, and belongs to the field of deep learning and data mining. The method can solve the problem of low accuracy of existing false news judgment. The method comprises the following steps: determining text judgment factors and image judgment factors corresponding to text feature sequences and image feature sequences respectively through a self-attention mechanism; determining a text-image sequence and an image-text sequence through an attention mechanism, the text feature sequences and the image feature sequences; obtaining a modal fusion judgment factor according to the text-image sequence and the image-text sequence; determining prediction results corresponding to the text judgment factors, the image judgment factors and the modal fusion judgment factor through a loss function; and determining the authenticity of a news data set corresponding to the training set according to the prediction results.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and data mining, and more specifically relates to a method and apparatus for identifying fake news. Background Technology

[0002] In recent years, online websites and self-media platforms have proliferated as a means of disseminating real-time information. However, this has also led to the rampant spread of various forms of fake news. News refers to the latest or most noteworthy information about events that have already occurred. Fake news, on the other hand, refers to a form of public opinion that uses false information to deceive those involved in order to achieve a specific purpose. It fails to reflect the true nature of objective facts and contains false elements in its reporting.

[0003] To prevent the spread of fake news, there are generally two methods. One is to review accounts through network behavior analysis. This involves analyzing an account's past behavior, social network relationships, and information forwarding patterns to detect key accounts and identify and eliminate sources and dissemination points that create and spread fake news. The other is to monitor published news content and use techniques such as entity alignment, association reasoning, and feature extraction to mine information contained in the specific content of the news (text and images) to determine the authenticity of the news and decide whether to allow its publication.

[0004] Various methods for detecting critical accounts include clustering, random walks, step analysis, and entropy minimization discretization. Researchers have used these methods to design various mechanisms that can be used to detect such accounts and generate early warnings. However, most of these methods heavily rely on textual and user metadata features.

[0005] Later researchers focused on studying writing styles, social connections, and user behavior to detect fake news. To improve factual analysis of news content, researchers began using knowledge graphs; entity relationship information extracted from knowledge graphs can be used for common-sense reasoning about text content. However, with the increasing sophistication of image forgery technology, forging images has become a popular way to create fake news. To address this, researchers use methods such as generative adversarial networks, frequency domain feature detection, and ELA algorithms to detect forged images.

[0006] While all the aforementioned unimodal techniques can provide relatively good results, the task of fake news detection has always faced a significant challenge: an increasing number of news publishers are using both text and images to publish news. To overcome the limitations of unimodal detection methods, researchers have begun using multimodal approaches, attempting to extract features from multimodal data (i.e., text and images) and fuse them together to achieve better fake news detection results.

[0007] Some researchers use image captioning models to generate text descriptions for images in news articles, and then concatenate this text with the news text. This transforms a multimodal problem into a unimodal (text) problem. Based on this, a text-based fake news classifier can be trained to detect fake news. However, image captioning models are currently not mature enough, and can only generate text summarizing the main content of the image. The quality of the generated text limits the performance of such methods.

[0008] Some researchers extract text features and image features separately, then simply concatenate them, and finally use a classifier to perform a simple binary classification on the concatenated features. This type of method uses features from two modalities simultaneously, and the detection effect is indeed better than single-modal detection methods. However, this simple concatenation of two features fails to achieve cross-modal interaction and enhancement of features.

[0009] Some researchers use separately extracted text and image features to perform multi-layer fusion, and then classify the fusion results. This method simulates the information reception process when humans read news and can effectively promote the fusion of text and image features. However, this method loses the separate consideration of text and image features and cannot fully explore the information contained in the text and image features. Summary of the Invention

[0010] This invention provides a method and apparatus for identifying fake news, which can solve the problem of low accuracy in existing methods for judging fake news.

[0011] This invention provides a method for identifying fake news, including:

[0012] Text features of news files included in the training set are extracted using the BERT model to obtain a text feature sequence. Image features of the training set and included news images are extracted using the ResNet50 model to obtain an image feature sequence.

[0013] The text judgment factor and image judgment factor corresponding to the text feature sequence and the image feature sequence are determined sequentially by a self-attention mechanism.

[0014] The text-image sequence and the image-text sequence are determined sequentially through the attention mechanism, the text feature sequence, and the image feature sequence, and the modality fusion judgment factor is obtained based on the text-image sequence and the image-text sequence.

[0015] The prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor are determined sequentially by the loss function, and the authenticity of the news dataset corresponding to the training set is determined based on the prediction results.

[0016] Preferably, the step of sequentially determining the text judgment factor and image judgment factor corresponding to the text feature sequence and the image feature sequence respectively through a self-attention mechanism specifically includes:

[0017] Using the text feature sequence as input to the self-attention mechanism, the text judgment factor is obtained through the following formula:

[0018]

[0019] Using the image feature sequence as input to the self-attention mechanism, the image judgment factor is obtained through the following formula:

[0020]

[0021] in, Indicates the text judgment factor. Represents a text feature sequence. Represents the image judgment factor. Represents the image feature sequence. It is a trainable parameter matrix, where d represents the input of text features e t Or image features e i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. Or to The transpose operation.

[0022] Preferably, determining the text-image sequence through the attention mechanism, the text feature sequence, and the image feature sequence specifically includes:

[0023] Using the text feature sequence as the Query input to the attention mechanism and the image feature sequence as the Key and Value inputs to the attention mechanism, the text-image sequence is obtained through the following formula:

[0024]

[0025] in, Represents a text-image sequence. It is a trainable parameter matrix, where d represents the sequence of image features e. i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e t Represents a sequence of text features.

[0026] Preferably, determining the image-text sequence through the attention mechanism, the text feature sequence, and the image feature sequence specifically includes:

[0027] Using the image feature sequence as the Query input and the text feature sequence as the Key and Value inputs of the attention mechanism, the image-text sequence is obtained through the following formula:

[0028]

[0029] in, Represents a sequence of images and text. It is a trainable parameter matrix, where d represents the text feature sequence e t Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e i Represents image features.

[0030] Preferably, obtaining the modality fusion judgment factor based on the text-image sequence and the image-text sequence specifically includes:

[0031] By concatenating the text-image sequence and the image-text sequence, the modality fusion judgment factor is obtained using the following formula:

[0032]

[0033] in, Represents a text-image sequence. Represents a sequence of images and text. Indicates feature connection, e c This represents the modality fusion judgment factor.

[0034] Preferably, the step of determining the prediction results corresponding to the text judgment factor, image judgment factor, and modality fusion judgment factor sequentially through the loss function specifically includes:

[0035] The softmax layer sequentially transforms the text judgment factor, image judgment factor, and modality fusion judgment factor into text prediction results, image prediction results, and modality fusion prediction results for the news dataset corresponding to the training set.

[0036] The weight parameters for text prediction, image prediction, and modality fusion prediction are determined using the following formulas:

[0037]

[0038] The authenticity of the news dataset corresponding to the training set is determined using the following formulas based on the weight parameters and results of text prediction, image prediction, and modality fusion prediction:

[0039]

[0040] Where, p i p represents the predicted sequence of a certain modality decision factor after passing through a softmax layer. j This represents the text prediction result, image prediction result, or modality fusion prediction result, ∏ j≠i The expression represents the product operation, where j≠i means multiplying all sequences that are not the current prediction result. β is a parameter that controls the weights, ranging from [0,1]; m is the number of classification results, with a value of 2; q i This represents the weight parameters for text prediction, image prediction, or modality fusion prediction; n is the number of decision factors, l y This indicates the overall deviation of the current predicted value from the actual value.

[0041] Preferably, the maximum sequence length corresponding to the text feature sequence is 512 characters. When the sequence length of the news text is greater than 512 characters, the first 512 characters are selected. If the sequence length of the news text is less than 512 characters, zeros are added to the end of the news text until the sequence length of the news text is 512 characters.

[0042] Before extracting the news images included in the training set using the ResNet50 model, the resolution of the news images is uniformly set to 640×640; the dimension of the image feature sequence is 1×1024.

[0043] This invention provides a device for identifying fake news, comprising:

[0044] The unit is used to extract text features from the news files included in the training set using the BERT model, resulting in a text feature sequence. It also extracts image features from the training set and included news images using the ResNet50 model, resulting in an image feature sequence.

[0045] The first determining unit is used to sequentially determine the text judgment factor and the image judgment factor corresponding to the text feature sequence and the image feature sequence, respectively, through a self-attention mechanism;

[0046] The second determining unit is used to determine the text-image sequence and the image-text sequence sequentially through the attention mechanism, the text feature sequence and the image feature sequence, and to obtain the modality fusion judgment factor based on the text-image sequence and the image-text sequence;

[0047] The third determining unit is used to determine the prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor in turn through the loss function, and to determine the authenticity of the news dataset corresponding to the training set based on the prediction results.

[0048] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the aforementioned method for identifying fake news.

[0049] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the aforementioned method for identifying fake news.

[0050] This invention provides a method and apparatus for identifying fake news. The method includes: extracting text features from news files included in a training set using a BERT model to obtain a text feature sequence; extracting image features from the training set and included news images using a ResNet50 model to obtain an image feature sequence; determining text judgment factors and image judgment factors corresponding to the text feature sequence and the image feature sequence respectively using a self-attention mechanism; determining text-image sequences and image-text sequences sequentially using the attention mechanism, the text feature sequence, and the image feature sequence; obtaining a modality fusion judgment factor based on the text-image sequence and the image-text sequence; determining the prediction results corresponding to the text judgment factor, the image judgment factor, and the modality fusion judgment factor respectively using a loss function; and determining the authenticity of the news dataset corresponding to the training set based on the prediction results. This method aggregates text feature sequences and image feature sequences using a self-attention mechanism, thereby fully extracting text judgment factors from the text information contained in the text feature sequence and image judgment factors from the image information contained in the image feature sequence. Furthermore, by using an attention mechanism to fuse text and image features together, the resulting text-image and image-text sequences are fully linked to obtain modality fusion judgment factors. In other words, using text and image feature sequences, and applying self-attention and joint attention mechanisms, three judgment factors are obtained. Based on the performance of these three judgment factors, the veracity of news is judged, effectively improving the accuracy of fake news detection. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A schematic flowchart of a method for identifying fake news provided in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram illustrating the method for identifying fake news provided in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of a fake news identification device provided in an embodiment of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Figure 1 A schematic flowchart of a method for identifying fake news provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the method for identifying fake news provided in an embodiment of the present invention; the following is in conjunction with... Figure 1 and Figure 2 The method for identifying fake news provided in the embodiments of the present invention will be described in detail, such as... Figure 1 As shown, the method includes the following steps:

[0057] Step 101: Extract text features from the news files included in the training set using the BERT model to obtain a text feature sequence; extract image features from the training set and included news images using the ResNet50 model to obtain an image feature sequence.

[0058] Step 102: The text judgment factor and image judgment factor corresponding to the text feature sequence and the image feature sequence are determined sequentially through a self-attention mechanism.

[0059] Step 103: Through the attention mechanism, the text feature sequence, and the image feature sequence, determine the text-image sequence and the image-text sequence in sequence, and obtain the modality fusion judgment factor based on the text-image sequence and the image-text sequence;

[0060] Step 104: The prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor are determined sequentially by the loss function, and the authenticity of the news dataset corresponding to the training set is determined based on the prediction results.

[0061] Before identifying news, this method requires data processing of the collected news collection, specifically including the following methods:

[0062] 1) Data Cleaning: Removing redundant information from news datasets. Typical datasets contain a lot of data-related information, including post ID, poster, post time, post content, repost count, comment count, like count, news tags, etc. Text cleaning mainly involves removing redundant information other than post IDs, post content, news image IDs, and news tags. Additionally, since the dataset is crawled from online platforms or forums, the post content may contain emoticons, escape characters, and various meaningless characters. Text cleaning also needs to remove these extraneous characters from the news content, converting the content into plain text.

[0063] 2) Data Alignment: Since news text and news images have a one-to-many relationship (one text corresponds to multiple images), only the first image associated with the news text is retained, and other images in the dataset are deleted. The text and images in the dataset are matched one-to-one to ensure that each news text with a specific number has exactly one corresponding image.

[0064] 3) Data partitioning: After data alignment, organize the remaining data according to news number, news text, news image, and tags. Then, divide the dataset into training and test sets according to a certain ratio. This ratio is generally chosen as 8:2.

[0065] In step 101, a pre-trained BERT (Bidirectional Encoder Representation from Transformers) model is selected to extract text features from the news files included in the training set. During extraction, the parameters that need to be trained remain unchanged; only an additional output layer needs to be added for fine-tuning, which can achieve good performance in a variety of downstream tasks. In this embodiment of the invention, taking the text in the "WEIBO" dataset as an example, the pre-trained BERT model is used to extract features from all texts. Specifically, a news text is defined as d = {c1, c2, ..., c...} n} where n is the length of the text sequence. The upper limit of the input for the BERT model is 512 characters. If the text is too long, it is truncated at 512 characters; if the text is too short, zeros are added to the end to reach 512 characters. The BERT model extracts text features into a text feature sequence:

[0066] Furthermore, a pre-trained ResNet50 model is used to extract image features from news images. In practical applications, the ResNet50 model is a residual network model proposed by Kaiming He's team in 2015. This model adds direct connections to the network, allowing the raw input information to be directly passed to subsequent layers, thus enabling the simultaneous extraction of both shallow and deep features of the image. At the same time, the ResNet50 model has few parameters and good generalization ability; using a pre-trained ResNet-50 model to extract image features can save computational power and time.

[0067] In this embodiment of the invention, taking images from the WEIBO dataset as an example, a pre-trained ResNet50 model is used to extract image features from all news images included in the training set. Since the sizes of the news images vary, all news images are first resampled at a resolution of 640×640. Then, the resampled news images are encoded into 1×1024-dimensional image features using the pre-trained ResNet50 model, resulting in an image feature sequence.

[0068] In practical applications, the attention mechanism is a mapping function that takes several inputs and returns the same number of outputs. Each input includes three representations: Key, Query, and Value. The attention mechanism determines the weighted allocation of target features through the interaction of Key, Query, and Value. For example, if the Query input to the attention mechanism is a text matrix, and the Key and Value are image matrices, the attention mechanism measures the degree of attention the text gives to different regions of the image and weights the image features accordingly.

[0069] Self-attention is a special case of attention mechanisms. In self-attention, the three inputs—Key, Query, and Value—are the same. For example, if the inputs—Query, Key, and Value—are all text feature matrices, then the attention mechanism represents a weighted representation of the text sequences' focus on themselves.

[0070] In step 102, a self-attention mechanism is selected. For the text feature sequence and image feature sequence obtained in step 101, the text judgment factor corresponding to the text feature sequence and the image judgment factor corresponding to the image feature sequence are obtained respectively. Specifically:

[0071] Text feature sequence Given the three inputs Key, Query, and Value to the self-attention mechanism, the text judgment factor of the output of the self-attention mechanism is expressed by formula (1):

[0072]

[0073] in, Indicates the text judgment factor. It is a trainable parameter matrix, where d represents the input of text features e. t Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, Softmax(·), indicates that the softmax function is used to normalize (·).

[0074] Furthermore, the image feature sequence Given the three inputs Key, Query, and Value to the self-attention mechanism, the image judgment factor output by the self-attention mechanism is expressed by formula (2):

[0075]

[0076] in, Represents the image judgment factor. It is a trainable parameter matrix, where d represents the input of image features e. i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, Softmax(·), indicates that the softmax function is used to normalize (·).

[0077] In step 103, after determining the text judgment factor and the image judgment factor in step 102, it is necessary to perform a feature fusion process on the text and image. Specifically:

[0078] The text feature sequence is used as the Query input of the attention mechanism, and the image feature sequence is used as the Key and Value input of the attention mechanism. Since the attention mechanism measures the attention of the text to different regions of the image and weights the image features, the result is the "text-weighted image sequence", which is simply called the text-image sequence. The text-image sequence is obtained through formula (3):

[0079]

[0080] in, Represents a text-image sequence. It is a trainable parameter matrix, where d represents the sequence of image features e. i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e t Represents a sequence of text features.

[0081] Furthermore, the image feature sequence is used as the Query input of the attention mechanism, and the text feature sequence is used as the Key and Value input of the attention mechanism. The attention mechanism measures the attention of the image to different fields of the text and weights the text features. The result is the "image-weighted text sequence", or simply the image-text sequence. The image-text sequence is obtained through the following formula (4):

[0082]

[0083] in, Represents a sequence of images and text. It is a trainable parameter matrix, where d represents the text feature sequence e t Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e i Represents image features.

[0084] Furthermore, by concatenating the text-image sequence and the image-text sequence, the modality fusion judgment factor can be obtained. The modality fusion judgment factor can be expressed by the following formula (5):

[0085]

[0086] in, Represents a text-image sequence. Represents a sequence of images and text. Indicates feature connection, e c This represents the modality fusion judgment factor.

[0087] Before step 104, it is necessary to determine the text prediction results, image prediction results, and modality fusion prediction results corresponding to the text judgment factor, image judgment factor, and modality fusion judgment factor. Specifically, the softmax layer sequentially transforms the text judgment factor, image judgment factor, and modality fusion judgment factor into text prediction results, image prediction results, and modality fusion prediction results for the news dataset corresponding to the training set.

[0088] Since the purpose of the fake news identification method provided in this embodiment of the invention is to detect fake news, which is a binary classification task, the sigmoid function is used in the softmax layer to achieve the binary classification effect. The sigmoid function can map a real number to the interval (0,1), and the value range of the result is (0,1).

[0089] The specific formula for using the softmax layer is shown below:

[0090] p i =sigmoid(ωe+b) (6)

[0091] Where e represents the decision factor for a certain mode, which can be taken as: e c ω and b are learnable weight parameters. ω has the dimension of the transpose of e, ensuring it can be multiplied by e. b has the dimension of the product of ωe, ensuring it can be added to ωe. The final values ​​of ω and b are determined jointly by the deep learning network and the training set. i The prediction sequence of a certain modality decision factor after passing through the softmax layer represents the prediction result of that modality and can be expressed as: P t P i P c .

[0092] In an embodiment of the present invention, P t express The prediction sequence after the softmax layer represents the predicted text result; P i express The prediction sequence after the softmax layer represents the prediction result of the image; P c e c The prediction sequence after the softmax layer represents the prediction result of modality fusion.

[0093] Furthermore, the text prediction results, image prediction results, and modal fusion prediction results are comprehensively judged by the following formula (7), that is, the weight parameters of text prediction, image prediction, and modal fusion prediction corresponding to the text prediction results, image prediction results, and modal fusion prediction results are determined sequentially by formula (7):

[0094]

[0095] Where, p j This represents the text prediction result, image prediction result, or modality fusion prediction result, ∏ j≠i The expression represents the product operation, where j≠i means multiplying all sequences that are not the current prediction result. β is a parameter that controls the weights and can be adjusted according to the proportion of text, images, and modal fusion information in different types of news, with a value range of [0,1]. m is the number of classification results; in this task, the results are only divided into true news and fake news, so m is set to 2 here. q i This represents the weight parameters for text prediction, image prediction, or modality fusion prediction.

[0096] Furthermore, the authenticity of the news dataset corresponding to the training set is determined by the following formula (8) based on the weight parameters and text prediction results of text prediction, the weight parameters and image prediction results of image prediction, and the weight parameters and modality fusion prediction results of modality fusion prediction:

[0097]

[0098] Where, q i These represent the weight parameters for text prediction, image prediction, or modality fusion prediction. p is the summation symbol, representing the summation of all class criterion factors. i This represents the prediction sequence of a certain modality's decision factor after passing through the softmax layer, where n is the number of decision factors, which is 3 in this case. y This represents the overall deviation of the current predicted value from the actual value. That is, when l y The smaller the value, the more accurate the model's prediction, meaning a higher accuracy in determining the authenticity of news.

[0099] In summary, this invention provides a method for identifying fake news. The method includes: extracting text features from news files included in a training set using a BERT model to obtain a text feature sequence; extracting image features from the training set and included news images using a ResNet50 model to obtain an image feature sequence; sequentially determining text judgment factors and image judgment factors corresponding to the text feature sequence and the image feature sequence using a self-attention mechanism; sequentially determining a text-image sequence and an image-text sequence using the attention mechanism, the text feature sequence, and the image feature sequence; obtaining a modality fusion judgment factor based on the text-image sequence and the image-text sequence; sequentially determining the prediction results corresponding to the text judgment factor, the image judgment factor, and the modality fusion judgment factor using a loss function; and determining the authenticity of the news dataset corresponding to the training set based on the prediction results. The above method, based on the extraction of text and image features, employs a self-attention mechanism to aggregate text and image feature sequences. This allows for the full extraction of text judgment factors containing textual information and image judgment factors containing image information from the image feature sequence. Furthermore, by using a joint attention mechanism to fuse text and image features, the resulting text-image and image-text sequences are fully linked to obtain modality fusion judgment factors. In other words, by using text and image feature sequences and applying self-attention and joint attention mechanisms, three judgment factors are obtained. The veracity of news is judged based on the combined performance of these three judgment factors, effectively improving the accuracy of fake news detection.

[0100] Based on the same inventive concept, this invention provides a device for identifying fake news. Since the principle by which this device solves the technical problem is similar to that of a method for identifying fake news, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0101] Figure 3This is a schematic diagram of a fake news identification device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the device includes a obtaining unit 201, a first determining unit 202, a second determining unit 203, and a third determining unit 204, specifically:

[0102] Unit 201 is obtained, which is used to extract text features of news files included in the training set through the BERT model to obtain a text feature sequence, and to extract image features of the training set and included news images through the ResNet50 model to obtain an image feature sequence.

[0103] The first determining unit 202 is used to sequentially determine the text judgment factor and the image judgment factor corresponding to the text feature sequence and the image feature sequence respectively through a self-attention mechanism;

[0104] The second determining unit 203 is used to determine the text-image sequence and the image-text sequence sequentially through the attention mechanism, the text feature sequence and the image feature sequence, and to obtain the modality fusion judgment factor based on the text-image sequence and the image-text sequence;

[0105] The third determining unit 203 is used to determine the prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor in turn through the loss function, and determine the authenticity of the news dataset corresponding to the training set based on the prediction results.

[0106] Preferably, the first determining unit 202 is specifically used for:

[0107] Using the text feature sequence as input to the self-attention mechanism, the text judgment factor is obtained through the following formula:

[0108]

[0109] Using the image feature sequence as input to the self-attention mechanism, the image judgment factor is obtained through the following formula:

[0110]

[0111] in, Indicates the text judgment factor. Represents a text feature sequence. Represents the image judgment factor. Represents the image feature sequence. It is a trainable parameter matrix, where d represents the input of text features e t Or image features e i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. Or to The transpose operation.

[0112] Preferably, the second determining unit 203 is specifically used for:

[0113] Using the text feature sequence as the Query input to the attention mechanism and the image feature sequence as the Key and Value inputs to the attention mechanism, the text-image sequence is obtained through the following formula:

[0114]

[0115] in, Represents a text-image sequence. It is a trainable parameter matrix, where d represents the sequence of image features e. i Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e t Represents a sequence of text features.

[0116] Preferably, the second determining unit 203 is specifically used for:

[0117] Using the image feature sequence as the Query input and the text feature sequence as the Key and Value inputs of the attention mechanism, the image-text sequence is obtained through the following formula:

[0118]

[0119] in, Represents a sequence of images and text. It is a trainable parameter matrix, where d represents the text feature sequence e t Through the mapping dimension after the fully connected layer, T represents the mapping dimension. The transpose operation, e i Represents image features.

[0120] Preferably, the second determining unit 203 is specifically used for:

[0121] By concatenating the text-image sequence and the image-text sequence, the modality fusion judgment factor is obtained using the following formula:

[0122]

[0123] in, Represents a text-image sequence. Represents a sequence of images and text. Indicates feature connection, e c This represents the modality fusion judgment factor.

[0124] Preferably, the third determining unit 204 is specifically used for:

[0125] The softmax layer sequentially transforms the text judgment factor, image judgment factor, and modality fusion judgment factor into text prediction results, image prediction results, and modality fusion prediction results for the news dataset corresponding to the training set.

[0126] The weight parameters for text prediction, image prediction, and modality fusion prediction are determined using the following formulas:

[0127]

[0128] The authenticity of the news dataset corresponding to the training set is determined using the following formulas based on the weight parameters and results of text prediction, image prediction, and modality fusion prediction:

[0129]

[0130] Where, p j This represents the text prediction result, image prediction result, or modality fusion prediction result, ∏ j≠i The expression represents the product operation, where j≠i means multiplying all sequences that are not the current prediction result. β is a parameter that controls the weights, ranging from [0,1]; m is the number of classification results, with a value of 2; q i This represents the weight parameters for text prediction, image prediction, or modality fusion prediction; n is the number of decision factors, l y This indicates the overall deviation of the current predicted value from the actual value.

[0131] Preferably, the obtaining unit is specifically used for: the maximum sequence length corresponding to the text feature sequence is 512 characters; when the sequence length of the news text is greater than 512 characters, the first 512 characters are selected; if the sequence length of the news text is less than 512 characters, zeros are added to the end of the news text until the sequence length of the news text is 512 characters.

[0132] Before extracting the news images included in the training set using the ResNet50 model, the resolution of the news images is uniformly set to 640×640; the dimension of the image feature sequence is 1×1024.

[0133] It should be understood that the units included in the above-described fake news identification device are merely a logical division based on the functions implemented by the device. In practical applications, the units can be superimposed or split. Furthermore, the functions implemented by the fake news identification device provided in this embodiment correspond one-to-one with the fake news identification method provided in the above-described embodiment. The more detailed processing flow implemented by the device has been described in detail in the first embodiment of the method described above, and will not be described in detail here.

[0134] Another embodiment of the present invention provides a computer device, the computer device including: a processor and a memory; the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the electronic device executes each step of the fake news identification method in the method flow shown in the above method embodiment.

[0135] Another embodiment of the present invention provides a computer-readable storage medium storing computer instructions that, when executed on a computer device, cause the computer device to perform each step of the fake news identification method in the method flow shown in the above method embodiment.

[0136] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0137] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Thus, if these modifications and variations of this invention fall under the claims of this invention and their equivalents...

Claims

1. A method for identifying fake news, characterized in that, include: Text features of news files included in the training set are extracted using the BERT model to obtain a text feature sequence. Image features of the training set and included news images are extracted using the ResNet50 model to obtain an image feature sequence. The text judgment factor and image judgment factor corresponding to the text feature sequence and the image feature sequence are determined sequentially by a self-attention mechanism. The text-image sequence and the image-text sequence are determined sequentially through the attention mechanism, the text feature sequence, and the image feature sequence, and the modality fusion judgment factor is obtained based on the text-image sequence and the image-text sequence. The prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor are determined sequentially by the loss function, and the authenticity of the news dataset corresponding to the training set is determined based on the prediction results. The step of sequentially determining the text judgment factor and image judgment factor corresponding to the text feature sequence and the image feature sequence respectively through a self-attention mechanism specifically includes: Using the text feature sequence as input to the self-attention mechanism, the text judgment factor is obtained through the following formula: Using the image feature sequence as input to the self-attention mechanism, the image judgment factor is obtained through the following formula: The step of determining the prediction results corresponding to the text judgment factor, image judgment factor, and modality fusion judgment factor sequentially through the loss function specifically includes: The softmax layer sequentially transforms the text judgment factor, image judgment factor, and modality fusion judgment factor into text prediction results, image prediction results, and modality fusion prediction results for the news dataset corresponding to the training set. The weight parameters for text prediction, image prediction, and modality fusion prediction are determined using the following formulas: The authenticity of the news dataset corresponding to the training set is determined using the following formulas based on the weight parameters and results of text prediction, image prediction, and modality fusion prediction: in, Indicates the text judgment factor. , representing a text feature sequence, Represents the image judgment factor. , representing the image feature sequence, , , It is a trainable parameter matrix. This indicates the text features Or image features The mapping dimension after the fully connected layer Indicates to Or to The transpose operation, This represents the predicted sequence of a certain modality decision factor after passing through a softmax layer. This indicates the text prediction result, image prediction result, or modality fusion prediction result. This represents the product operation. This means multiplying over all sequences that are not the current prediction result. It is a parameter that controls the weights, and its value ranges from [0,1]. The number of classification results, with a value of 2; This represents the weight parameters for text prediction, image prediction, or modality fusion prediction; n is the number of decision factors. This indicates the overall deviation of the current predicted value from the actual value.

2. The method as described in claim 1, characterized in that, The process of determining the text-image sequence through an attention mechanism, the text feature sequence, and the image feature sequence specifically includes: Using the text feature sequence as the Query input to the attention mechanism and the image feature sequence as the Key and Value inputs to the attention mechanism, the text-image sequence is obtained through the following formula: in, Represents a text-image sequence. , , It is a trainable parameter matrix. This represents the sequence of image features. The mapping dimension after the fully connected layer Indicates to The transpose operation, Represents a sequence of text features.

3. The method as described in claim 1, characterized in that, The process of determining the image-text sequence through the attention mechanism, the text feature sequence, and the image feature sequence specifically includes: Using the image feature sequence as the Query input and the text feature sequence as the Key and Value inputs of the attention mechanism, the image-text sequence is obtained through the following formula: in, Represents a sequence of images and text. , , It is a trainable parameter matrix. This indicates that the text feature sequence The mapping dimension after the fully connected layer Indicates to The transpose operation, Represents image features.

4. The method as described in claim 1, characterized in that, The process of obtaining the modality fusion judgment factor based on the text-image sequence and the image-text sequence specifically includes: By concatenating the text-image sequence and the image-text sequence, the modality fusion judgment factor is obtained using the following formula: in, Represents a text-image sequence. Represents a sequence of images and text. Indicates feature connection, This represents the modality fusion judgment factor.

5. The method as described in claim 1, characterized in that, The maximum length of the text feature sequence is 512 characters. When the sequence length of the news text is greater than 512 characters, the first 512 characters are selected. If the sequence length of the news text is less than 512 characters, zeros are added to the end of the news text until the sequence length of the news text is 512 characters. Before extracting the news images included in the training set using the ResNet50 model, the resolution of the news images is uniformly set to 640×640; the dimension of the image feature sequence is 1×1024.

6. A device for identifying fake news, characterized in that, include: The unit is used to extract text features from the news files included in the training set using the BERT model, resulting in a text feature sequence. It also extracts image features from the training set and included news images using the ResNet50 model, resulting in an image feature sequence. The first determining unit is used to sequentially determine the text judgment factor and the image judgment factor corresponding to the text feature sequence and the image feature sequence, respectively, through a self-attention mechanism; The second determining unit is used to determine the text-image sequence and the image-text sequence sequentially through the attention mechanism, the text feature sequence and the image feature sequence, and to obtain the modality fusion judgment factor based on the text-image sequence and the image-text sequence; The third determining unit is used to determine the prediction results corresponding to the text judgment factor, image judgment factor and modality fusion judgment factor respectively through the loss function, and determine the authenticity of the news dataset corresponding to the training set based on the prediction results. The first determining unit is specifically used to: take the text feature sequence as input to the self-attention mechanism, and obtain the text judgment factor using the following formula: Using the image feature sequence as input to the self-attention mechanism, the image judgment factor is obtained through the following formula: The third determining unit is specifically used for: The softmax layer sequentially transforms the text judgment factor, image judgment factor, and modality fusion judgment factor into text prediction results, image prediction results, and modality fusion prediction results for the news dataset corresponding to the training set. The weight parameters for text prediction, image prediction, and modality fusion prediction are determined using the following formulas: The authenticity of the news dataset corresponding to the training set is determined using the following formulas based on the weight parameters and results of text prediction, image prediction, and modality fusion prediction: in, Indicates the text judgment factor. , representing a text feature sequence, Represents the image judgment factor. , representing the image feature sequence, , , It is a trainable parameter matrix. This indicates the text features Or image features The mapping dimension after the fully connected layer Indicates to Or to The transpose operation, This represents the predicted sequence of a certain modality decision factor after passing through a softmax layer. This indicates the text prediction result, image prediction result, or modality fusion prediction result. This represents the product operation. This means multiplying over all sequences that are not the current prediction result. It is a parameter that controls the weights, and its value ranges from [0,1]. The number of classification results, with a value of 2; This represents the weight parameters for text prediction, image prediction, or modality fusion prediction; n is the number of decision factors. This indicates the overall deviation of the current predicted value from the actual value.

7. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the method for identifying fake news as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, causes the processor to perform the method for identifying fake news as described in any one of claims 1-5.

Images