A method, apparatus and electronic device for fake news detection

Abstract

This invention discloses a method, apparatus, and electronic device for detecting fake news, relating to the field of intelligent detection technology. The method involves acquiring multimodal fake news data to be detected; extracting external knowledge information matching the multimodal fake news data to obtain combined multimodal data; determining the intermodal similarity score of the combined multimodal data; and determining the corresponding detection result based on the modal similarity score and a preset similarity score threshold. By integrating entity information from images and text in news posts within the multimodal fake news data to be detected into the corresponding matching external knowledge information from an external knowledge graph, the method effectively utilizes background knowledge information to detect fake news, achieving good detection results and ensuring the reliability and stability of fake news detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, and in particular to a method, apparatus and electronic device for detecting fake news. Background Technology

[0002] In recent years, with the rapid development of internet information technology and the iterative updates of mobile terminals such as smartphones and tablets, people are more inclined to obtain the latest news from social media and news apps. This method of obtaining news is more convenient and faster, and it can be widely disseminated with a simple forward. However, due to the low cost and widespread reach of news obtained through social media and news apps, the authenticity of news stories is questioned, and some news items may be false every day. These false news stories often generate huge traffic and attract attention, but they can have a negative impact on society and the public, causing unnecessary trouble.

[0003] Currently, detecting fake news often relies on manual verification, which is a massive undertaking requiring significant human and material resources and lacks timeliness, making timely debunking impossible. However, the development and application of machine learning, particularly the rapid advancements in deep learning for image and text recognition, have made it possible to automate the identification of fake news. Existing multimodal methods primarily target multimodal news data containing both images and text. They use corresponding feature extractors to extract features from both images and text separately, then either simply concatenate the two feature sets for classification or fuse them using an attention mechanism, and finally classify the fused output.

[0004] However, the above-mentioned methods for detecting fake news only involve simple mining of image and text features. For forged image and text information, especially when the forged image and text are semantically consistent, the lack of prior knowledge makes it impossible to make a correct judgment, which reduces the reliability and stability of fake news detection. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and electronic device for detecting fake news, thereby solving the problem that the lack of prior knowledge information makes it impossible to make correct judgments, which reduces the reliability and stability of fake news detection.

[0006] In a first aspect, the present invention provides a method for detecting fake news, the method comprising:

[0007] Acquire multimodal fake news data to be detected;

[0008] External knowledge information matching the multimodal fake news data is extracted to obtain combined multimodal data;

[0009] Determine the intermodal similarity score of the combined multimodal data;

[0010] The corresponding detection result is determined based on the intermodal similarity score and the preset similarity score threshold.

[0011] Using the above technical solution, the fake news detection method provided by this invention acquires multimodal fake news data to be detected; extracts external knowledge information matching the multimodal fake news data to obtain combined multimodal data; determines the inter-modal similarity score of the combined multimodal data; and determines the corresponding detection result based on the inter-modal similarity score and a preset similarity score threshold. By integrating entity information from images and text in news posts within the multimodal fake news data to be detected into the corresponding matched external knowledge information from an external knowledge graph, the method effectively utilizes background knowledge information to detect fake news, achieving good detection results and ensuring the reliability and stability of fake news detection.

[0012] In one possible implementation, the step of extracting external knowledge information that matches the multimodal fake news data to obtain combined multimodal data includes:

[0013] Extract image entity information and text entity information from the multimodal fake news data respectively;

[0014] The external knowledge information is determined based on the text entity information, the image entity information, and the external knowledge information database;

[0015] The external knowledge information, the text entity information, and the image entity information are processed to obtain the combined multimodal data.

[0016] In one possible implementation, determining the external knowledge information based on the text entity information, the image entity information, and the external knowledge information database includes:

[0017] The text entity information and the image entity information are linked to an external knowledge information database using an entity linking tool to determine the external knowledge information that matches the text entity information and the image entity information.

[0018] In one possible implementation, the combined multimodal data includes target text information and target image information; the step of processing the external knowledge information, the text entity information, and the image entity information to obtain the combined multimodal data includes:

[0019] The external knowledge information and the text entity information are concatenated to obtain the target text information;

[0020] The target image information is determined by performing target data processing on the image entity information.

[0021] In one possible implementation, determining the inter-modal similarity score of the combined multimodal data includes:

[0022] The target image information and the target text information are fused using a visual language model to obtain target fake news data;

[0023] The target fake news data is classified based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data.

[0024] In one possible implementation, determining the corresponding detection result based on the inter-modal similarity score and a preset similarity score threshold includes:

[0025] If the intermodal similarity score is greater than or equal to the similarity score threshold, the multimodal fake news data to be detected is determined to be real news data;

[0026] If the intermodal similarity score is less than the similarity score threshold, the multimodal fake news data to be detected is determined to be fake news data.

[0027] Secondly, the present invention also provides a fake news detection device, the device comprising:

[0028] The acquisition module is used to acquire multimodal fake news data to be detected;

[0029] An extraction module is used to extract external knowledge information that matches the multimodal fake news data to obtain combined multimodal data;

[0030] The first determining module is used to determine the intermodal similarity score of the combined multimodal data;

[0031] The second determining module is used to determine the corresponding detection result based on the inter-modal similarity score and the preset similarity score threshold.

[0032] In one possible implementation, the extraction module includes:

[0033] An extraction submodule is used to extract image entity information and text entity information from the multimodal fake news data, respectively;

[0034] The first determining submodule is used to determine the external knowledge information based on the text entity information, the image entity information, and the external knowledge information database.

[0035] The data processing submodule is used to process the external knowledge information, the text entity information and the image entity information to obtain the combined multimodal data.

[0036] The first determining submodule includes:

[0037] The first determining unit is used to link the text entity information and the image entity information to an external knowledge information database through an entity linking tool, and to determine the external knowledge information that matches the text entity information and the image entity information;

[0038] The combined multimodal data includes target text information and target image information; the data processing submodule includes:

[0039] The splicing processing unit is used to splice the external knowledge information with the text entity information to obtain the target text information;

[0040] The second determining unit is used to perform target data processing on the image entity information to determine the target image information.

[0041] In one possible implementation, the first determining module includes:

[0042] The fusion processing submodule is used to fuse the target image information and the target text information through a visual language model to obtain target fake news data;

[0043] The classification submodule is used to classify the target fake news data based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data;

[0044] The second determining module includes:

[0045] The second determining submodule is used to determine the multimodal fake news data to be detected as real news data when the intermodal similarity score is greater than or equal to the similarity score threshold.

[0046] The third determining submodule is used to determine the multimodal fake news data to be detected as fake news data when the intermodal similarity score is less than the similarity score threshold.

[0047] The beneficial effects of the fake news detection device provided in the second aspect are the same as those of the fake news detection method described in the first aspect or any possible implementation of the first aspect, and will not be repeated here.

[0048] Thirdly, the present invention also provides an electronic device comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the device to perform the fake news detection device described in any possible implementation of the second aspect.

[0049] The beneficial effects of the electronic device provided in the third aspect are the same as those of the fake news detection device described in the second aspect or any possible implementation of the second aspect, and will not be repeated here. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0051] Figure 1 A flowchart illustrating a method for detecting fake news provided in an embodiment of this application is shown;

[0052] Figure 2 A flowchart illustrating another method for detecting fake news provided in an embodiment of this application is shown;

[0053] Figure 3 This illustration shows a schematic diagram of the model structure of a fake news detection model provided in an embodiment of this application;

[0054] Figure 4 This paper illustrates a structural flowchart of a fake news detection device provided in an embodiment of this application.

[0055] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Detailed Implementation

[0057] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0058] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0059] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0060] Figure 1 The following is a flowchart illustrating a method for detecting fake news provided in an embodiment of this application. Figure 1 As shown, the fake news detection method includes:

[0061] Step 101: Obtain the multimodal fake news data to be detected.

[0062] Step 102: Extract external knowledge information that matches the multimodal fake news data to obtain combined multimodal data.

[0063] In this application, image entity information and text entity information can be extracted from the multimodal fake news data respectively; the external knowledge information can be determined based on the text entity information, the image entity information and the external knowledge information database; and the external knowledge information, the text entity information and the image entity information can be processed to obtain the combined multimodal data.

[0064] Step 103: Determine the intermodal similarity score of the combined multimodal data.

[0065] In this application, the target image information and the target text information can be fused using a visual language model to obtain target fake news data; the target fake news data can be classified based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data.

[0066] Step 104: Determine the corresponding detection result based on the intermodal similarity score and the preset similarity score threshold.

[0067] In this application, the multimodal fake news data to be detected can be determined to be real news data if the intermodal similarity score is greater than or equal to the similarity score threshold; and the multimodal fake news data to be detected can be determined to be fake news data if the intermodal similarity score is less than the similarity score threshold.

[0068] The fake news detection method provided in this invention acquires multimodal fake news data to be detected; extracts external knowledge information matching the multimodal fake news data to obtain combined multimodal data; determines the inter-modal similarity score of the combined multimodal data; and determines the corresponding detection result based on the inter-modal similarity score and a preset similarity score threshold. By integrating entity information from images and text in news posts within the multimodal fake news data to be detected into the corresponding matched external knowledge information from an external knowledge graph, the method effectively utilizes background knowledge information to detect fake news, achieving good detection results and ensuring the reliability and stability of fake news detection.

[0069] Figure 2 This document illustrates a flowchart of another fake news detection method provided in an embodiment of this application. See also... Figure 2 The method for detecting fake news includes:

[0070] Step 201: Obtain the multimodal fake news data to be detected.

[0071] In this application, Figure 3 This illustration shows a schematic diagram of the model structure of a fake news detection model provided in an embodiment of this application, as shown below. Figure 3 As shown, the fake news detection model may include a multimodal news acquisition unit 301, which can acquire multimodal fake news data to be detected.

[0072] Step 202: Extract the image entity information and text entity information from the multimodal fake news data respectively.

[0073] In this application, the multimodal news acquisition unit 301 can extract image entity information and text entity information from multimodal fake news data.

[0074] Step 203: Determine the external knowledge information based on the text entity information, the image entity information, and the external knowledge information database.

[0075] See also in this application. Figure 3The fake news detection model may further include a knowledge graph unit 302 and an image target detection unit 303, both connected to the multimodal news acquisition unit 301. The knowledge graph unit 302 and the image target detection unit 303 are interconnected. The image target detection unit 303 can process image entity information and, through an entity linking tool, link the text entity information and the image entity information to an external knowledge information base in the knowledge graph unit 302 to determine the external knowledge information that matches the text entity information and the image entity information.

[0076] In this application, the knowledge graph unit can use an external knowledge graph, WiKidata, which contains a large amount of information from the web and is stored in the form of triples. Specifically, the use of external knowledge involves extracting entity information from images and text in news posts. For entities in images, Fast-R-CNN, i.e., an image object detection unit, is used to detect target entities in the image, obtaining entity representations Ei.

[0077] The entity linking tool TAGME is used to link entities in the text (proper nouns, personal names, place names, etc., Et) and target entities (Ei) obtained from the image to the knowledge graph Wikidata. The entities are matched and aligned with the entities existing in the knowledge graph. The TransE model is used to encode the aligned entities and relations in the knowledge graph.

[0078] Step 204: Process the external knowledge information, the text entity information, and the image entity information to obtain the combined multimodal data.

[0079] In this application, the combined multimodal data includes target text information and target image information. The specific implementation process of step 204 above may include: concatenating the external knowledge information with the text entity information to obtain the target text information; and performing target data processing on the image entity information to determine the target image information.

[0080] Specifically, the aligned external knowledge information is used as a supplement to obtain the external knowledge information encoding T. kg External knowledge information T kg The text information T is concatenated with the original news text information T to form the final text information T. f +T kg .

[0081] Step 205: The target image information and the target text information are fused using a visual language model to obtain target fake news data.

[0082] See Figure 3The fake news detection model also includes a visual language model 304 connected to the multimodal news acquisition unit 301 and the knowledge graph unit 302 respectively. The visual language model can be the CLIP model, which is a contrastive language visual pre-training model. This model is pre-trained using a large amount of Internet data and trained using 400 million image-text pairs collected by OpenAI. It can achieve good results in downstream tasks. This application uses the CLIP model for image and text matching and discrimination.

[0083] Specifically, the CLIP model includes a text encoder and an image encoder. After encoding, the text and visual embeddings are mapped to the same space. Using contrastive learning, the distance between matching image-text embeddings is shortened, while the distance between mismatched image-text embeddings is widened. Two encoders are used to process the text and image data respectively. The text encoder uses the BERT model, and the image encoder can use the Vision Transformer (ViT) model. The text feature T... f =text_encoder(T), image features I f =image_encoder(I).

[0084] In this application, the target image information and the target text information can be fused using a visual language model to obtain target fake news data.

[0085] Specifically, the aligned external knowledge information is used as a supplement to obtain the external knowledge information encoding T. kg External knowledge information T kg The text information T is concatenated with the original news text information T to form the final text information T. f +T kg Finally, the text information T f +T kg The original image information I, which is the target image information, is input into the visual language model to obtain the final result F, which is the target fake news data.

[0086] Step 206: Classify the target fake news data based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data.

[0087] See Figure 3 The fake news detection model also includes a classification prediction unit 305 connected to the visual language model 304. The classification prediction unit can classify the target fake news data based on a preset activation function to obtain the prediction score corresponding to the multimodal fake news data.

[0088] In this application, a prediction score can be obtained by inputting the Sigmoid activation function into a fully connected layer for classification.

[0089]

[0090] Among them, W f and b f For weights and bias parameters, For predicted labels.

[0091] It should be noted that the classification loss can be calculated using the cross-entropy loss function. The objective function is to minimize the cross-entropy loss, thereby correctly predicting true and false news.

[0092]

[0093] Where y is the real label. For predicting labels, L represents the loss function value.

[0094] Step 207: If the intermodal similarity score is greater than or equal to the similarity score threshold, determine that the multimodal fake news data to be detected is real news data.

[0095] In this application, the visual language model 304 can determine that the multimodal fake news data to be detected is real news data when the intermodal similarity score is greater than or equal to the similarity score threshold.

[0096] Step 208: If the intermodal similarity score is less than the similarity score threshold, the multimodal fake news data to be detected is determined to be fake news data.

[0097] In this application, the visual language model 304 can determine that the multimodal fake news data to be detected is fake news data when the intermodal similarity score is less than the similarity score threshold.

[0098] In this embodiment, the similarity score threshold is not specifically limited, and can be set according to the actual application scenario.

[0099] This invention addresses the characteristics of multimodal fake news data by aligning it with a large-scale external knowledge graph. It effectively incorporates prior knowledge information from entities in images and text, and utilizes the CLIP (Graph-Text Contrast Learning) model to accurately determine image-text matching, achieving good performance on general datasets. By integrating entity information from images and text in news posts into the external knowledge graph, it effectively utilizes background knowledge to detect fake news. Furthermore, it introduces a contrastive language-image pre-trained model for fake news detection, combining it with external knowledge information to achieve good detection results.

[0100] The fake news detection method provided in this invention acquires multimodal fake news data to be detected; extracts external knowledge information matching the multimodal fake news data to obtain combined multimodal data; determines the inter-modal similarity score of the combined multimodal data; and determines the corresponding detection result based on the inter-modal similarity score and a preset similarity score threshold. By integrating entity information from images and text in news posts within the multimodal fake news data to be detected into the corresponding matched external knowledge information from an external knowledge graph, the method effectively utilizes background knowledge information to detect fake news, achieving good detection results and ensuring the reliability and stability of fake news detection.

[0101] Figure 4 This application provides a schematic diagram of the structure of a fake news detection device according to an embodiment of the present application. Figure 4 As shown, the fake news detection device 400 includes:

[0102] The acquisition module 401 is used to acquire multimodal fake news data to be detected;

[0103] Extraction module 402 is used to extract external knowledge information that matches the multimodal fake news data to obtain combined multimodal data;

[0104] The first determining module 403 is used to determine the inter-modal similarity score of the combined multimodal data;

[0105] The second determining module 404 is used to determine the corresponding detection result based on the intermodal similarity score and the preset similarity score threshold.

[0106] In one possible implementation, the extraction module includes:

[0107] An extraction submodule is used to extract image entity information and text entity information from the multimodal fake news data, respectively;

[0108] The first determining submodule is used to determine the external knowledge information based on the text entity information, the image entity information, and the external knowledge information database.

[0109] The data processing submodule is used to process the external knowledge information, the text entity information and the image entity information to obtain the combined multimodal data.

[0110] The first determining submodule includes:

[0111] The first determining unit is used to link the text entity information and the image entity information to an external knowledge information database through an entity linking tool, and to determine the external knowledge information that matches the text entity information and the image entity information;

[0112] The combined multimodal data includes target text information and target image information; the data processing submodule includes:

[0113] The splicing processing unit is used to splice the external knowledge information with the text entity information to obtain the target text information;

[0114] The second determining unit is used to perform target data processing on the image entity information to determine the target image information.

[0115] In one possible implementation, the first determining module includes:

[0116] The fusion processing submodule is used to fuse the target image information and the target text information through a visual language model to obtain target fake news data;

[0117] The classification submodule is used to classify the target fake news data based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data;

[0118] The second determining module includes:

[0119] The second determining submodule is used to determine the multimodal fake news data to be detected as real news data when the intermodal similarity score is greater than or equal to the similarity score threshold.

[0120] The third determining submodule is used to determine the multimodal fake news data to be detected as fake news data when the intermodal similarity score is less than the similarity score threshold.

[0121] The fake news detection device provided in this invention acquires multimodal fake news data to be detected; extracts external knowledge information matching the multimodal fake news data to obtain combined multimodal data; determines the intermodal similarity score of the combined multimodal data; and determines the corresponding detection result based on the intermodal similarity score and a preset similarity score threshold. By integrating entity information from images and text in news posts within the multimodal fake news data to be detected into the corresponding matching external knowledge information from an external knowledge graph, the device effectively utilizes background knowledge information to detect fake news, achieving good detection results and ensuring the reliability and stability of fake news detection.

[0122] This invention provides a fake news detection device, applicable to a system including a controller and at least one detection circuit electrically connected to the controller, such as... Figures 1 to 3 To avoid duplication, the methods for detecting fake news shown will not be described again here.

[0123] The electronic device in this embodiment of the invention can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment of the invention does not impose specific limitations.

[0124] The electronic device in this embodiment of the invention can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment of the invention does not impose specific limitations.

[0125] Figure 5 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention is shown. Figure 5 As shown, the electronic device 500 includes a processor 510.

[0126] like Figure 5 As shown, the processor 510 described above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention.

[0127] like Figure 5 As shown, the electronic device 500 may further include a communication line 540. The communication line 540 may include a path for transmitting information between the components.

[0128] Optional, such as Figure 5As shown, the above-described electronic device may further include a communication interface 520. There may be one or more communication interfaces 520. The communication interface 520 may use any transceiver-like device for communicating with other devices or communication networks.

[0129] Optional, such as Figure 5 As shown, the electronic device may further include a memory 530. The memory 530 stores computer execution instructions for implementing the present invention, and its execution is controlled by a processor. The processor executes the computer execution instructions stored in the memory to implement the method provided in the embodiments of the present invention.

[0130] like Figure 5 As shown, memory 530 can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 530 can exist independently and be connected to processor 510 via communication line 540. Memory 530 can also be integrated with processor 510.

[0131] Optionally, the computer execution instructions in the embodiments of the present invention may also be referred to as application code, and the embodiments of the present invention do not specifically limit this.

[0132] In a specific implementation, as one example, such as Figure 5 As shown, processor 510 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 in the CPU.

[0133] In a specific implementation, as one example, such as Figure 5 As shown, the terminal device may include multiple processors, such as Figure 5 The first processor 5101 and the second processor 5102 are included. Each of these processors can be a single-core processor or a multi-core processor.

[0134] Figure 6 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Figure 6 As shown, the chip 600 includes one or more processors 510.

[0135] Optional, such as Figure 6 As shown, the chip also includes a communication interface 520 and a memory 530. The memory 530 may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0136] In some implementations, such as Figure 6 As shown, memory 530 stores the following elements: execution modules or data structures, or subsets thereof, or extended sets thereof.

[0137] In embodiments of the present invention, such as Figure 6 As shown, the corresponding operation is executed by calling the operation instructions stored in the memory (which can be stored in the operating system).

[0138] like Figure 6 As shown, the processor 510 controls the processing operations of any one of the terminal devices. The processor 510 can also be called a central processing unit (CPU).

[0139] like Figure 6 As shown, memory 530 may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory 530 may also include NVRAM. For example, in an application, memory, communication interfaces, and memory are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 6 The general labeled all buses as Bus System 640.

[0140] like Figure 6As shown, the methods disclosed in the above embodiments of the present invention can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0141] On the one hand, a computer-readable storage medium is provided, which stores instructions that, when executed, implement the functions performed by the terminal device in the above embodiments.

[0142] On the one hand, a chip is provided that is used in a terminal device. The chip includes at least one processor and a communication interface. The communication interface and at least one processor are coupled together. The processor is used to run instructions to implement the functions performed by the fake news detection method in the above embodiments.

[0143] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0144] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0145] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for detecting fake news, characterized in that, The method includes: Acquire multimodal fake news data to be detected; External knowledge information matching the multimodal fake news data is extracted to obtain combined multimodal data; Determine the intermodal similarity score of the combined multimodal data; The corresponding detection result is determined based on the intermodal similarity score and the preset similarity score threshold; The step of extracting external knowledge information that matches the multimodal fake news data to obtain combined multimodal data includes: Extract image entity information and text entity information from the multimodal fake news data respectively; The external knowledge information is determined based on the text entity information, the image entity information, and the external knowledge information database; The external knowledge information, the text entity information, and the image entity information are processed to obtain the combined multimodal data; The step of determining the external knowledge information based on the text entity information, the image entity information, and the external knowledge information base includes: The text entity information and the image entity information are linked to an external knowledge information database using an entity linking tool, thereby determining the external knowledge information that matches the text entity information and the image entity information; The combined multimodal data includes target text information and target image information; the process of processing the external knowledge information, the text entity information, and the image entity information to obtain the combined multimodal data includes: The external knowledge information and the text entity information are concatenated to obtain the target text information; The target image information is determined by performing target data processing on the image entity information.

2. The method according to claim 1, characterized in that, Determining the inter-modal similarity score of the combined multimodal data includes: The target image information and the target text information are fused using a visual language model to obtain target fake news data; The target fake news data is classified based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data.

3. The method according to claim 2, characterized in that, The step of determining the corresponding detection result based on the inter-modal similarity score and a preset similarity score threshold includes: If the intermodal similarity score is greater than or equal to the similarity score threshold, the multimodal fake news data to be detected is determined to be real news data; If the intermodal similarity score is less than the similarity score threshold, the multimodal fake news data to be detected is determined to be fake news data.

4. A fake news detection device, characterized in that, The device includes: The acquisition module is used to acquire multimodal fake news data to be detected; An extraction module is used to extract external knowledge information that matches the multimodal fake news data to obtain combined multimodal data; The first determining module is used to determine the intermodal similarity score of the combined multimodal data; The second determining module is used to determine the corresponding detection result based on the inter-modal similarity score and the preset similarity score threshold; The extraction module includes: An extraction submodule is used to extract image entity information and text entity information from the multimodal fake news data, respectively; The first determining submodule is used to determine the external knowledge information based on the text entity information, the image entity information, and the external knowledge information database. The data processing submodule is used to process the external knowledge information, the text entity information and the image entity information to obtain the combined multimodal data. The first determining submodule includes: The first determining unit is used to link the text entity information and the image entity information to an external knowledge information database through an entity linking tool, and to determine the external knowledge information that matches the text entity information and the image entity information; The combined multimodal data includes target text information and target image information; the data processing submodule includes: The splicing processing unit is used to splice the external knowledge information with the text entity information to obtain the target text information; The second determining unit is used to perform target data processing on the image entity information to determine the target image information.

5. The apparatus according to claim 4, characterized in that, The first determining module includes: The fusion processing submodule is used to fuse the target image information and the target text information through a visual language model to obtain target fake news data; The classification submodule is used to classify the target fake news data based on a preset activation function to obtain the intermodal similarity score corresponding to the multimodal fake news data; The second determining module includes: The second determining submodule is used to determine the multimodal fake news data to be detected as real news data when the intermodal similarity score is greater than or equal to the similarity score threshold. The third determining submodule is used to determine the multimodal fake news data to be detected as fake news data when the intermodal similarity score is less than the similarity score threshold.

6. An electronic device, characterized in that, include: One or more processors; The device and one or more machine-readable media thereon storing instructions, which, when executed by the one or more processors, cause the device to perform the fake news detection device as described in claim 4 or 5.

Images