False information detection method, device, equipment and medium

Abstract

The present disclosure provides a false information detection method, including: obtaining social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected; when the image to be detected contains a second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected; extracting a feature vector of the image to be detected from the image to be detected by combining semantic features in the spatial domain and physical features in the frequency domain; fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and inputting the fused feature vector to be detected into a pre-trained classification model to output a detection result. The present disclosure also provides a false information detection device, equipment, storage medium and program product.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically to a false information detection method, device, equipment, medium and product. Background Art

[0002] With the increasing popularity of various social media, various types of information (such as text, pictures, videos, audio, etc.) on social media have quickly become a hot spot for mass information consumption due to their fast transmission speed, multiple acquisition channels, and low threshold for generation. Rapid and efficient detection of false news is a technical problem that urgently needs to be solved. Summary of the invention

[0003] In view of the above problems, the present disclosure provides a false information detection method, apparatus, device, medium and program product.

[0004] According to a first aspect of the present disclosure, a false information detection method is provided, comprising:

[0005] Acquire social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected;

[0006] In the case where the image to be detected includes the second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected;

[0007] Combining the semantic features in the spatial domain and the physical features in the frequency domain, a feature vector of the image to be detected is extracted from the image to be detected;

[0008] Fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and

[0009] The fused feature vector to be detected is input into the pre-trained classification model and the detection result is output.

[0010] According to an embodiment of the present disclosure, when the image to be detected includes a second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected includes:

[0011] Encoding the first text to be detected so as to add a text identifier to be detected, wherein the text identifier to be detected includes: a sentence of the first text to be detected, a sentence separator in the first text to be detected, and a feature vector to be detected;

[0012] Extracting the feature vector to be detected to obtain information of the first text to be detected;

[0013] In a case where the image to be detected contains a second text to be detected, extracting information about the second text to be detected from the image to be detected;

[0014] Using the text separator, splicing the information of the second text to be detected with the information of the first text to be detected to obtain spliced ​​information;

[0015] Extract the feature vector of the text to be detected from the splicing information.

[0016] According to an embodiment of the present disclosure, combining semantic features in the spatial domain and physical features in the frequency domain, extracting a feature vector of the image to be detected from the image to be detected includes:

[0017] Input the image to be detected into the convolutional neural network and output the matrix of physical features in the frequency domain;

[0018] Based on the matrix of physical features in the frequency domain, the long short-term memory model is used to extract semantic features in the spatial domain to obtain a weight matrix;

[0019] Based on the matrix of physical features in the frequency domain and the weight matrix, the feature vector of the image to be detected is obtained.

[0020] According to an embodiment of the present disclosure, the pre-trained classification model is obtained by a pre-training method, and the pre-training method includes:

[0021] Obtaining target social media information generated within a preset time interval, wherein the target social media information includes false information, true information, and information labels, and both the false information and the true information include an image and a first text;

[0022] When the image contains a second text, extracting a text feature vector from the first text and the second text;

[0023] Combining the semantic features in the spatial domain and the physical features in the frequency domain, an image feature vector is extracted from the image;

[0024] Fusing the text feature vector and the image feature vector to obtain a fused feature vector;

[0025] Inputting the fused feature vector into the classification model to obtain the classification result; and

[0026] The parameters of the classification model are adjusted based on the classification results and information labels, and the adjusted classification model is used as the pre-trained classification model.

[0027] According to an embodiment of the present disclosure, obtaining target social media information generated within a preset time interval includes:

[0028] Obtaining raw social media information generated within a preset time interval;

[0029] Preprocessing the original social media information to obtain preprocessed social media information;

[0030] Cluster the preprocessed social media information to obtain true information clusters and false information clusters;

[0031] The target social media information is obtained by filtering from the real information cluster and the false information cluster respectively.

[0032] According to an embodiment of the present disclosure, the original social media information is preprocessed to obtain the preprocessed social media information including:

[0033] Filtering social media information including the image and the first text from the original social media information to obtain first processed social media information;

[0034] In the case where the first processed social media information includes a plurality of images, selecting a representative image as an image to obtain the preprocessed social media information;

[0035] and / or

[0036] According to a preset pixel threshold, the social media information after the first processing is screened to obtain pre-processed social media information.

[0037] A second aspect of the present disclosure provides a false information detection device, comprising:

[0038] An acquisition module, used to acquire social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected;

[0039] A first extraction module is used to extract a feature vector of the text to be detected from the first text to be detected and the second text to be detected when the image to be detected contains the second text to be detected;

[0040] The second extraction module is used to extract a feature vector of the image to be detected from the image to be detected by combining the semantic features in the spatial domain and the physical features in the frequency domain;

[0041] A fusion module, used for fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and

[0042] The detection module is used to input the fused feature vector to be detected into a pre-trained classification model and output a detection result.

[0043] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned false information detection method.

[0044] The fourth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned false information detection method.

[0045] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned false information detection method when executed by a processor.

[0046] According to the embodiments of the present disclosure, feature vectors of the text to be detected and the image to be detected are extracted respectively, wherein the extraction of the feature vector of the text to be detected introduces the text information in the image, and the extraction of the feature vector of the image to be detected introduces the semantic features in the spatial domain and the physical features in the frequency domain. The feature vectors are then fused and passed through a pre-trained classification model to obtain the detection result, thereby jointly solving the problems of incomplete information representation and loss of text semantics during the information detection process. The detection speed is fast and the detection accuracy is high, which is conducive to the rapid and efficient detection of social media information. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0048] Figure 1 The application scenario diagram of the false information detection method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown;

[0049] Figure 2 A flowchart of a false information detection method according to an embodiment of the present disclosure is schematically shown;

[0050] Figure 3 A flowchart of a method for extracting a feature vector of a text to be detected from a first text to be detected and a second text to be detected when the image to be detected contains a second text to be detected according to an embodiment of the present disclosure is schematically shown;

[0051] Figure 4 A flowchart of a method for extracting a feature vector of an image to be detected from an image to be detected by combining semantic features in the spatial domain and physical features in the frequency domain according to an embodiment of the present disclosure is schematically shown;

[0052] Figure 5 The structure diagram of a CNN-LSTM network with an attention mechanism according to an embodiment of the present disclosure is schematically shown;

[0053] Figure 6 A flowchart of a false information detection method according to another embodiment of the present disclosure is schematically shown;

[0054] Figure 7A flowchart of a method for pre-training a pre-trained classification model according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 8 A training diagram of a GBDT classification model according to an embodiment of the present disclosure is schematically shown;

[0056] Fig. 9 A structural block diagram of a false information detection device according to an embodiment of the present disclosure is schematically shown; and

[0057] Fig.10 A block diagram of an electronic device suitable for implementing a false information detection method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0058] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0059] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0060] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0061] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0062] In the technical solution disclosed in the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.

[0063] For false information detection, it is mainly carried out by manually extracting text features, user features, communication features and other potential attribute features, and constructing classifiers. This feature extraction-based method can make full use of the experience and knowledge summarized by experts, but there are also certain problems. This process requires manual feature extraction, and the technology, means and forms of false information generation are constantly updated, while expert knowledge and experience have a certain lag, making it difficult to keep up with the times to deal with new forms of false information. Social media content carriers are diverse and information is relatively scattered. There is no unified form to effectively integrate these highly related information.

[0064] Since deep learning models can automatically learn the features contained in the data set and abandon the tedious feature construction process, deep neural networks can learn the feature representation of images and sentences more accurately than traditional manual extraction. Based on different deep learning models, the research extracts features of different modal data and deeply fuses them to solve the problem of insufficient information representation in single-modal news, and has achieved certain results in multimodal false information detection. For example, a false news detection model attRNN that extracts image, text and social background features fuses text and images through an attention mechanism and finally classifies them. On this basis, a false news detection model EANN based on adversarial neural network is also established to learn the common features of events, thereby improving the detection accuracy. Multimodal variational autoencoder MVAE, which aims to discover the correlation between each modality, so as to obtain a better multimodal feature fusion representation for detection tasks. A simpler framework SpotFake directly uses a pre-trained model to simply fuse the extracted text and image features for classification. However, these models can only perform simple concatenation and fusion of multimodal feature vectors in the detection module, and then directly use softmax as a simple classifier. In this way, the fused features may contain redundant and invalid features or lack important features, resulting in poor generalization and low accuracy.

[0065] Based on this, an embodiment of the present disclosure provides a method for detecting false information, including: obtaining social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected; when the image to be detected contains a second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected; extracting a feature vector of the image to be detected from the image to be detected by combining semantic features in the spatial domain and physical features in the frequency domain; fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and inputting the fused feature vector to be detected into a pre-trained classification model to output a detection result.

[0066] Figure 1The application scenario diagram of the false information detection method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0067] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0068] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as financial product applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0069] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0070] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0071] It should be noted that the false information detection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the false information detection device provided in the embodiment of the present disclosure can generally be set in the server 105. The false information detection method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the false information detection device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0072] The false information detection method provided in the embodiment of the present disclosure may also be executed by the terminal devices 101, 102, 103. Accordingly, the false information detection apparatus provided in the embodiment of the present disclosure may also be generally arranged in the terminal devices 101, 102, 103. The false information detection method provided in the embodiment of the present disclosure may also be executed by other terminals different from the terminal devices 101, 102, 103. Accordingly, the false information detection apparatus provided in the embodiment of the present disclosure may also be arranged in other terminals different from the terminal devices 101, 102, 103.

[0073] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0074] The following will be based on Figure 1 The scene described by Figure 2~Figure 8 The false information detection method of the embodiment of the present disclosure is described in detail.

[0075] Figure 2 The flowchart of the false information detection method according to the embodiment of the present disclosure is schematically shown.

[0076] like Figure 2 As shown, the false information detection method 200 of this embodiment includes operations S201 to S205.

[0077] In operation S201, social media information to be detected is obtained, wherein the social media information to be detected includes an image to be detected and a first text to be detected.

[0078] According to an embodiment of the present disclosure, the social media information to be detected may include Weibo platform information, Twitter platform information, Facebook platform information, Zhihu platform information, and information disclosed by other APPs.

[0079] In operation S202, when the image to be detected includes the second text to be detected, a feature vector of the text to be detected is extracted from the first text to be detected and the second text to be detected.

[0080] According to an embodiment of the present disclosure, when the detection software detects that the image to be detected contains the second text to be detected, a text feature vector extractor can be used to extract a feature vector of the text to be detected from the first text to be detected and the second text to be detected.

[0081] In operation S203, a feature vector of the image to be detected is extracted from the image to be detected by combining the semantic features in the spatial domain and the physical features in the frequency domain.

[0082] According to the embodiments of the present disclosure, since false information and real information have different characteristics at both the physical and semantic levels: at the physical level, the pixels of the image in the false information may be very low, which can be clearly reflected in the frequency domain, specifically using the physical feature representation in the frequency domain, such as the periodic features on the frequency band; at the semantic level, the image in the false information also exhibits some obvious characteristics in the pixel domain (also called the spatial domain), for example, the publishers of false information tend to use the stimulating colors of the image to attract and mislead readers for rapid dissemination, and usually show exaggerated visual effects and emotional provocations, specifically using the semantic feature representation in the spatial domain, such as color features.

[0083] According to an embodiment of the present disclosure, a semantic feature weight matrix in the spatial domain may be constructed through the spatial domain, and then a feature vector of the image to be detected may be captured in the frequency domain.

[0084] In operation S204, the feature vector of the text to be detected and the feature vector of the image to be detected are fused to obtain a fused feature vector to be detected.

[0085] According to the embodiments of the present disclosure, in order to effectively merge the two features of the to-be-detected text feature vector and the to-be-detected image feature vector and ensure the consistency of the features, linear connection may be selected for fusion.

[0086] For example, the obtained fused feature vector C to be detected can be expressed as shown in formula (1):

[0087] (1)

[0088] in, represents the feature vector of the text to be detected, Represents the feature vector of the image to be detected, symbol Represents the operation of connecting the feature vector of the text to be detected and the feature vector of the image to be detected.

[0089] In operation S205, the fused feature vector to be detected is input into a pre-trained classification model, and a detection result is output.

[0090] According to an embodiment of the present disclosure, the detection result may be true information or false information.

[0091] According to the disclosed embodiment, feature vectors of the text to be detected and the image to be detected are extracted respectively, wherein the extraction of the feature vector of the text to be detected introduces the text information in the image, and the extraction of the feature vector of the image to be detected introduces the semantic features in the spatial domain and the physical features in the frequency domain. The feature vectors are then fused and passed through a pre-trained classification model to obtain the detection result, thereby jointly solving the problems of incomplete information representation and loss of text semantics during information detection. The detection speed is fast and the detection accuracy is high, which is conducive to the rapid and efficient detection of social media information.

[0092] Figure 3 A flowchart of a method for extracting a feature vector of a text to be detected from a first text to be detected and a second text to be detected when an image to be detected contains a second text to be detected according to an embodiment of the present disclosure is schematically shown.

[0093] like Figure 3 As shown, in this embodiment, when the image to be detected contains the second text to be detected, the method 300 for extracting the feature vector of the text to be detected from the first text to be detected and the second text to be detected includes operations S301 to S305.

[0094] In operation S301, the first text to be detected is encoded so as to add a text identifier to be detected, wherein the text identifier to be detected includes: a sentence of the first text to be detected, a sentence separator in the first text to be detected, and a feature vector to be detected.

[0095] According to an embodiment of the present disclosure, the first text to be detected can be expressed as shown in formula (2):

[0096] (2)

[0097] Among them, w n Represents the nth word in the first text to be detected.

[0098] Encode the first text to be detected T, and then add the sentence of the first text to be detected , the sentence separator in the first text to be detected , feature vector to be detected And other signs.

[0099] In operation S302, a feature vector to be detected is extracted to obtain information of a first text to be detected.

[0100] According to an embodiment of the present disclosure, by extracting the feature vector to be detected , and use it as the information of the first text to be detected. For example, the enhanced representation model of knowledge integration (ERNIE) can be used for encoding, tagging and extraction, and the The corresponding 768-dimensional feature vector to be detected is used as the information representation x of the first text to be detected t , as shown in formula (3):

[0101] (3)

[0102] In operation S303, when the image to be detected includes the second text to be detected, information about the second text to be detected is extracted from the image to be detected.

[0103] According to an embodiment of the present disclosure, the information of the second text to be detected in the image can be extracted through the OCR text detection model pre-trained by Baidu. After data preprocessing, for example, the second text to be detected in the image to be detected can be represented as a word sequence , as shown in formula (4):

[0104] (4)

[0105] Among them, m n Represents the nth word in the second text to be detected.

[0106] In operation S304, the information of the second text to be detected is concatenated with the information of the first text to be detected by using a text delimiter to obtain concatenated information.

[0107] According to an embodiment of the present disclosure, the semantic interaction between the first to-be-detected text and the second to-be-detected text can be separated by a text separator and spliced ​​into a sequence to obtain splicing information, which can be expressed as T O.

[0108] In operation S305, a feature vector of the text to be detected is extracted from the splicing information.

[0109] According to an embodiment of the present disclosure, the feature vector of the text to be detected is extracted from the splicing information. O is input into the ERNIE network to extract the feature vector x of the text to be detected to , as shown in formula (5):

[0110] (5)

[0111] According to an embodiment of the present disclosure, a feature vector of a text to be detected can be extracted based on the ERNIE (enhanced representation from knowledge integration) model. ERNIE is a knowledge-enhanced semantic representation model. The structure of ERNIE is similar to that of BERT, both of which use multi-layer Transformers as basic encoders and implement modeling of contextual information through a self-attention mechanism. Unlike BERT, ERNIE masks semantic units such as words and entities, and expands some knowledge-based Chinese corpora for pre-training, which can better model prior semantic knowledge such as entity concepts, thereby further improving the semantic representation ability of the model. ERNIE can not only be used as a context encoder to generate sentence expressions, but also as a knowledge storage device, implicitly utilizing a large amount of factual knowledge stored in the model when generating sentence expressions. Therefore, using ERNIE as a feature vector extractor for a text to be detected can simultaneously model the characteristics of the text to be detected at the presentation layer and the semantic layer.

[0112] According to the embodiments of the present disclosure, by interacting the semantics of text information in the social media information to be detected with the semantics of text information in the image to be detected, semantic features of the text to be detected are obtained, thereby solving the problems of incomplete information representation and semantic loss in the information detection model.

[0113] Figure 4 A flowchart of a method for extracting a feature vector of an image to be detected from an image to be detected by combining semantic features in the spatial domain and physical features in the frequency domain according to an embodiment of the present disclosure is schematically shown; Figure 5 The structure diagram of a CNN-LSTM network with an attention mechanism according to an embodiment of the present disclosure is schematically shown.

[0114] like Figure 4 As shown, the method 400 of extracting a feature vector of an image to be detected from an image to be detected by combining semantic features in the spatial domain and physical features in the frequency domain in this embodiment includes operations S401 to S403.

[0115] In operation S401, the image to be detected is input into a convolutional neural network, and a matrix of physical features in the frequency domain is output.

[0116] According to an embodiment of the present disclosure, the image to be detected may be preprocessed before being input into a convolutional neural network. For example, the image to be detected may be resized so that it automatically adapts to the input window size.

[0117] According to an embodiment of the present disclosure, the physical characteristics in the frequency domain may include periodic characteristics on the frequency segment.

[0118] In operation S402, based on the matrix of physical features in the frequency domain, a long short-term memory model is used to extract semantic features in the spatial domain to obtain a weight matrix.

[0119] According to an embodiment of the present disclosure, the matrix of physical features in the frequency domain output by the convolutional neural network can be input into the long short-term memory model to extract semantic features in the spatial domain to obtain a weight matrix, wherein the semantic features in the spatial domain may include color features.

[0120] In operation S403, a feature vector of the image to be detected is obtained based on the matrix of physical features in the frequency domain and the weight matrix.

[0121] According to the embodiment of the present disclosure, the matrix of physical features in the frequency domain may be multiplied by the weight matrix, and a reconstruction operation may be performed to obtain a feature vector of the image to be detected after weight allocation.

[0122] According to the embodiment of the present disclosure, the image to be detected can be input as follows: Figure 5 In the CNN-LSTM network structure with attention mechanism shown in the figure, the matrix of physical features in the frequency domain is output by the convolutional neural network. The matrix of physical features in the frequency domain output by the convolutional neural network is input into the LSTM, which extracts semantic features in the spatial domain and generates a weight matrix. The matrix of physical features in the frequency domain output by the CNN is multiplied by the weight matrix, and the feature vector of the image to be detected after weight distribution is obtained after the reconstruction operation. The modal features of the image to be detected are extracted based on the CNN-LSTM model, which can effectively combine the semantic level features in the pixel domain of the image to be detected and the physical level features in the frequency domain to construct a CNN-LSTM network with attention mechanism.

[0123] According to the embodiments of the present disclosure, a weight matrix is ​​constructed using the pixel domain (spatial domain) to purposefully capture the features of the image to be detected in the frequency domain. This can effectively extract the feature vector of the image to be detected, which is beneficial to the representation of false information and real information in the image in the information detection model and improves the accuracy of the model.

[0124] Figure 6 The following schematically shows a flow chart of a false information detection method according to another embodiment of the present disclosure.

[0125] like Figure 6 As shown, the false information detection method 600 of this embodiment includes operations S601 to S609.

[0126] In operation S601, social media information to be detected is obtained, wherein the social media information to be detected includes an image to be detected and a first text to be detected.

[0127] According to an embodiment of the present disclosure, the social media information to be detected may include Weibo platform information, Twitter platform information, Facebook platform information, Zhihu platform information, and information disclosed by other APPs.

[0128] In operation S602, when the first text to be detected in the social media information to be detected and the second text to be detected is included in the image to be detected, the second text to be detected in the social media information to be detected is encoded.

[0129] According to an embodiment of the present disclosure, when the detection software detects that the image to be detected contains the second text to be detected, the first text to be detected and the second text to be detected can be encoded by a text feature vector encoder.

[0130] In operation S603, a text vector is obtained.

[0131] In operation S604, the text vector is input into the ERNIE model to extract a feature vector of the text to be detected.

[0132] In operation S605 , the image in the social media information to be detected is encoded.

[0133] In operation S606, an image vector is obtained.

[0134] In operation S607, the image vector is input into the CNN-LSTM model to extract a feature vector of the image to be detected.

[0135] According to the embodiment of the present disclosure, semantic features in the spatial domain and physical features in the frequency domain are introduced when extracting feature vectors of the image to be detected.

[0136] In operation S608, the feature vector of the text to be detected and the feature vector of the image to be detected are fused to obtain a fused vector to be detected.

[0137] In operation S609, the fusion vector to be detected is input into the trained GBDT classifier, and a classification result, ie, a detection result, is output.

[0138] Figure 7 A flowchart of a method for pre-training a pre-trained classification model according to an embodiment of the present disclosure is schematically shown.

[0139] like Figure 7 As shown, the pre-trained classification model of this embodiment includes operations S701 to S706 through the pre-training method 700.

[0140] In operation S701, target social media information generated within a preset time interval is obtained, wherein the target social media information includes false information, true information, and information tags, and both the false information and the true information include an image and a first text.

[0141] According to an embodiment of the present disclosure, the preset time interval may be a period of time in the past. Social media information may include Weibo platform information, WeChat platform information, QQ platform information, Zhihu platform information, etc. Social media information generated within the preset time interval may be obtained through online data acquisition software. The target social media information is obtained by processing the social media information. Information tags may be obtained through official certification.

[0142] For example, the past period of time may be the past month, such as 2021.05.01~2021.05.31, the past six months, such as 2021.03.01~2021.08.31, the past year, such as 2021.01.01~2021.12.31, etc. Officially certified false information and real information generated between 2021.01.01 and 2021.12.31 can be obtained through online data acquisition software. After preprocessing such as retaining the information containing both the image and the first text, the target social media information is obtained.

[0143] In operation S702, when the image contains the second text, a text feature vector is extracted from the first text and the second text.

[0144] According to an embodiment of the present disclosure, the second text in the image and the first text in the target social media information may be extracted by a text extraction model to obtain a text feature vector.

[0145] For example, a sentence extracted from a first text in the target social media information may be encoded to obtain a text feature vector of the first text. A model for extracting text from an image may be used to directly extract a second text from the image to obtain a text feature vector of the second text.

[0146] In operation S703, an image feature vector is extracted from the image by combining the semantic features in the spatial domain and the physical features in the frequency domain.

[0147] In operation S704, the text feature vector and the image feature vector are fused to obtain a fused feature vector.

[0148] In operation S705, the fused feature vector is input into a classification model to obtain a classification result.

[0149] According to an embodiment of the present disclosure, the classification model may be a GBDT classification model or other binary classification models. The classification result may be true information or false information.

[0150] In operation S706, the parameters of the classification model are adjusted based on the classification results and the information labels, and the adjusted classification model is used as a pre-trained classification model.

[0151] According to the embodiments of the present disclosure, the error value of the model can be calculated based on the classification results and the information labels. If the error value is large and does not meet the preset threshold, the model parameters are adjusted and retrained until the error value meets the preset threshold to obtain a trained classification model, and the trained classification model is used as a pre-trained classification model. The preset threshold can be determined based on the accuracy of the actual training model. The trained classification model can also be obtained by calculating the loss function until the model converges.

[0152] According to the embodiments of the present disclosure, feature vectors of text and image are extracted respectively, wherein the extraction of text feature vector introduces text information in the image, and the extraction of image feature vector introduces semantic features in the spatial domain and physical features in the frequency domain. Together, the problem of incomplete information representation of the pre-trained classification model and loss of text semantics is solved. By fusing the features of text and image and inputting them into the classification model, the accuracy of the trained classification model is improved.

[0153] According to an embodiment of the present disclosure, obtaining target social media information generated within a preset time interval includes:

[0154] Obtaining raw social media information generated within a preset time interval;

[0155] Preprocessing the original social media information to obtain preprocessed social media information;

[0156] Cluster the preprocessed social media information to obtain true information clusters and false information clusters;

[0157] The target social media information is obtained by filtering from the real information cluster and the false information cluster respectively.

[0158] Among them, the original social media information can be obtained based on the fake news dataset built on the Chinese Sina Weibo platform. This dataset contains all the news messages officially certified as fake from 2012.05 to 2016.01 on the Weibo official rumor reporting platform, as well as the Weibo messages of real news of the same period collected from the Xinhua News Agency's hot news discovery system. The original social media information is preprocessed, such as deleting news posts with unqualified image pixels, deleting news posts containing only text or pictures, and saving only one representative picture for news posts with multiple illustrations.

[0159] According to an embodiment of the present disclosure, the pre-processed social media information is clustered to obtain true information clusters and false information clusters. The clustering can be performed with false information and true information as cluster centers, respectively. The target social media information is screened from the true information cluster and the false information cluster, respectively. The clustered true information cluster and the false information cluster can be divided proportionally according to a certain ratio to obtain a training set, a verification set, and a test set. This avoids the problem of model overfitting caused by the fact that the training data, verification data, and test data may contain the same information. The training set is screened as the target social media information.

[0160] It should be noted that post-clustering screening can improve the generalization ability of the model.

[0161] For example, when the certain ratio is 3:1:1, the relevant data indicators of the final training set, validation set and test set are shown in Table 1 below.

[0162]

[0163] According to an embodiment of the present disclosure, the original social media information is preprocessed to obtain the preprocessed social media information including:

[0164] Filtering social media information including an image and a first text from the original social media information to obtain first processed social media information; when the first processed social media information includes multiple images, filtering a representative image as an image to obtain preprocessed social media information;

[0165] and / or

[0166] According to a preset pixel threshold, the social media information after the first processing is screened to obtain the pre-processed social media information, wherein the preset pixel threshold can be determined according to the performance of the actual feature extraction model.

[0167] For example, social media information that only contains the first text or image can be deleted. Social media information can include news posts. In the case of containing both the first text and the image, news posts with unqualified image pixels can be deleted. If the news post contains both the first text and the image, and the image has multiple illustrations, only one representative image can be saved for the news post with multiple illustrations.

[0168] Figure 8 The training diagram of the GBDT classification model according to an embodiment of the present disclosure is schematically shown.

[0169] According to the embodiments of the present disclosure, Figure 8 As shown, the classification model can be a GBDT classification model.

[0170] For the GBDT classification model, the training data D that integrates the feature vector and the information label can be selected as the feature input of the GBDT classification model to build a tree model, iteratively train to form a strong learner, and obtain the training result, that is, true information or false information, by calculating the output.

[0171] For example, the obtained fusion feature vector samples and information labels can be used to form a training data set as formula (6):

[0172] (6)

[0173] Among them, x n represents the nth fused feature vector of the input; y n is the information label corresponding to the nth fused feature vector.

[0174] In this way, a tree model is constructed and iterative training is performed to form a strong learner. , the probability of the final training result is given by The output is as shown in formula (7):

[0175] (7)

[0176] Among them, the loss function can adopt the log-likelihood loss function, as shown in formula (8):

[0177] (8)

[0178] According to the embodiment of the present disclosure, the fused feature vector is input into GBDT for training, and GBDT combines the input features, calculates the probability, and outputs the classification result of true and false information. The threshold for classifying true and false information can be determined as 50%. If the calculated probability is greater than this threshold, it is false information, and if it is less than this threshold, it is true information.

[0179] According to the disclosed embodiment, the classification model combines the input feature vectors to improve the detection accuracy and enhance the generalization, and can better identify the authenticity of the information. However, directly using softmax as a classifier will cause feature semantic loss and low generalization, which will limit the identification ability of the model.

[0180] According to the embodiment of the present disclosure, the trained classification model can also be evaluated. For example, the accuracy, precision, recall and The four indicators of value are used to effectively evaluate the results produced by the trained classification model. The results can be divided into four categories, namely TP (True Positives), TN (True Negatives), FP (False Positives) and FN (False Negatives), among which TP means that it is a positive class in the information label and is actually classified as a positive class. TN means that it is a negative class in the information label and is actually classified as a negative class. FP means that it is a negative class in the information label, but is classified as a positive class in the actual classification. FN means that it is a positive class in the information label, but is classified as a negative class in the actual classification. Based on these four categories, the four indicators of accuracy (Accuracy), precision (Precision), recall (Recall) and The calculation methods of the values ​​are as follows:

[0181] (9)

[0182] (10)

[0183] (11)

[0184] (12)

[0185]

[0186] It should be noted that the hardware environment requirements for completing the embodiments of the present disclosure are relatively high, for example, it can be completed in the experimental environment shown in Table 2.

[0187] According to the embodiments of the present disclosure, a comparative evaluation may also be performed on models that may be used in the false information detection method.

[0188] For example, (1) based on the single text modality model:

[0189] TextCNN: Text classification based on convolutional neural network. Three different sizes of convolution kernels are used, with heights of 3, 4, and 5. The number of convolution kernels of each type is set to 100.

[0190] BERT: Pre-trained language model. The BERT model fine-tuned on the text data of this public training set is used for comparison. The pre-trained BERT model bert-base-chinese comes from the GitHub open source project Transformer.

[0191] ERNIE: The ERNIE model fine-tuned on the text data of this public training set is used for comparison. The pre-trained ERNIE model nghuyong / ernie-1.0 comes from the GitHub open source project Trans-former.

[0192] (2) Based on single visual modality model:

[0193] VGG19: widely used as a visual feature extractor. The VGG19 model pre-trained on the ImageNet dataset is fine-tuned on the image data in this public training set.

[0194] ResNet152: The ResNet152 model pre-trained on the ImageNet dataset is fine-tuned on the image data in this public training set.

[0195] CNN-LSTM: The CNN-LSTM model pre-trained on the ImageNet dataset is fine-tuned on the image data in this public training set.

[0196] (3) Multimodal-based models

[0197] attRNN: A recurrent neural network based on the attention mechanism, which is used to integrate the features of three modalities: text, vision, and social context. The text part is modeled using LSTM, and the image part is extracted using pre-trained VGG19. In order to ensure fairness in the comparison, the part that processes social features is removed in the specific implementation.

[0198] EANN: A neural network based on event adversarial mechanism. By introducing event classifier as an auxiliary task, the model is guided to learn multimodal features that are unrelated to events. The model uses TextCNN and pre-trained VGG19 to extract text and visual modal features respectively, and splices the two modal features as multimodal feature expressions of fake news, which are input into the fake news classifier and news event classifier.

[0199] KMGCN: A knowledge-guided multimodal graph convolutional network. The concepts corresponding to the named entities appearing in the text are extracted from the external encyclopedia knowledge graph as external knowledge. A graph is constructed for each input multimodal news. The nodes of the graph include the words in the text, the concepts corresponding to the text entities, and the names of the objects recognized in the picture. The nodes are initialized by pre-trained Word2Vec word vectors, and the weights of the edges are set to the PMI values ​​of the two words. The graph expression is obtained by a two-layer graph convolutional network and maximum pooling for fake news classification.

[0200] Based on the above model, the experimental results shown in Table 3 can be obtained.

[0201] Based on the above experimental results, it can be seen that the present disclosure uses the ERNIE+CNN-LSTM multimodal model for feature extraction, which significantly exceeds other comparative methods in classification accuracy, indicating that the false information detection method based on deep feature fusion and semantic understanding proposed in the present disclosure can indeed effectively improve the performance of information detection. In particular, in terms of information recall rate, the method proposed in the present disclosure exceeds other methods by more than 7 percentage points, indicating that the false information detection method provided by the embodiment of the present disclosure can detect omitted information by fully mining multimodal semantic clues.

[0202]

[0203] Among the multimodal methods, KMGCN is significantly lower than other comparison methods. The main reason may be that GCN has poor modeling capabilities for short texts such as social media information such as Weibo, and cannot well reflect the role of external knowledge on this basis. In addition, KMGCN only extracts object label information in the image, which is insufficient for image semantic modeling.

[0204] The method based on single text modality is better than the method based on single visual modality, indicating that false information detection mainly relies on text clues. The method based on multimodality is better than the single modality method with the same subnetwork structure, indicating that text and image modalities can provide complementary clues for false information detection tasks. Among them, the ERNIE+CNN-LSTM multimodal model used in this disclosure has an accuracy improvement of 4.3 percentage points compared with ERNIE, further proving the importance of image semantic features.

[0205] Among the methods based on single text modality, pre-trained language models are superior to traditional text modeling methods such as CNN and RNN. This improvement comes from the more powerful modeling ability of Transformer on the one hand, and the linguistic knowledge learned by pre-trained language models from a large amount of pre-training corpus on the other. ERNIE performs better than BERT, which shows that adding entity concept knowledge can enhance the semantic understanding of news and thus improve the detection effect of information.

[0206] Based on the above false information detection method, the present disclosure also provides a false information detection device. Fig. 9 The device is described in detail.

[0207] Fig. 9 The structural block diagram of the false information detection device according to an embodiment of the present disclosure is schematically shown.

[0208] like Fig. 9As shown, the false information detection device 900 of this embodiment includes an acquisition module 910, a first extraction module 920, a second extraction module 930, a fusion module 940 and a detection module 950.

[0209] The acquisition module 910 is used to acquire the social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected. In one embodiment, the acquisition module 910 can be used to perform the operation S201 described above, which will not be described in detail here.

[0210] The first extraction module 920 is used to extract the feature vector of the text to be detected from the first text to be detected and the second text to be detected when the image to be detected contains the second text to be detected. In one embodiment, the first extraction module 920 can be used to perform the operation S202 described above, which will not be repeated here.

[0211] The second extraction module 930 is used to extract the feature vector of the image to be detected from the image to be detected by combining the semantic features in the spatial domain and the physical features in the frequency domain. In one embodiment, the second extraction module 930 can be used to perform the operation S203 described above, which will not be repeated here.

[0212] The fusion module 940 is used to fuse the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected. In one embodiment, the fusion module 940 can be used to perform the operation S204 described above, which will not be described in detail here.

[0213] The detection module 950 is used to input the fused feature vector to be detected into a pre-trained classification model and output a detection result. In one embodiment, the detection module 950 can be used to perform the operation S205 described above, which will not be described in detail here.

[0214] According to an embodiment of the present disclosure, any multiple modules of the acquisition module 910, the first extraction module 920, the second extraction module 930, the fusion module 940 and the detection module 950 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the first extraction module 920, the second extraction module 930, the fusion module 940 and the detection module 950 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 910, the first extraction module 920, the second extraction module 930, the fusion module 940 and the detection module 950 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0215] Fig.10 A block diagram of an electronic device suitable for implementing an information detection model training method and an information detection method according to an embodiment of the present disclosure is schematically shown.

[0216] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1009 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0217] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0218] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1008 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1009 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1009 as needed.

[0219] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0220] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0221] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0222] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0223] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0224] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0225] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0226] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0227] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0228] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A false information detection method, comprising: Acquire social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected; In a case where the image to be detected includes a second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected; Combining the semantic features in the spatial domain and the physical features in the frequency domain, extracting a feature vector of the image to be detected from the image to be detected; Fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and Inputting the fused feature vector to be detected into a pre-trained GBDT classification model, so that the GBDT classification model combines the fused feature vector to be detected and the information label corresponding to the fused feature vector to be detected to calculate the probability, and outputs the detection result based on the probability; The step of combining the semantic features in the spatial domain and the physical features in the frequency domain to extract the feature vector of the image to be detected from the image to be detected comprises: Inputting the image to be detected into a convolutional neural network, and outputting a matrix of physical features in the frequency domain; Inputting the matrix of physical features in the frequency domain into a long short-term memory model to extract semantic features in the spatial domain and generate a weight matrix; The matrix of physical features in the frequency domain is multiplied by the weight matrix, and a reconstruction operation is performed to obtain the feature vector of the image to be detected after weight allocation.

2. The method according to claim 1, wherein: In the case where the image to be detected includes a second text to be detected, extracting a feature vector of the text to be detected from the first text to be detected and the second text to be detected comprises: Encoding the first text to be detected by using the knowledge-integrated enhanced representation model so as to add a text identifier to be detected, wherein the text identifier to be detected includes: a sentence of the first text to be detected, a sentence separator in the first text to be detected, and a feature vector to be detected; Extracting the feature vector to be detected to obtain information of the first text to be detected; In a case where the image to be detected contains the second text to be detected, extracting information about the second text to be detected from the image to be detected; Using a text delimiter, concatenating the information of the second text to be detected with the information of the first text to be detected to obtain concatenated information; The feature vector of the text to be detected is extracted from the splicing information.

3. The method according to claim 1, wherein: The pre-trained classification model is obtained by a pre-training method, and the pre-training method includes: Acquire target social media information generated within a preset time interval, wherein the target social media information includes false information, true information, and information tags, and both the false information and the true information include an image and a first text; In the case where the image contains a second text, extracting a text feature vector from the first text and the second text; extracting an image feature vector from the image by combining the semantic features in the spatial domain and the physical features in the frequency domain; Fusing the text feature vector and the image feature vector to obtain a fused feature vector; Inputting the fused feature vector into a classification model to obtain a classification result; and The parameters of the classification model are adjusted based on the classification result and the information label, and the classification model after the adjustment is used as the pre-trained classification model.

4. The method according to claim 3, wherein: The step of obtaining target social media information generated within a preset time interval includes: Obtaining raw social media information generated within a preset time interval; Preprocessing the original social media information to obtain preprocessed social media information; Clustering the preprocessed social media information to obtain true information clusters and false information clusters; The target social media information is obtained by filtering from the real information cluster and the false information cluster respectively.

5. The method according to claim 4, wherein: The preprocessing of the original social media information to obtain the preprocessed social media information includes: Filtering social media information including the image and the first text from the original social media information to obtain first processed social media information; In the case where the first processed social media information includes a plurality of images, selecting a representative image as the image to obtain the pre-processed social media information; and / or The pre-processed social media information is obtained by filtering the first processed social media information according to a preset pixel threshold.

6. A false information detection device, comprising: An acquisition module, configured to acquire social media information to be detected, wherein the social media information to be detected includes an image to be detected and a first text to be detected; A first extraction module, configured to extract a feature vector of a text to be detected from the first text to be detected and the second text to be detected when the image to be detected contains a second text to be detected; A second extraction module is used to extract a feature vector of the image to be detected from the image to be detected by combining the semantic features in the spatial domain and the physical features in the frequency domain; A fusion module, used for fusing the feature vector of the text to be detected and the feature vector of the image to be detected to obtain a fused feature vector to be detected; and A detection module, used for inputting the fused feature vector to be detected into a pre-trained GBDT classification model, so that the GBDT classification model calculates the probability after combining the fused feature vector to be detected and the information label corresponding to the fused feature vector to be detected, and outputs the detection result based on the probability; The step of combining the semantic features in the spatial domain and the physical features in the frequency domain to extract the feature vector of the image to be detected from the image to be detected comprises: Inputting the image to be detected into a convolutional neural network, and outputting a matrix of physical features in the frequency domain; Inputting the matrix of physical features in the frequency domain into a long short-term memory model to extract semantic features in the spatial domain and generate a weight matrix; The matrix of physical features in the frequency domain is multiplied by the weight matrix, and a reconstruction operation is performed to obtain the feature vector of the image to be detected after weight allocation.

7. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images