Multi-modal rumor detection method based on metric learning

Abstract

The invention discloses a multi-mode rumor detection method based on metric learning, and belongs to the technical field of natural language processing. The method comprises the following steps: firstly, carrying out feature extraction on a text and a picture by respectively adopting pre-trained BERT and VGG-19 models, then mining a local feature relationship of each modal by using syntactic analysis and adaptive attention, and then converting two modal features into same-dimension vectors in spatial mapping; the method comprises the steps that firstly, feature relations between modals and in the modals are determined in sequence by using comparative learning and triple learning in metric learning, finally, feature vectors of all the modals added with local feature relations are input into a feature fusion unit for feature fusion, and the fused features are used for information detection of a rumor detector. According to the method, the rumor detection accuracy is further improved through local feature connection in the modals and feature interaction between the modals.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a multimodal rumor detection method based on metric learning. Background Art

[0002] Compared with traditional rumors, online rumors have the characteristics of fast propagation speed, wide impact range, and multiple presentation methods. Once rumors spread, they will bring social unrest, cause panic among people, and even cause unnecessary economic losses. Therefore, timely and effective identification of online rumors is of great significance to maintaining social order and protecting people's property safety. At present, most information on the Internet is presented in multimodal forms such as text + pictures. Rumors spread in a multimodal way are more confusing than those spread in a single mode. The amount of information contained in multimodal rumors is much larger than that of single text rumors. Therefore, the method of detecting rumors only by relying on text information can no longer accurately and effectively identify rumors in the detection of multimodal rumors. As for multimodal information, existing studies only consider the splicing method between different modal features, ignoring the feature connection within the modality and the feature constraint relationship between the modalities. In view of this, it is necessary to design a multimodal rumor detection method based on metric learning to solve the above problems. Summary of the invention

[0003] The purpose of the present invention is to provide a multimodal rumor detection method based on metric learning, which can effectively capture the feature relationship between modalities and modalities, and achieve more accurate rumor identification.

[0004] To achieve the above objectives, the present invention provides a multimodal rumor detection method based on metric learning, which comprises the following steps: Step 1: Use the pre-trained BERT and VGG-19 models to extract modal features for text and images respectively; Step 2: Use syntactic dependency analysis to extract local relationships of text features, use image attention mechanism to mine local relationships of visual features, and dynamically adjust the weights of each modal feature to achieve information flow within the modality. Step 3, transform the two modal features into vectors of the same dimension in spatial mapping; Step 4, using contrastive learning and triplet learning in metric learning to sequentially establish intra-modal and inter-modal distance functions, confirm inter-modal and intra-modal feature relationships, realize information interaction, and determine the connection between data; the contrastive learning calculates the similarity of text features and visual features under the same label and continuously optimizes the contrastive loss function to train the model to distinguish between rumors and non-rumor information in the form of text + picture combinations, and the triplet learning calculates the intra-modal rumor and non-rumor distance, and uses the triplet loss function to continuously optimize the distance between the two to train the model to distinguish between rumors and non-rumor information of the same modality; Step 5, making full use of the effective data provided by the input information, and fusing the text features and visual features added with the local feature relationship in the vector input feature fusion unit; Step 6: The fused features are used for the rumor detector to detect the information to determine the truth or falsity of the input information. The rumor detector is a fully connected layer neural network learning classifier with ReLU and Softmax activation functions. Step 7: Use the cross entropy loss function to optimize the rumor classifier and obtain the final optimization target.

[0005] As a further improvement of the present invention, in step 1, a word segmentation operation is performed on each sentence in the text, and then the feature representation of the text is output through the Encoder operation inside BERT; the image is converted into RGB format, and then input into the VGG-19 model, and the representation features of the image are extracted through 16 convolutional layers and 3 fully connected layers. The feature matrix V obtained in the last layer is connected to a fully connected layer, and the visual features of the image are finally obtained through the ReLU activation function.

[0006] As a further improvement of the present invention, in step 2, the local relationship extraction of text features is realized by using syntactic dependency analysis, and the specific steps are as follows: syntactic analysis is used to generate the interdependence between word vectors, and the syntax tree of the input text is constructed using Google's open source SyntaxNet system, and then different weights are assigned to each word according to the degree of contribution of different words, and finally the vector representation of the word is updated by combining the word vector and the attention weight.

[0007] As a further improvement of the present invention, in step 2, the local relationship of visual features is mined using the image attention mechanism, and the specific operation is as follows: the sentence vector obtained by syntactic analysis is used as the adaptive condition of the visual feature, and the adaptive vector of the visual feature is confirmed according to the relationship between the modalities, thereby calculating the local relationship of the visual feature.

[0008] As a further improvement of the present invention, in step 4, the contrastive learning maximizes the similarity of the same type of labels under different modalities by minimizing the contrastive loss function. After continuous training, the model accurately distinguishes multimodal information pairs with inconsistent labels to obtain the contrastive loss.

[0009] As a further improvement of the present invention, in step 4, the triplet learning trains the model parameters by minimizing the triplet loss, so that the model encodes the input information, and obtains that the distance of the same label information in the single modality is constantly approaching, and the distance of the different label information is constantly expanding, thereby mining the relationship between rumors and non-rumors in the single modality, and using the Mahalanobis distance to calculate the data information in the space to obtain the triplet loss.

[0010] As a further improvement of the present invention, in step 7, the rumor classifier is optimized using the cross entropy loss function to obtain the cross entropy loss of rumor classification, and the sum of the contrast loss, triplet loss and cross entropy loss of rumor classification is taken as the final optimization target. Beneficial Effects

[0011] This paper proposes a multimodal rumor detection method based on metric learning. This method is based on a multimodal rumor detection model based on metric learning. The information of the model is input in the form of text + picture, and the pre-trained model is used to extract features from text and pictures respectively. This method uses syntactic dependency analysis and attention mechanism to mine local modal relationships and dynamically adjust the weights of each modal feature. It then introduces metric learning to determine the data distance between rumors and non-rumors. It also fuses the processed features of each modality and uses the fused features for model training. This method can extract more complete semantic expressions by extracting local relationships between text features and visual features and mining the relationships between features of each modality, thereby enabling more accurate rumor identification and improving the accuracy of rumor detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The figure is an overall diagram of a multimodal rumor detection model based on metric learning of the present invention; Figure 2 The present invention is a workflow diagram of a multimodal rumor detection method based on metric learning; Figure 3 for Figure 2 Schematic diagram of the process of syntactic structure analysis in Chinese; Figure 4 for Figure 2 Flowchart of the specific operations of adaptive attention extraction in ; Figure 5 Ablation experiment results on Weibo dataset; Figure 6 Ablation experiment results on Twitter dataset. DETAILED DESCRIPTION

[0013] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It will be appreciated that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the inventions. It should also be noted that, for ease of description, only portions related to the relevant inventions are shown in the accompanying drawings. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0014] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. This paper proposes a multimodal rumor detection model based on metric learning. The multimodal rumor detection model based on metric learning consists of the following parts: (1) Feature extractor: Use BERT and VGG-19 to extract features from text information and image information respectively; (2) Local Relation Extractor: It uses syntactic dependency analysis and image adaptation techniques to find the local relations between text features and visual features, respectively, to achieve information flow within the modality. (3) Mapping space: Mapping feature representations under different modalities to the same space to facilitate distance calculation of data; (4) Metric learning: Modeling the distance functions within and between modalities, realizing the interaction between information, and determining the connection between data; (5) Feature fusion: Fuse the processed text features with visual features to make full use of the effective data provided by the input information; (6) Rumor detection: The classifier is learned through a fully connected neural network to judge the truthfulness of the input information.

[0015] The workflow of this model is as follows Figure 1 As shown in the figure. Information is input into the model in the form of text + pictures, and pre-trained models are used to extract features of text and pictures respectively. In order to extract complete semantic expressions, it is necessary to extract local relations between text features and visual features. Considering the impact of information complementarity between multimodal information on detection results, the input information is divided into two categories: rumors and non-rumors, and the information interaction between modalities is realized by optimizing the contrast loss function. At the same time, the triple loss function is optimized according to the labels of rumors and non-rumors in the same modality to realize the information flow within the modality, and fully explore the effective information provided by text and vision.

[0016] The present invention provides a multimodal rumor detection method based on metric learning, taking into account the attention mechanism. The overall process of the method is as follows: Figure 2 As shown, it mainly includes the following steps: Step 1: Use the pre-trained BERT and VGG-19 models to extract features for text and images respectively. The operations of extracting text features using the BERT model are as follows: First, we perform word segmentation on each sentence in the text. Then the sentence can be represented by a combination of words W = {w1, w2, w3, ..., wn} (n represents the number of words in each sentence). Then, we use the Encoder inside BERT to output the feature representation of the text. The output text feature is T: T=BERT(W1,W2…W n ) (1) The operations of extracting image features using the VGG-19 model are as follows: First, the image is converted into RGB format, and then input into the VGG-19 model. The representation features of the image are extracted through 16 convolutional layers and 3 fully connected layers. The feature matrix V obtained from the last layer is connected to a fully connected layer, and the visual features of the image are finally obtained through the ReLU activation function. The output visual features are recorded as O: O=σ(W O *V+b O ) (2) Among them, W0 is the coefficient matrix of the fully connected layer, b0 represents the offset, and σ represents the ReLU activation function.

[0017] Step 2: Since there are implicit connections between local features in each modality, syntactic dependency analysis is used to extract local relationships of text features, and adaptive attention is used to mine local relationships of visual features. The specific operations are as follows: Syntactic dependency analysis can process the syntactic structure of the input text information. Therefore, in order to extract local relations in text, this paper uses syntactic dependency analysis to generate the interdependence between word vectors. The syntax tree of the input text is constructed using the Google open source SyntaxNet system, and then different weights are assigned to each word according to the contribution of different words. Finally, the word vector representation is updated by combining the word vector and the attention weight. The updated word vector representation is denoted as T wi : Among them, Attention(wi) represents the weight of the i-th word after syntactic dependency analysis.

[0018] Because words are the smallest units that make up sentences, phrase representations are combined based on word vectors. Phrase features are constructed based on the relationship tree obtained from syntactic dependency analysis. They are represented as weighted averages of word vectors with attention weights. Then, phrase vectors are used to combine into sentence vector representations. The whole process is as follows: Figure 3 shown. T s =[phrase1,phrase2,…,phrase p ] (5) Where k represents the number of words that make up the phrase, and p represents the number of phrase combinations contained in the sentence.

[0019] Considering the information flow between modalities, this paper uses text information as an auxiliary condition for extracting local relations of images, and extracts local relations by referring to the Adaptive-SA module proposed by Qi Lixin et al. The sentence vector obtained by syntactic dependency analysis is used as the adaptive condition of visual features, and the adaptive vector of visual features is confirmed according to the relationship between modalities, so as to calculate the local relations of visual features. The specific operation is as follows Figure 4 shown.

[0020] The sentence vector is fused with the visual representation, and the weights of different regional objects in the visual features are calculated through the Sigmoid activation function to form an adaptive vector: G=σ(W G *[T s ,O]) (6) Self-attention is used to calculate the correlation between the regions within the image, thereby determining the local relationship of the visual features. Since the keywords constrain the visual features under the adaptive condition, when calculating the self-attention of the visual features, it is necessary to update the mapping of the visual features in different vector spaces through the adaptive vector: O′ Q =(1+G)*O Q (7) O′ K =(1+G)*O K (8) The local relations of visual features are computed based on the spatial mapping vectors under adaptation conditions. Where d represents the dimension of the word vector, w0 represents the training parameter, and O' represents the dimension adjustment of the concatenated visual features using the fully connected layer to ensure that it has the same dimension as the sentence vector.

[0021] Step 3: Convert the two modal features into vectors of the same dimension in the spatial mapping. Since different feature extractors obtain different feature information, it is impossible to directly calculate the text features and visual features. Therefore, it is necessary to project each single modality into a common vector space and perform subsequent distance calculations and feature fusion operations in the vector space.

[0022] Assuming that the text features have N d1-dimensional vectors and the visual features have M d2-dimensional vectors, we convert their different dimensions into d-dimensional vector representations using the following formula: Among them, W v ,W t is the weight matrix, b v ,b t is the offset.

[0023] In step 4, contrastive learning and triplet learning in metric learning are used to capture the feature relationships within and between modalities respectively.

[0024] Step 4.1, maximize the similarity of the same type of labels under different modalities by minimizing the contrast loss function. After continuous training, the model can accurately distinguish multimodal information pairs with inconsistent labels.

[0025] Assume t p ,t n ,o p ,o n Respectively represent text non-rumor, text rumor, image non-rumor, image rumor, then t p , o p Combination (t p ,o p ) indicates non-rumor pair; t n , o n Combination (t n ,o n ) indicates that the rumor is right, and both combinations are normal. p , o n When combined, it means that the text and image labels do not match, and this situation is directly judged as a rumor. In summary, this article uses cosine similarity to calculate the similarity value of the combination, and assigns high scores to similar label combinations and low scores to different label combinations, and obtains the contrast loss L1. The calculation formula is as follows: Where D = {(t p , o p ), (t n , o n )} represents the text-image information pair with the same label, sim() represents the similarity value of the text-image pair, and α1 represents the maximum difference between the similarity values ​​of the same label pair and the different label pairs.

[0026] Step 4.2, by minimizing the triplet loss to train the model parameters, the model encodes the input information, and obtains that the distance between the same label information in the single modality is getting closer and the distance between the different label information is getting larger, thereby mining the relationship between rumors and non-rumors in the single modality.

[0027] Based on contrastive learning, triplet learning introduces anchor samples, and further determines the relative relationship within and between classes through the distance between positive and negative samples and anchor points.

[0028] Taking text features as an example, assuming t a represents the anchor point sample, t prepresents the positive sample with the same label as the anchor sample, and tn represents the negative sample with the opposite label as the anchor sample. Through triple learning, the distance between the positive sample and the anchor is smaller than the distance between the negative sample and the anchor, thus completing the classification task. The data information in the classic Mahalanobis distance calculation space is used to obtain the triple loss L2, which is calculated as follows: D(t i , t j )=(t i -t j ) T M(t i -t j ) (14) Among them, M is semi-positive definite and symmetric, and the eigenvalues ​​of M must all be non-negative. L2=∑ H max(0,α2-Δ o (D(t a , t p )-D(t a , t n ))) (15) Where H = {(t a , t p , t n )} represents the sample type of the same modality, and α2 represents the interval parameter between positive samples and negative samples. When classifying text information, considering the constraint effect of image information on text content, the distance between image information is used as an auxiliary condition to help accurately extract the intra-modal relationship, that is: Δ O =Sign(||o a -o p ||2-||o α -o n ||2) (16) Among them, a , o p , o n They represent visual anchor points, visual positive samples, and visual negative samples respectively. The processing flow of visual features is similar to that of text features.

[0029] Step 5: After obtaining text features and visual features through pre-training, syntactic dependency analysis and adaptive attention are used to obtain the local feature relationship of the modality and update the extracted features, and the updated text features and visual features are spliced ​​and fused.

[0030] Since different features contribute differently to the model, this paper uses two fully connected layer neural networks to calculate the attention weight of each feature vector, and fuses the text features and visual features through the attention weight. In order to avoid the accidental loss of modal features, the attention weight is added by 1. F=(1+α)[T s ,O′] (18) Among them, W1 TO 、W2 TO represents the learning parameter matrix, b1 TO 、b2 TO Indicates the offset.

[0031] Step 6: The fused features are used in the rumor detector to detect information to determine the truthfulness of the input information.

[0032] The fused multimodal features are input into two fully connected layers with ReLU and Softmax activation functions. The rumor detector is denoted as D: The final model predicts the probability that the information is a rumor:

[0033] Step 7: Optimize the target.

[0034] Assume that the rumor label is 1 and the non-rumor label is 0, and use the cross entropy loss function to optimize the rumor classifier. The cross entropy loss of rumor classification is obtained, that is: Taking into account the constraints between multimodal information, the sum of contrast loss, triple loss and cross entropy loss of rumor classification is taken as the final optimization target of this model, that is: L=α1L1+α2L2+L3 (22)

[0035] Example

[0036] The data sets involved in the experiment in this paper are from Weibo and Twitter. The Weibo data set is provided by Jin's team. The specific rumor content comes from the false rumor posts verified by the rumor-refuting system of the Weibo platform from May 2012 to January 2016, and the non-rumor content comes from the posts certified by Xinhua News Agency. The Twitter data set comes from MediaEval2015, in which the multimodal information includes the text information, picture information, release time and user characteristics of the post. Referring to the data processing method in the literature of Dhruv K, other information is filtered out, and only text information and picture information are retained. The statistical information of the data set is shown in Table 1.

[0037] Table 1 Dataset related information statistics

[0038] Experimental environment: Windows×11_64; GTX3050; python3.8; pytorch1.2.0, tensorflow2.2.0.

[0039] For text features, the BERT model is used for feature extraction. The text length is set to 50 and the word vector is 768. For visual features, the penultimate layer output vector of VGG-19 is used, and its dimension is 4096. The fully connected layer of text and visual features is 1024, and the activation function is the Relu function. The multimodal fusion dimension is 1024, and the fusion layer k=2.

[0040] The batch size of the model is 128, the number of training times is 300, the learning rate is 0.0005, the dropout is 0.4, and the optimizer selects Adam to find the optimal parameters of the network.

[0041] This paper uses the commonly used evaluation indicators in classification tasks: accuracy, precision, recall and F1 score as the basis for analysis, and conducts comparative experiments on the BERT model, VGG-19 model, EANN model, att-RNN model, MRML model, MLMRD model, MLMRD-O model, MLMRD-T, and MLMRD- model in turn, analyzing the impact of feature relationships on the experimental results from multiple angles.

[0042] Table 2 shows the evaluation indicators of each model.

[0043] It can be seen from Table 2 that the evaluation indicators of this model are better than other models, indicating that this model can better extract effective information from text and pictures to complete the rumor detection task under multimodal conditions. Observing the experimental data, it is found that the performance of the two single-modal models BERT and VGG-19 is the worst, indicating that the information extracted by the multimodal model can be effectively complementary, and the combination of text and picture information can improve the accuracy of the model. The performance of MLMRD is better than that of EANN and att-RNN. The reason is that the EANN model only uses a simple splicing method to complete feature fusion, and the att-RNN model omits the connection between modalities when exploring the feature relationship of modalities. The MLMRD model takes into account the fluidity of information, takes the connection between modalities into account in the model and effectively fuses the features. The analysis results show that the information between different modalities can influence and restrict each other, and the connection between modalities affects the detection effect of the model. MLMRD adds a local relationship extraction operation on the basis of MRML, assigns different weights to input features according to the connection between local information, gives full play to the role of key features on the model, and simultaneously determines the feature relationship within and between modalities and performs multimodal feature fusion. The accuracy is higher than that of MRML, indicating that the local relationship within the modality affects the complete representation of the modal semantics, and there is information interaction between multimodal information that affects the accurate representation of each modality. Introducing these two factors into the model has a significant effect on improving the accuracy of rumor detection.

[0044] The above comparison results show that the metric learning and local relationship extraction added to this model have certain effects in obtaining the feature relationships within and between modalities. Compared with the latest model MRML, the precision, recall, F1 value and accuracy are improved by 1.5%, 2.9%, 2.7% and 1.1% respectively.

[0045] In order to verify the impact of multimodality and modal feature relationships on this model, an ablation experiment was specially designed to analyze the role of each module. MLMRD-O means that only text features are considered; MLMRD-T means that only visual features are considered; MLMRD- means that the extraction of modal relationships by metric learning is removed.

[0046] Performance test experiments were conducted on two public datasets, Twitter and Weibo. Table 3 lists the various indicator data of this ablation experiment. The accuracy rates reached 92.8% and 85.2% respectively. The comparison results are as follows: Figure 5 , Figure 6 As shown in the figure, the results show that adding local feature relationships and the connections between modal features into the rumor detection method can further improve the accuracy of rumor detection.

[0047] Table 3 shows the various indicators of this ablation experiment

[0048] As can be seen from the figure, the accuracy of the unimodal model is lower than that of the multimodal model, indicating that the multimodal model can extract more useful features, so multimodal research is meaningful for rumor detection. The accuracy of MLMRD-T is lower than that of MLMRD-O because the effective information provided by the image is less than that provided by the text content. Compared with the MLMRD-experiment that removes local relationships, the accuracy of MLMRD is higher, indicating that there is a certain connection between the internal features of the image and text information. In the feature extraction process of the image and text, the constraint between the internal information ensures the value of the key features. Therefore, MLMRD can extract more accurate representation features and the detection accuracy is higher.

[0049] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multimodal rumor detection method based on metric learning, characterized in that: It includes the following steps: Step 1: Use the pre-trained BERT and VGG-19 models to extract modal features for text and images respectively; Step 2: Use syntactic dependency analysis to extract local relationships of text features, use image attention mechanism to mine local relationships of visual features, and dynamically adjust the weights of each modal feature to achieve information flow within the modality. Step 3, transform the two modal features into vectors of the same dimension in spatial mapping; Step 4: Use contrastive learning and triple learning in metric learning to establish intra-modal and inter-modal distance functions in turn, confirm the feature relationship between modalities and within modalities, realize the interaction between information, and determine the connection between data; The contrastive learning calculates the similarity of text features and visual features under the same label and continuously optimizes the contrastive loss function to train the model to distinguish between rumors and non-rumor information in the form of text + picture combination; the triplet learning calculates the distance between rumors and non-rumor within the modality, and uses the triplet loss function to continuously optimize the distance between the two to train the model to distinguish between rumors and non-rumor information in the same modality; Step 5, making full use of the effective data provided by the input information, and fusing the text features and visual features added with the local feature relationship in the vector input feature fusion unit; Step 6: The fused features are used for the rumor detector to detect the information to determine the truth or falsity of the input information. The rumor detector is a fully connected layer neural network learning classifier with ReLU and Softmax activation functions. Step 7: Use the cross entropy loss function to optimize the rumor classifier and obtain the final optimization target.

2. The multimodal rumor detection method according to claim 1, characterized in that: In step 1, each sentence in the text is segmented, and then the feature representation of the text is output through the Encoder operation inside BERT; the image is converted into RGB format and then input into the VGG-19 model. The representation features of the image are extracted through 16 convolutional layers and 3 fully connected layers. The feature matrix V obtained from the last layer is connected to a fully connected layer, and the visual features of the image are finally obtained through the ReLU activation function.

3. The multimodal rumor detection method according to claim 2, characterized in that: In step 2, the local relationship extraction of text features is realized by using syntactic dependency analysis. The specific steps are as follows: syntactic analysis is used to generate the interdependence between word vectors, and the syntax tree of the input text is constructed using Google's open source SyntaxNet system. Then, different weights are assigned to each word according to the degree of contribution of different words. Finally, the word vector and attention weight are combined to update the vector representation of the word.

4. The multimodal rumor detection method according to claim 3, characterized in that: In step 2, the local relationship of visual features is mined using the image attention mechanism. The specific operation is as follows: the sentence vector obtained by syntactic analysis is used as the adaptive condition of the visual feature, the adaptive vector of the visual feature is confirmed according to the relationship between the modalities, and the local relationship of the visual feature is calculated.

5. The multimodal rumor detection method according to claim 4, characterized in that: In step 4, the contrastive learning maximizes the similarity of the same type of labels under different modalities by minimizing the contrastive loss function. After continuous training, the model can accurately distinguish multimodal information pairs with inconsistent labels and obtain the contrastive loss.

6. The multimodal rumor detection method according to claim 5, characterized in that: In step 4, the triplet learning trains the model parameters by minimizing the triplet loss, so that the model encodes the input information, and obtains that the distance between the same label information in the single modality is constantly approaching, and the distance between the different label information is constantly expanding, thereby mining the relationship between rumors and non-rumors in the single modality, and using the Mahalanobis distance to calculate the data information in the space to obtain the triplet loss.

7. The multimodal rumor detection method according to claim 6, characterized in that: In step 7, the rumor classifier is optimized using the cross entropy loss function to obtain the cross entropy loss of rumor classification, and the sum of the contrast loss, triple loss and cross entropy loss of rumor classification is taken as the final optimization target.

Images