A rumor identification method and device based on multi-modal adversarial contrast learning

Abstract

The present invention relates to a method for training a rumor identification model based on multi-modal adversarial contrast learning, including obtaining sample data text-image pairs and related comments; extracting the preliminary text features and preliminary image features of the text-image pairs and inputting them into a modality discriminator; using adversarial learning to minimize and maximize the modality discriminator loss to obtain modality-invariant features; extracting the stance change features and sentiment change features based on temporal changes of the related comments and inputting them into a feature discriminator; using adversarial learning to minimize and maximize the feature discriminator loss to update the feature discriminator and obtain fused specific features; splicing the modality-invariant features and the fused specific features and inputting them into a multi-layer perceptron to generate a comprehensive multi-modal representation, calculating the contrast loss with positive and negative samples, and minimizing to update the parameters of the multi-layer perceptron; constructing an objective loss function and minimizing to update the parameters of the multi-layer perceptron; during adversarial training, maximizing the objective loss function to update the parameters of the modality discriminator and the feature discriminator.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a method for training a rumor identification model based on multi-modal adversarial contrast learning, a rumor identification method, a device and a computer-readable storage medium. Background Art

[0002] Most existing rumor detection methods focus on text data and extract different text features for rumor detection. With the development of multimedia technology, visual content has become an important part of rumors, attracting and even misleading consumers to achieve event discussion and rapid dissemination. In recent years, multi-modal rumor detection has become a new research hotspot, and learning effective feature representations from heterogeneous multi-modal data has also brought greater challenges to the rumor detection task. Previous work mainly fused cross-modal information through some simple operations, such as concatenation, addition, etc., or attention mechanisms. These methods are easy to implement. Although they can achieve multi-modal fusion, they do not consider the differences between modalities and samples, resulting in the limitations of existing multi-modal rumor detection models.

[0003] Sun et al. proposed a graph adversarial contrast model GACL, which characterized the spread of rumors with a graph structure and then combined adversarial and contrast learning to enhance the representation of the rumor spread structure; the GACL model applied adversarial learning and contrast learning to the rumor spread structure to enhance the graph representation learned by graph convolution, so as to improve the robustness in the rumor detection task. Wang et al. proposed an event adversarial neural network EANN for multi-modal fake news detection, using an adversarial learning strategy to remove the differences between events in the samples and adopting a simple concatenation strategy during modality fusion; the EANN model used an adversarial neural network method to remove the differences between different events in the samples, so that the model could achieve stable rumor detection results for posts regardless of the event. Xie et al. proposed a stance detection and reasoning network SERN, which was dedicated to extracting the stance information of netizens from comment information to infer whether the post was a rumor; SERN proposed a method for automatically learning user stance information in comment data, and the model used the extracted stance information to assist rumor detection.

[0004] However, all of the above models lack further research on modality fusion, do not consider the alignment between modalities, and fail to solve the heterogeneity problem in multi-modal data. In actual application scenarios, almost all posts on social media currently contain multi-modal information. Most of the previous methods only consider the unimodal text information. Although the models for extracting text semantic information are already very mature, relying solely on text semantic information also makes it difficult to correctly classify some samples. In actual scenarios, multi-modal information is available and can provide more clues for the rumor detection task. The introduction of multi-modal information can provide us with more sufficient information. We can combine text information and image information to jointly identify rumors. However, the current related research on multi-modal rumor detection ignores the differences between modalities and does not fully consider the modality alignment problem. Simple multi-modal fusion methods such as splicing or summing may introduce noise into the model, which will weaken the role of multi-modal data and is not conducive to developing a more robust model. Summary of the Invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that multi-modal rumor detection does not consider the differences between modalities, resulting in poor detection effects.

[0006] To solve the above technical problem, the present invention provides a method for training a rumor discrimination model based on multi-modal adversarial contrast learning, including:

[0007] Obtain the text-image pairs and related comments of the sample data in the training set;

[0008] Extract features from the text-image pairs to obtain preliminary text features and preliminary image features, and input them into the modality discriminator;

[0009] Using adversarial learning, minimize and maximize the modality discriminator loss, update the modality discriminator parameters, align and fuse the input preliminary text features and the preliminary image features to generate modality-invariant features;

[0010] Extract the stance change features and sentiment change features based on the temporal change of the related comments, and input them into the feature discriminator;

[0011] Using adversarial learning, minimize and maximize the feature discriminator loss, update the feature discriminator parameters, align and fuse the stance change features and the sentiment change features to generate fused specific features;

[0012] After splicing the modality-invariant features and the fused specific features, input them into a multi-layer perceptron to generate a comprehensive multi-modal representation, calculate the contrast loss with positive and negative samples, minimize the contrast loss, and update the multi-layer perceptron parameters;

[0013] Construct an objective loss function based on the modality discriminator loss, the feature discriminator loss, the contrastive loss, and the multi-layer perceptron loss;

[0014] Minimize the objective loss function to update the model parameters of the multi-layer perceptron; maximize the objective loss function to update the model parameters of the modality discriminator and the feature discriminator;

[0015] Obtain a multi-modal rumor discrimination model including a modality discriminator, a feature discriminator, and a multi-layer perceptron.

[0016] In an embodiment of the present invention, the extraction of relevant comments is based on the stance change feature and the sentiment change feature of temporal changes, including:

[0017] Use SERN to obtain the stance feature of relevant comments;

[0018] Use a sentiment analysis platform to obtain the sentiment feature of relevant comments;

[0019] Use RNN to capture the temporal changes of relevant comments, and combine the stance feature and the sentiment feature to generate a stance change feature and a sentiment change feature.

[0020] In an embodiment of the present invention, the initial text features are obtained by encoding using a language representation model, and the language representation model includes but is not limited to RoBERTa, BERT, XLNet, and ALBERT.

[0021] In an embodiment of the present invention, the initial image features are extracted using a picture feature extractor, and the picture feature extractor includes but is not limited to ResNet50, Vgg19, and ViT.

[0022] In an embodiment of the present invention, the contrastive loss function is expressed as:

[0023]

[0024] where K represents the total number of samples, l k represents the comprehensive multi-modal representation, l p is a positive sample with the same label as l k l a is a negative sample with a different label from l k ; A(k) represents the set of negative samples, P(k) represents the set of positive samples; sim(·) represents the cosine similarity calculation formula; the temperature coefficient τ is a hyperparameter used to control the discrimination degree of the model for negative samples.

[0025] In an embodiment of the present invention, the modality discriminator loss, the feature discriminator loss, and the multi-layer perceptron loss are all cross-entropy losses.

[0026] In one embodiment of the present invention, the functional expression of the cross-entropy loss is:

[0027]

[0028] where K represents the total number of sample data in the training set, y i represents the true label of the i-th sample, represents the model prediction value of the i-th sample.

[0029] The embodiment of the present invention also provides a rumor identification method based on multi-modal adversarial contrast learning, including:

[0030] Obtain the text-image pair and related comments of the sample to be detected, extract the preliminary text features, preliminary image features, stance change features and sentiment change features, and input them into the multi-modal rumor identification model trained by the multi-modal adversarial contrast learning rumor identification model training method according to any one of claims 1 to 7;

[0031] Use the modality discriminator to align the preliminary text features and the preliminary image features to obtain modality-invariant features;

[0032] Use the feature discriminator to fuse the stance change features and the sentiment change features to obtain the fused specific features;

[0033] Concatenate the modality-invariant features and the fused specific features, input them into a multi-layer perceptron, and obtain the rumor identification result.

[0034] The embodiment of the present invention also provides a device for the rumor identification method based on multi-modal adversarial contrast learning as described above, including:

[0035] A sample acquisition module for obtaining the text-image pair and related comments of the sample to be detected;

[0036] A feature encoding module for encoding and obtaining the preliminary text features and preliminary image features of the text-image pair;

[0037] A feature extraction module for capturing the stance features and sentiment features of the related comments, and obtaining the stance change features and sentiment change features according to the temporal change;

[0038] A feature fusion module for using the modality discriminator to align the preliminary text features and the preliminary image features to obtain modality-invariant features; using the feature discriminator to fuse the stance change features and the sentiment change features to obtain the fused specific features;

[0039] A rumor identification module, which is used to splice the modality-invariant features and the fusion-specific features, input them into a multi-layer perceptron, and obtain a rumor identification result.

[0040] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the rumor identification method based on multi-modal adversarial contrast learning as described above are implemented.

[0041] The above technical solution of the present invention has the following advantages compared with the prior art:

[0042] The rumor identification model training method based on multi-modal adversarial contrast learning according to the present invention takes into account the differences between multi-modal information, and for the first time introduces adversarial learning and contrast learning into the multi-modal fusion process; uses the initial text features and the initial image features to train a modality discriminator through adversarial learning to achieve the alignment of text features and image features; captures the emotional change features and stance change features based on temporal changes, and trains a feature discriminator through adversarial learning to eliminate the differences between the emotional change features and the stance change features, and obtain fusion-specific features; the present invention uses a modality discriminator and a feature discriminator to eliminate the differences between different modalities in the samples. In the multi-layer perceptron, contrast learning is used to remove the differences between multi-modal samples by shortening the distance between the current sample and the positive example sample and increasing the distance from the negative example sample, and it is easier to learn the common features of the same type of samples. And the present invention optimizes and updates the model parameters of the multi-modal rumor identification model by constructing an objective loss function and minimizing-maximizing the objective loss function, significantly improving the rumor identification performance. [[ID=�10]] Description of the Drawings

[0043] In order to make the content of the present invention easier to be clearly understood, the following further details the present invention according to the specific embodiments of the present invention in conjunction with the drawings, where

[0044] Figure 1 is a flowchart of the steps of the rumor identification model training method based on multi-modal adversarial contrast learning provided by the present invention;

[0045] Figure 2 is an identification flowchart of the rumor identification method based on multi-modal adversarial contrast learning provided by the present invention. Detailed Embodiments

[0046] The following further illustrates the present invention in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the examples given are not intended to limit the present invention.

[0047] Since most previous rumor detection tasks have focused on unimodal features of text, while in fact the vast majority of posts on social media provide multimodal data. Multimodal data provides more information for the rumor detection task, and there are differences between multimodal information. However, previous methods have not fully considered the problem of modality alignment. Therefore, the present invention proposes a rumor discrimination method based on multimodal adversarial contrast learning. Using the adversarial contrast fusion method, it considers the problems of modality alignment and sample alignment, and solves the modality alignment problem to help the multimodal rumor discrimination model perform better rumor detection.

[0048] Referring to Figure 1 As shown, the training method of the rumor discrimination model based on multimodal adversarial contrast learning of the present invention includes:

[0049] S101: Construct a training set based on an open-source dataset, and extract the text-image pairs and related comments of the sample data in the training set;

[0050] S102: Extract features from the text-image pairs to obtain preliminary text features and preliminary image features, and input them into the modality discriminator; using adversarial learning, minimize-maximize the modality discriminator loss, and update the modality discriminator to align the preliminary text features and the preliminary image features to obtain modality-invariant features of the sample data;

[0051] S102-1: Use a language representation model for encoding to obtain preliminary text features, and the language representation model includes but is not limited to RoBERTa, BERT, XLNet, and ALBER;

[0052] S102-2: Use an image feature extractor to extract preliminary image features, and the image feature extractor includes but is not limited to ResNet50, Vgg19, and ViT;

[0053] S102-3: Input the preliminary text features and the preliminary image features into the modality discriminator; use adversarial learning to minimize-maximize to train the modality discriminator to obtain modality-invariant features; the modality discriminator is a binary classifier.

[0054] S103: Extract the stance change features and sentiment change features based on the temporal change of the related comments, and input them into the feature discriminator; using adversarial learning, minimize-maximize the feature discriminator loss, and update the feature discriminator to fuse the stance change features and the sentiment change features to generate fused specific features;

[0055] S103-1: Use SERN to obtain the stance features of the related comments;

[0056] S103-2: Use a sentiment analysis platform to obtain the sentiment features of the related comments;

[0057] S103-3: Use the RNN to capture the temporal changes of relevant comments, combine the position features and the sentiment features to generate position change features and sentiment change features;

[0058] S103-4: Input the position change features and the sentiment change features into a feature discriminator for adversarial learning, train the feature discriminator, and generate fused specific features; the feature discriminator is a binary classifier.

[0059] S104: After concatenating the modality-invariant features and the fused specific features, input them into a multi-layer perceptron to generate a comprehensive multi-modal representation, calculate the contrast loss with positive and negative samples, minimize the contrast loss, and update the parameters of the multi-layer perceptron;

[0060] The contrast loss function is expressed as:

[0061]

[0062] where K represents the total number of samples, l k represents the comprehensive multi-modal representation, l p is the positive sample with the same label as l k and l a is the negative sample with a different label from l k ; A(k) represents the set of negative samples, P(k) represents the set of positive samples; sim(·) represents the cosine similarity calculation formula; the temperature coefficient τ is a hyperparameter used to control the discrimination degree of the model for negative samples.

[0063] S105: Construct an objective loss function according to the modality discriminator loss, the feature discriminator loss, the contrast loss, and the multi-layer perceptron loss;

[0064] Minimize the objective loss function to update the model parameters of the multi-layer perceptron; maximize the objective function to update the model parameters of the modality discriminator and the feature discriminator;

[0065] where the modality discriminator loss, the feature discriminator loss, and the multi-layer perceptron loss are all cross-entropy losses; the cross-entropy loss is expressed as:

[0066]

[0067] where K represents the total number of sample data in the training set, y i represents the true label of the i-th sample, represents the model prediction value of the i-th sample;

[0068] The objective loss function is expressed as:

[0069] L 目标= L1 + L2 + L3 + L sup1

[0070] Among them, L1 represents the modal discriminator loss, L2 represents the feature discriminator loss, L3 represents the multi-layer perceptron loss, and L sup1 represents the contrast loss.

[0071] S106: Obtain a multi-modal rumor discrimination model including a modal discriminator, a feature discriminator, and a multi-layer perceptron.

[0072] Specifically, since the encoding spaces of text and images are inconsistent, that is, the pre-encoding models are inconsistent, there are significant differences between the initial text features and the initial image features. A modal discriminator is introduced to distinguish which type of modal features are input, and by minimizing the modal discriminator loss, it is easy to learn the differential representations between different modalities. However, in the present invention, it is necessary to fuse the features of different modalities. The greater the difference, the more unfavorable it is for feature fusion. Therefore, the present invention introduces adversarial learning to remove the differences between modalities by minimizing-maximizing the modal discriminator loss and the feature discriminator loss, promoting the fusion of features of different modalities in sample data. By introducing a contrast learning strategy in the multi-layer perceptron to learn the common points between similar multi-modal samples and narrow the differences between multi-modal samples. And the present invention constructs an objective loss function and minimizes-maximizes the objective loss function to optimize and update the model parameters of the rumor discrimination model based on multi-modal adversarial contrast learning, significantly improving the rumor discrimination performance.

[0073] Based on the above embodiments, the embodiments of the present invention further provide a multi-modal rumor discrimination method for a model trained by the above-mentioned rumor discrimination model training method based on multi-modal adversarial contrast learning, including:

[0074] S201: Obtain the text-image pair and related comments of the sample to be detected, extract the initial text features, initial image features, stance change features, and sentiment change features, and input them into the multi-modal rumor discrimination model trained by the above-mentioned rumor discrimination model training method based on multi-modal adversarial contrast learning;

[0075] Use a language representation model for encoding to obtain initial text features, and the language representation model includes but is not limited to RoBERTa, BERT, XLNet, and ALBERT; use a picture feature extractor to extract initial image features, and the picture feature extractor includes but is not limited to ResNet50, Vgg19, and ViT;

[0076] Use SERN to obtain the stance features of relevant comments; use a sentiment analysis platform to obtain the sentiment features of relevant comments; use an RNN to capture the temporal changes of relevant comments, and combine the stance features and the sentiment features to generate stance change features and sentiment change features;

[0077] S202: Align the preliminary text features and the preliminary image features using a modality discriminator to obtain modality-invariant features;

[0078] S203: Use a feature discriminator to fuse the stance change features and the sentiment change features to obtain fused specific features;

[0079] S204: Concatenate the modality-invariant features and the fused specific features, input them into a multi-layer perceptron, and obtain the rumor discrimination result.

[0080] Refer to Figure 2 As shown, specifically, in an embodiment of the present invention, the sample data to be detected consists of a piece of text, an image, and related comments. Encode the image sample through the ResNet50 model to obtain the preliminary image features; pass the text sample through RoBERTa, construct the input according to the semantic matching task of RoBERTa, process the text into the form of "[CLS] sentence [SEP]", and use the Transformer layer for encoding to obtain the preliminary text features; use a modality discriminator to align the preliminary image features and the preliminary text features to obtain modality-invariant features. Use a neural network to obtain the sentiment features and stance features contained in each related comment; introduce an RNN network to capture the temporal change information to obtain the sentiment change features and stance change features; use a feature discriminator to fuse the sentiment change features and the stance change features to obtain fused specific features. Concatenate the modality-invariant features of the sample to be detected and the fused specific features, and send them into a multi-layer perceptron for classification and discrimination to obtain the rumor discrimination result.

[0081] Based on the above embodiments, an embodiment of the present invention further provides an apparatus for the rumor discrimination method based on multi-modal adversarial contrast learning as described above, including:

[0082] A sample acquisition module 100 for acquiring the text-image pair and related comments of the sample to be detected;

[0083] A feature encoding module 200 for encoding to obtain the preliminary text features and the preliminary image features of the text-image pair;

[0084] A feature extraction module 300 for capturing the stance features and sentiment features of related comments, and obtaining the stance change features and sentiment change features according to the temporal change;

[0085] A feature fusion module 400 for using a modality discriminator to align the preliminary text features and the preliminary image features to obtain modality-invariant features; using a feature discriminator to fuse the stance change features and the sentiment change features to obtain fused specific features;

[0086] A rumor identification module 500 is configured to splice the modality-invariant features and the fusion-specific features, input them into a multi-layer perceptron, and obtain a rumor identification result.

[0087] The rumor identification device based on multi-modal adversarial contrast learning in this embodiment is used to implement the aforementioned rumor identification method based on multi-modal adversarial contrast learning. Therefore, the specific implementation manners in the rumor identification device based on multi-modal adversarial contrast learning can be seen in the embodiment part of the aforementioned rumor identification method based on multi-modal adversarial contrast learning. For example, a sample acquisition module 100 is configured to acquire a text-image pair of a sample to be detected and related comments; a feature encoding module 200 and a feature extraction module 300 are used to implement step S201 in the aforementioned rumor identification method based on multi-modal adversarial contrast learning; a feature fusion module 400 is used to implement steps S202 and S203 in the aforementioned rumor identification method based on multi-modal adversarial contrast learning; a rumor identification module 500 is used to implement step S204 in the aforementioned rumor identification method based on multi-modal adversarial contrast learning. Therefore, the specific implementation manners can refer to the descriptions of the corresponding respective part embodiments and will not be elaborated herein.

[0088] Based on the above embodiments, on the Pheme dataset and the Fakeddit dataset, the rumor identification method based on multi-modal adversarial contrast learning provided by the present invention and existing methods are used to identify rumors in the samples in the dataset. The existing methods for comparison include BERT, BERT+VGG, MCAN, and SERN. The evaluation indicators accuracy ACC, precision Prec, recall Rec, and F1 score are used to evaluate the identification results. The comparison of the identification results is shown in Table 1:

[0089] Table 1: Comparison of Identification Results

[0090]

[0091] As can be seen from Table 1, the rumor identification method based on multi-modal adversarial contrast learning provided by the present invention has the best accuracy when identifying rumors on both datasets, and the precision, recall, and F1 score also have high ratios. Therefore, the rumor identification method based on multi-modal adversarial contrast learning provided by the present invention has a high correct rate and good detection effect in rumor identification.

[0092] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the aforementioned rumor identification method based on multi-modal adversarial contrast learning are implemented.

[0093] The training method of the rumor discrimination model based on multi-modal adversarial contrast learning according to the present invention takes into account the differences between multi-modal information, and for the first time introduces adversarial learning and contrast learning into the multi-modal fusion process; uses the preliminary text features and the preliminary image features to train a modality discriminator through adversarial learning to achieve alignment between modalities; captures the emotional change features and stance change features based on temporal changes, and trains a feature discriminator through adversarial learning to eliminate the differences between the emotional change features and the stance change features and obtain fused specific features; uses the modality discriminator and the feature discriminator to eliminate the differences of different modalities in the samples. In the multi-layer perceptron, through contrast learning, by shortening the distance between the current sample and the positive example sample and increasing the distance from the negative example sample, the differences between multi-modal samples are removed, and it is easier to learn the common features of the same type of samples. And the present invention optimizes and updates the model parameters of the multi-modal rumor discrimination model by constructing an objective loss function and minimizing-maximizing the objective loss function, significantly improving the rumor discrimination performance.

[0094] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0096] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the steps of the process Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps of the functions specified in one block or a plurality of blocks.

[0098] Obviously, the above embodiments are only examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A method for training a rumor discrimination model based on multi-modal adversarial contrast learning, characterized in that Including: Obtain the text-image pairs and related comments of the sample data in the training set; Extract features from the text-image pairs to obtain preliminary text features and preliminary image features, and input them into the modality discriminator; Using adversarial learning, minimize and maximize the modality discriminator loss, update the modality discriminator parameters, align and fuse the input preliminary text features and preliminary image features to generate modality-invariant features; Extract the stance change features and sentiment change features of the related comments based on temporal changes, and input them into the feature discriminator; Using adversarial learning, minimize and maximize the feature discriminator loss, update the feature discriminator parameters, align and fuse the stance change features and sentiment change features to generate fused specific features; After splicing the modality-invariant features and the fused specific features, input them into a multi-layer perceptron to generate a comprehensive multi-modal representation, calculate the contrast loss with positive and negative samples, minimize the contrast loss, and update the multi-layer perceptron parameters; Construct an objective loss function according to the modality discriminator loss, the feature discriminator loss, the contrast loss, and the multi-layer perceptron loss; Minimize the objective loss function to update the model parameters of the multi-layer perceptron; maximize the objective loss function to update the model parameters of the modality discriminator and the feature discriminator; Obtain a multi-modal rumor discrimination model including a modality discriminator, a feature discriminator, and a multi-layer perceptron.

2. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 1, wherein, The extraction of the stance change features and sentiment change features of the related comments based on temporal changes includes: Use SERN to obtain the stance features of the related comments; Use a sentiment analysis platform to obtain the sentiment features of the related comments; Use an RNN to capture the temporal changes of the related comments, and combine the stance features and the sentiment features to generate stance change features and sentiment change features.

3. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 1, characterized in that The preliminary text features are obtained by encoding using a language representation model, and the language representation model includes but is not limited to RoBERTa, BERT, XLNet, and ALBERT.

4. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 1, characterized in that The preliminary image features are extracted using a picture feature extractor, and the picture feature extractor includes but is not limited to ResNet50, Vgg19, and ViT.

5. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 1, wherein The contrast loss function is expressed as: ， Among them, represents the total number of samples, represents the comprehensive multi-modal representation, is the positive sample with the same label as ; is the negative sample with a different label from ; represents the set of negative samples, represents the set of positive samples; represents the cosine similarity calculation formula; the temperature coefficient is a hyperparameter used to control the discrimination of the model for negative samples.

6. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 1, wherein, The modality discriminator loss, the feature discriminator loss, and the multi-layer perceptron loss are all cross-entropy losses.

7. The method for training a rumor identification model based on multi-modal adversarial contrast learning according to claim 6, wherein The functional expression of the cross-entropy loss is: ， Among them, represents the total number of sample data in the training set, represents the true label of the i-th sample, represents the model prediction value of the i-th sample.

8. A rumor identification method based on multi-modal adversarial contrast learning, characterized in that, Including: Obtain the text-image pairs and related comments of the sample to be detected, extract the preliminary text features, preliminary image features, stance change features, and sentiment change features, and input them into the multi-modal rumor discrimination model trained by the rumor discrimination model training method based on multi-modal adversarial contrast learning according to any one of claims 1 to 7; Use the modality discriminator to align the preliminary text features and the preliminary image features to obtain modality-invariant features; Use the feature discriminator to fuse the stance change features and the sentiment change features to obtain fused specific features; Splice the modality-invariant features and the fused specific features, and input them into a multi-layer perceptron to obtain the rumor discrimination result.

9. An apparatus for a rumor identification method based on multi-modal adversarial contrastive learning as described in claim 8, characterized in that, Including: A sample acquisition module for obtaining the text-image pairs and related comments of the sample to be detected; A feature encoding module, configured to obtain the initial text features and initial image features of the text image pair through encoding; A feature extraction module, configured to capture the stance features and sentiment features of relevant comments, and obtain the stance change features and sentiment change features according to the temporal changes; A feature fusion module, configured to align the initial text features and the initial image features by using a modality discriminator to obtain modality-invariant features; Fuse the stance change features and the sentiment change features by using a feature discriminator to obtain fused specific features; A rumor discrimination module, configured to splice the modality-invariant features and the fused specific features and input them into a multi-layer perceptron to obtain a rumor discrimination result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps in the rumor discrimination method based on multi-modal adversarial contrast learning as described in claim 8.

Images