Unsupervised cross-domain rumor detection method based on teacher-student learning framework

Abstract

The present application belongs to the technical field of rumor detection, and discloses an unsupervised cross-domain rumor detection method based on a teacher-student learning framework, comprising the following steps: using source domain data to perform supervised training on a teacher detection model, and synchronizing the trained parameters to a student detection model to complete parameter initialization of the model; using the teacher detection model to perform pseudo-label annotation on target domain data, and screening high-quality pseudo-labels of the target domain data; using the student detection model to perform self-training on the target domain data, combining cross-entropy loss of the high-quality pseudo-labels and contrastive learning loss to obtain training loss of the target domain data; using the training loss of the target domain data to optimize the parameters of the student detection model in a reverse propagation manner, and using a moving exponential average algorithm to smoothly superimpose the parameters of the training-optimized student detection model into the parameters of the teacher detection model, and returning to step S2 to start the next round of training. The method improves the effect of unsupervised cross-domain rumor detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of rumor detection and relates to an unsupervised cross-domain rumor detection method based on a teacher-student learning framework. Background Art

[0002] The task of rumor detection generally refers to judging the authenticity of text content posted on social media. Existing rumor detection technologies mostly use social media information with authenticity labels to conduct supervised training on the model, which can, to a certain extent, enable the model to have the ability to detect rumors about similar topic information. However, in actual application scenarios, the ever-changing event topics on social media have large contextual differences, which makes it difficult for existing rumor detection technologies to extend the learned rumor feature detection patterns to other event topics, resulting in low detection accuracy in test data of different topics. Faced with the massive amount of social media information, it is obviously unrealistic to label the training data for rumor detection in each topic area. Therefore, how to achieve effective detection of rumor information in different topic areas in the absence of labeled data is a technical problem that needs to be solved urgently.

[0003] The differences in semantic and propagation structure characteristics between data from different subject areas are the main reason for the poor generalization of rumor detection technology. Existing cross-domain rumor detection technologies mainly focus on strengthening the model's ability to represent semantic features, using the Transformer model to learn semantic features and bridging the differences in semantic features between domains through unsupervised methods such as cross-attention and contrastive learning. However, the Transformer model is suitable for processing sequential data. Existing cross-domain rumor detection technologies mostly simplify the tree-like propagation structure of social media information into a text sequence, which results in the loss of information propagation structure characteristics and reduces the effectiveness of cross-domain rumor detection technology. Summary of the Invention

[0004] In response to the technical problem that existing cross-domain rumor detection technology may cause the loss of information propagation features and reduce the detection effectiveness, the present invention provides an unsupervised cross-domain rumor detection method based on a teacher-student learning framework. It adopts an unsupervised self-training method, combined with a reweighted cross-entropy loss function and two types of contrast learning loss functions, to impose consistency regularization constraints on the feature vectors output by the teacher detection model and the student detection model, thereby improving the effect of unsupervised cross-domain rumor detection.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an unsupervised cross-domain rumor detection method based on a teacher-student learning framework, comprising the following steps:

[0007] S1: constructing a teacher detection model and a student detection model with the same structure;

[0008] S2: obtaining source domain data and target domain data;

[0009] S3: performing supervised training on the teacher detection model using the source domain data, and synchronizing the trained parameters to the student detection model to complete the parameter initialization of the teacher detection model and the student detection model;

[0010] S4: performing pseudo-label annotation on the target domain data using the teacher detection model, and screening high-quality pseudo-labels of the target domain data;

[0011] S5: performing self-training on the target domain data using the student detection model, combining the cross-entropy loss of the high-quality pseudo-labels and the contrastive learning loss to obtain the training loss of the target domain data;

[0012] S6: optimizing the student detection model parameters using the training loss of the target domain data, and smoothing the training optimized student detection model parameters and adding them to the teacher detection model parameters using the moving exponential average algorithm, returning to step S2 to start the next round of training.

[0013] The teacher detection model and the student detection model of the present application adopt two detection models with the same structure, each of which is composed of a pre-trained language model and a graph convolutional neural network model. For the pre-trained language model, Roberta model can be selected but is not limited to. For the graph convolutional neural network model, BiGCN model can be selected but is not limited to. The model architecture of BiGCN is composed of two graph convolutional neural networks with the same structure, which are called top-down graph convolutional neural network and bottom-up graph convolutional neural network, respectively. Each graph convolutional neural network contains two layers of graph convolutional layers for processing directed graph input data in the corresponding direction. In addition, in the feature extraction process, the features of the root node in the previous layer are spliced with the features of all nodes in the next layer between the layers of the model, and the semantic feature extraction of the root node, i.e. the original news information, is strengthened.

[0014] The present application represents social media information with propagation graphs, each propagation graph being composed of a root node, a child node and an edge, the root node being original information such as blogs, news, etc., the child node being comment information, and the edge representing the interaction relationship between comment information. The propagation graph is a directed graph, and the direction of the edge needs to be set as top-down and bottom-up respectively to simulate the propagation and aggregation of information. The source domain data and the target domain data are both divided into two categories of non-rumor and rumor, and the non-rumor data is marked as 0 and the rumor data is marked as 1.

[0015] In the above technical solution of the present application, the specific steps of performing supervised training on the teacher detection model using the source domain data in step S3 include:

[0016] performing edge discard on the source domain data;

[0017] concatenating the root node text and each sub-node text to form a node pair, and performing semantic feature coding on the text content of the node pair;

[0018] calculating the cross-entropy loss of the teacher detection model, and iteratively optimizing the parameters of the teacher detection model; the calculation formula of the cross-entropy loss of the teacher detection model is as follows:

[0019]

[0020] wherein: |B Tea is the data quantity of the current training round of the teacher detection model, y i is the source domain data, is the probability that the prediction sample in the source domain data is a rumor.

[0021] In the above technical solution of the present application, the step S4 specifically comprises the following steps:

[0022] S401: performing edge discard on the target domain data;

[0023] S402: using the teacher detection model trained by the source domain data in step S3 to perform pseudo-label annotation on the target domain data;

[0024] S403: taking the logical values of non-rumors and rumors output by the teacher detection model as the confidence of pseudo-label classification;

[0025] S404: combining the pseudo-label quantities of non-rumors and rumors in the target domain data, setting the upper limit and lower limit of the confidence of pseudo-labels, and obtaining the confidence threshold of non-rumors and rumors in the target domain data;

[0026] S405: screening the pseudo-labels of non-rumors and rumors in the target domain data according to the confidence threshold of non-rumors and rumors in the target domain data, and obtaining high-quality pseudo-labels of non-rumors and rumors in the target domain data.

[0027] In the above technical solution of the present application, the calculation formula of the confidence threshold T0 of the non-rumors in the target domain data is as follows:

[0028]

[0029] The calculation formula of the confidence threshold T1 of the rumors in the target domain data is as follows:

[0030]

[0031] wherein: p high and p​low are respectively the upper and lower limits of the confidence of the pseudo label, and n0 and n1 are respectively the number of non-rumor and rumor in the pseudo label of the target domain data in the current round.

[0032] In the technical scheme of the present application, the step S5 specifically comprises the following steps:

[0033] S501: Set the upper and lower limits of the cross-entropy loss weight according to the number of high-quality pseudo labels of non-rumor and rumor in the target domain data, and obtain the cross-entropy loss weight of non-rumor and rumor in the high-quality pseudo label;

[0034] S502: Use the student detection model to perform self-training on the target domain data, combine the cross-entropy loss weight of non-rumor and rumor in the high-quality pseudo label and the cross-entropy loss, and obtain the cross-entropy loss of the high-quality pseudo label after reweighting;

[0035] S503: Obtain the training loss of the target domain data according to the cross-entropy loss of the high-quality pseudo label after reweighting, the class-level contrastive learning loss and the sample-level contrastive learning loss.

[0036] In the technical scheme of the present application, the calculation formula of the cross-entropy loss weight ω0 of the non-rumor in the high-quality pseudo label is as follows:

[0037]

[0038] The calculation formula of the cross-entropy loss weight ω1 of the rumor in the high-quality pseudo label is as follows:

[0039]

[0040] wherein: ω high and ω low are respectively the upper and lower limits of the cross-entropy loss weight, and n ‘ 0 and n ‘ 1 are respectively the number of non-rumor and rumor in the high-quality pseudo label of the target domain data in the current round.

[0041] In the technical scheme of the present application, the calculation formula of the cross-entropy loss of the non-rumor in the high-quality pseudo label is as follows:

[0042]

[0043] The calculation formula of the cross-entropy loss of the rumor in the high-quality pseudo label is as follows:

[0044]

[0045] The cross-entropy loss of the high-quality pseudo label after reweighting is The calculation formula of the class-level contrast learning loss L is as follows:

[0046]

[0047] Wherein: N is the number of high-quality pseudo-labels, p N is the number of high-quality pseudo-labels, is the probability that the predicted sample in the high-quality pseudo-label sample is a rumor.

[0048] In the technical scheme of the present application, the class-level contrast learning loss L The calculation formula of the class-level contrast learning loss L is as follows:

[0049]

[0050] The calculation formula of the sample-level contrast learning loss L is as follows:

[0051]

[0052] Wherein: in the class-level contrast learning loss L, k is the target domain data with high-quality pseudo-labels in the current round, N is the number of high-quality pseudo-labels, p N is the number of high-quality pseudo-labels, τ is a temperature coefficient, and sim(·) represents a cosine similarity; is the positive sample set in the current round of target domain data with the same high-quality pseudo-label prediction probability, a is an element in the positive sample set, N is the number of high-quality pseudo-labels, is the feature vector of q output from the teacher detection model;

[0053] In the sample-level contrast learning loss L, |B Stu | is the data amount of the student detection model in the current round of training, m is the target domain data in the current round of training, n is the target domain data traversed in the current round of training, and N is the number of high-quality pseudo-labels, N is the number of high-quality pseudo-labels.

[0054] In the technical scheme of the present application, the calculation formula of the training loss L T of the target domain data is as follows:

[0055]

[0056] Wherein: ​cross-entropy loss with high-quality pseudo-labels, category-level contrastive learning loss, sample-level contrastive learning loss, and λ1, λ2 and λ3 are weights of each loss, λ1+λ2+λ3=1.

[0057] In a second aspect, the present application provides an electronic device, comprising a memory and a processor, the processor and the memory complete mutual communication through a bus; the memory stores program instructions which can be executed by the processor, and the processor calling the program instructions can execute the above-mentioned unsupervised cross-domain rumor detection method based on the teacher-student learning framework.

[0058] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the above-mentioned unsupervised cross-domain rumor detection method based on the teacher-student learning framework.

[0059] Compared with the prior art, the present application has the following beneficial effects:

[0060] The method of the present application is driven by the pre-training language model and the graph convolutional neural network model, overcomes the semantic and propagation structure feature differences of social media information between different subject fields, and through the design of dynamic screening of high-quality pseudo-labels and the introduction of contrastive learning loss, the guiding ability of the teacher detection model to the student detection model in the target domain data during each training is strengthened.

[0061] The method of the present application adopts an unsupervised self-training manner, combines a reweighted cross-entropy loss function and two types of contrastive learning loss functions, and applies consistent regularization constraints to the feature vectors output by the teacher detection model and the student detection model, thereby improving the effect of unsupervised cross-domain rumor detection. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 It is a principle diagram of the unsupervised cross-domain rumor detection method based on the teacher-student learning framework of the present application. DETAILED DESCRIPTION

[0063] The following examples are used to illustrate the present application, but are not used to limit the protection scope of the present application. If not specifically indicated, the technical means used in the examples are conventional means known to those skilled in the art. The test methods in the following examples are conventional methods, unless otherwise specified.

[0064] Example 1

[0065] Reference Figure 1The shown principle diagram of the unsupervised cross-domain rumor detection method based on the teacher-student learning framework of the application. The embodiment of the unsupervised cross-domain rumor detection method based on the teacher-student learning framework specifically includes the following steps:

[0066] S1: Construct a teacher detection model and a student detection model with the same structure by using a Roberta model and a BiGCN model, and set the parameters of the teacher detection model as Tea and the parameters of the student detection model as Stu .

[0067] S2: Obtain source domain data and target domain data (as shown in Table 1), and set the source domain data as S and the target domain data as T . The source domain data and the target domain data are both divided into two categories of non-rumor and rumor, the non-rumor data is marked as 0, and the rumor data is marked as 1.

[0068] Table 1 Dataset data

[0069]

[0070] S3: Perform edge dropping on the input source domain data with a propagation graph structure according to a probability p drop , and set p drop to 0.1, splice the root node text and the child node text to form a node pair, encode the semantic features of the text content of the node pair by using a pre-trained language model, and calculate the cross-entropy loss of the teacher detection model by using formula (1) . Iteratively optimize the parameters of the teacher detection model, and synchronize the trained parameters to the student detection model, to complete the parameter initialization of the teacher detection model and the student detection model.

[0071] S4: Perform pseudo-label annotation on the target domain data of different themes by using the teacher detection model, and select high-quality pseudo-labels. Step S4 specifically includes the following steps:

[0072] S401: Perform edge dropping on the target domain data with a propagation graph structure according to a probability p drop , and set p drop to 0.1.

[0073] S402: Perform pseudo-label annotation on the target domain data by using the teacher detection model trained by the source domain data in step S3.

[0074] S403: Take the logical values of non-rumor and rumor output by the teacher detection model as the confidence of pseudo-label classification.

[0075] S404: Combine the number of pseudo labels of non-rumors and rumors in the target domain data, i.e. n0 and n1, and set the upper and lower limits of the confidence of the pseudo labels, i.e. p high =0.9, p low =0.7, and substitute into formula (2) and formula (3) to calculate the confidence thresholds of non-rumor and rumor in the target domain data, namely T0 and T1.

[0076] S405: Filter the pseudo labels of non-rumors and rumors in the target domain data according to the confidence thresholds of non-rumors and rumors in the target domain data, and obtain high-quality pseudo labels of non-rumors and rumors in the target domain data.

[0077] S5: Use the student detection model to self-train the target domain data, combining the cross entropy loss and contrastive learning loss of high-quality pseudo labels to obtain the training loss of the target domain data. Step S5 specifically includes the following steps:

[0078] S501: Combine the number of non-rumors and rumors in high-quality pseudo labels, i.e. n ‘ 0 and n ‘ 1. Set the upper and lower limits of the cross entropy loss weights for non-rumors and rumors, i.e. ω high =3,ω low =1, and substitute into formula (4) and formula (5) to obtain the cross entropy loss weights of rumors and non-rumors in high-quality pseudo labels, namely ω0 and ω1.

[0079] S502: Use formula (6) and formula (7) to calculate the cross entropy loss of non-rumor and rumor in high-quality pseudo labels respectively and Substitute into formula (8) to calculate the cross entropy loss of high-quality pseudo-label reweighting

[0080] S503: Calculate the category-level contrastive learning loss according to formula (9) and formula (10) and sample-level contrastive learning loss Set λ1 = 0.8, λ2 = λ3 = 0.1, and substitute into formula (11) to calculate the training loss L of the target domain data T .

[0081] S6: Use the training loss backpropagation of the target domain data to optimize the student detection model parameters, and use the moving exponential average algorithm to smoothly superimpose the parameters of the trained and optimized student detection model onto the teacher detection model parameters, and return to step S2 to start the next round of training.

[0082] It is worth noting that in the first round of training, the upper limit of the cross entropy loss weight ω between non-rumors and rumors needs to be set high and the lower limit ω lowand the upper limit ω of the cross-entropy loss weight of non-rumor and rumor high and the lower limit ω low And in each subsequent training round, the system dynamically adjusts these upper and lower limits according to the high-quality pseudo-labels obtained in each training round, to balance the quality and quantity of pseudo-labels of target domain data, and achieve dynamic screening of high-quality pseudo-labels.

[0083] The training round is set to 200 times, and the experimental results are as follows:

[0084] As shown in Table 2, WCT represents the test results obtained according to the steps of the present embodiment. WCT-init represents the test results of the model proposed in the present application trained only in the source domain data. TD-RvNN, BU-RvNN, Bi-GCN, PLAN, PPA-WAE, PPA-WAE are the test results of different detection methods trained only in the source domain data. UCD-RD is the test result of another unsupervised cross-domain rumor detection method. Difference represents the detection performance difference of the model proposed in the present application compared with WCT-init after completing the target domain training. It can be seen that the test results of the present application (WCC) are significantly better than those of other detection methods.

[0085] Table 2 Test results of unsupervised cross-domain rumor detection methods

[0086]

[0087]

[0088] The above-described embodiments are only preferred embodiments of the present application, merely used to explain the present application, and are not intended to limit the scope of the present application. For those skilled in the art, of course, other embodiments can be easily made by substitution or change according to the technical content disclosed in the present specification, and therefore, any changes and improvements made on the principles of the present application shall be included in the scope of the patent application of the present application.

Claims

1. An unsupervised cross-domain rumor detection method based on a teacher-student learning framework, characterized by: The following steps are involved: S1: Construct a teacher detection model and a student detection model with the same structure; S2: Obtain source domain data and target domain data; S3: Use source domain data to perform supervised training on the teacher detection model and synchronize the trained parameters to the student detection model to complete the parameter initialization of the teacher detection model and the student detection model; S4: Use the teacher detection model to pseudo-label the target domain data and screen out high-quality pseudo-labels for the target domain data. Step S4 specifically includes the following steps: S401: discarding the target domain data; S402: Pseudo-label the target domain data using the teacher detection model trained on the source domain data in step S3; S403: Using the logical values ​​of non-rumor and rumor output by the teacher detection model as the confidence of the pseudo label classification; S404: Based on the number of pseudo labels of non-rumors and rumors in the target domain data, set upper and lower limits of the confidence of the pseudo labels to obtain confidence thresholds of non-rumors and rumors in the target domain data; S405: Filtering the pseudo labels of non-rumors and rumors in the target domain data based on the confidence thresholds of non-rumors and rumors in the target domain data, respectively, to obtain high-quality pseudo labels of non-rumors and rumors in the target domain data; S5: Use the student detection model to self-train the target domain data, combine the cross entropy loss and contrastive learning loss of high-quality pseudo labels, and obtain the training loss of the target domain data; step S5 specifically includes the following steps: S501: Based on the number of high-quality pseudo labels of non-rumors and rumors in the target domain data, set the upper and lower limits of the cross-entropy loss weight to obtain the cross-entropy loss weights of non-rumors and rumors in the high-quality pseudo labels; S502: Use the student detection model to self-train the target domain data, combine the cross entropy loss weights and cross entropy loss of non-rumors and rumors in the high-quality pseudo labels, and obtain the reweighted cross entropy loss of the high-quality pseudo labels; S503: Obtaining the training loss of the target domain data based on the reweighted cross entropy loss, category-level contrastive learning loss, and sample-level contrastive learning loss of the high-quality pseudo labels; S6: Use the training loss backpropagation of the target domain data to optimize the student detection model parameters, and use the moving exponential average algorithm to smoothly superimpose the parameters of the trained and optimized student detection model onto the teacher detection model parameters, and return to step S2 to start the next round of training.

2. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that The source domain data and the target domain data are divided into two categories: non-rumor and rumor, and the non-rumor data is marked as 0 and the rumor data is marked as 1.

3. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 or 2 is characterized in that The specific steps of using source domain data to perform supervised training on the teacher detection model in step S3 include: Perform edge discard on source domain data; The root node text is concatenated with the text of each child node to form a node pair, and the text content of the node pair is encoded with semantic features; Calculate the cross entropy loss of the teacher detection model and iteratively optimize the parameters of the teacher detection model; the cross entropy loss of the teacher detection model The calculation formula is as follows: Where: |B Tea | is the amount of data for the current training round of the teacher detection model, y i is the source domain data, is the probability that the sample in the source domain data is a rumor.

4. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that The calculation formula of the confidence threshold T0 of the non-rumor in the target domain data is as follows: The calculation formula of the confidence threshold T1 of the rumor in the target domain data is as follows: Where: p high and p low are the upper and lower limits of the confidence of the pseudo-label, n0 and n1 are the number of non-rumors and rumors in the pseudo-label of the target domain data in the current round.

5. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that The calculation formula of the cross entropy loss weight ω0 of non-rumor in high-quality pseudo-label is as follows: The calculation formula of the cross entropy loss weight ω1 of the rumor described in the high-quality pseudo label is as follows: Where: high and ω low are the upper and lower limits of the cross entropy loss weight, n ‘ 0 and n ‘ 1 are the number of non-rumors and rumors in the high-quality pseudo labels of the target domain data in the current round.

6. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that Cross entropy loss of non-rumor values ​​in high-quality pseudo-labels The calculation formula is as follows: Cross entropy loss for rumors in high-quality pseudo-labels The calculation formula is as follows: The cross entropy loss of high-quality pseudo-label reweighted The calculation formula is as follows: in: is high-quality pseudo-label data, N p is the number of high-quality pseudo labels, is the probability that a sample with high-quality pseudo-labels is a rumor.

7. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that The class-level contrastive learning loss The calculation formula is as follows: The sample-level contrastive learning loss The calculation formula is as follows: Where: In the category-level contrastive learning loss, k is the target domain data with high-quality pseudo labels in the current round, is the feature vector of k output from the student detection model, N p is the number of high-quality pseudo labels, τ is the temperature coefficient, and sim(·) represents the cosine similarity; is the set of positive samples with the same prediction probability as the high-quality pseudo-label in the target domain data of the current round, a is an element in the positive sample set, is the feature vector of a output from the teacher detection model, q is the high-quality pseudo label traversed during the current round of training, is the feature vector of q output from the teacher detection model; In sample-level contrastive learning loss, |B Stu | is the amount of data for the current round of training of the student detection model, m is the target domain data for the current round of training, and n is the target domain data traversed during the current round of training. and are the output feature vectors of m output from the teacher detection model and the student detection model respectively, is the feature vector of n output from the Student detection model.

8. The unsupervised cross-domain rumor detection method based on the teacher-student learning framework according to claim 1 is characterized in that The training loss L of the target domain data T The calculation formula is as follows: in: is the cross entropy loss weighted by the high-quality pseudo-labels, is the category-level contrastive learning loss, is the sample-level contrastive learning loss, λ1, λ2, and λ3 are the weights of each loss, λ1+λ2+λ3=1.

Images