Semi-supervised stance detection method based on category-aware curriculum learning

Abstract

The present invention specifically relates to a semi-supervised stance detection method based on category-aware curriculum learning, including: detecting predicted stance labels of tweets in a stance detection model; training the model includes: pre-training the stance detection model through a set of labeled tweets; inputting a set of unlabeled tweets into the pre-trained stance detection model, outputting pseudo-labels of unlabeled tweets, and generating a set of unlabeled tweets with pseudo-labels; selecting corresponding unlabeled tweets from a set of unlabeled tweets with pseudo-labels based on curriculum learning to form a set of pseudo-labeled tweets; optimizing the pre-trained stance detection model through the set of pseudo-labeled tweets to obtain a final stance detection model. The present invention reduces the demand for labeled data for model training through semi-supervised stance detection, and at the same time pays more attention to the quality of pseudo-labels required for semi-supervised stance detection through a curriculum learning framework, and fully considers the problem of label category balance when selecting pseudo-labeled tweets.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet big data and machine learning, and in particular to a semi-supervised stance detection method based on category-aware curriculum learning. Background Art

[0002] Social media is a barometer of the emotional state of contemporary society. Users express their opinions on something, a person, or a topic through social media, which we call stance. Tweet stance detection is to detect the stance of the user's text on the target (such as support, opposition, neutrality) given a user's posted text and stance target. Stance detection is an emerging and rapidly developing task in natural language processing. One important point is that it is widely used on social media, such as hate speech detection, opinion research, fake news detection, etc.

[0003] In real-world scenarios, data labeling is an issue that researchers have to consider. Due to the high cost of labeling, it is impossible to label texts on social media one by one, resulting in a lack of labeled data. Neural networks perform well in dealing with text stance detection problems, but training neural network models requires a large amount of labeled data. Social media generates a huge amount of information every day, and it is extremely difficult to label enough labeled data for each application scenario.

[0004] Semi-supervised and active learning are common response mechanisms. Active learning requires manual confirmation of data during the training process. In order to save labor costs, semi-supervised detection of stances on social media has begun to be adopted in existing technologies. Semi-supervised stance detection is an important research work in the field of natural language processing.

[0005] However, existing semi-supervised stance detection generally generates pseudo labels for unlabeled data by voting based on the homogeneity of social networks, without paying attention to the quality of the generated pseudo labels, which leads to poor training results and detection performance of the semi-supervised stance detection model. At the same time, with the rapid development of artificial intelligence, the effectiveness of curriculum learning in deep learning has been proven. However, when selecting pseudo-label tweets through curriculum learning, the existing method does not pay attention to the problem of data category imbalance, resulting in scarce data in some categories. However, if all categories are selected under the same conditions, it is unfair to the weak class because the selected data is more biased towards the strong class. Finally, the classification of the model is biased towards the strong class, which also leads to poor training results and detection performance of the semi-supervised stance detection model.

[0006] Therefore, how to improve the training effect and detection performance of the semi-supervised stance detection model is a technical problem that needs to be solved urgently. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a semi-supervised stance detection method based on category-aware curriculum learning, reduce the demand for labeled data for model training through semi-supervised stance detection, and at the same time, pay more attention to the quality of pseudo-labels required for semi-supervised stance detection through the curriculum learning framework, and fully consider the problem of label category balance when selecting pseudo-label tweets, so as to improve the training effect and detection performance of the semi-supervised stance detection model, and ensure the accuracy and effectiveness of semi-supervised stance detection.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A semi-supervised stance detection method based on category-aware curriculum learning, including:

[0010] S1: Get the tweet to be predicted;

[0011] S2: Input the tweet to be predicted into the trained stance detection model and output the predicted stance label of the corresponding tweet;

[0012] Train the stance detection model by following the steps below:

[0013] S201: Obtain a set of labeled tweets and a set of unlabeled tweets as training data;

[0014] S202: pre-training a stance detection model using a set of labeled tweets to obtain a pre-trained stance detection model;

[0015] S203: Input the unlabeled tweet set into the pre-trained stance detection model, output the predicted stance label of each unlabeled tweet as its pseudo label, and then generate a set of unlabeled tweets with pseudo labels;

[0016] S204: performing course learning by selecting tweets according to the label category ratio, and then selecting corresponding unlabeled tweets from the unlabeled tweet set with pseudo labels to form a pseudo-labeled tweet set;

[0017] S205: Optimizing the pre-trained stance detection model through the pseudo-labeled tweet set until the model converges to obtain a trained stance detection model;

[0018] S3: The predicted stance label of the corresponding tweet is used as its stance detection result.

[0019] Preferably, the pre-training of the stance detection model is achieved through the following steps:

[0020] S2021: Get the text and hashtag of the labeled tweets from the collection of labeled tweets;

[0021] S2022: Extract text feature representation through bidirectional long short-term memory network to obtain text-based tweet representation;

[0022] S2023: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags;

[0023] S2024: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate the final tweet representation;

[0024] The formula is described as:

[0025]

[0026] Where: The final tweet indicates; represents a text-based representation of tweets; represents the tweet representation based on hashtag; (·;·) represents the concatenation operation;

[0027] S2025: Input the final tweet representation into a multi-layer perceptron and output the predicted stance label of the corresponding tweet;

[0028] The formula is described as:

[0029]

[0030] Where: Represents the predicted stance label; MLP represents multi-layer perceptron;

[0031] S2026: Calculate the label loss based on the predicted stance label and the true stance label of the tweet, and optimize the model parameters;

[0032] S2027: Repeat steps S2021 to S2026 until the stance detection model converges.

[0033] Preferably, the text-based tweet representation is generated by the following steps:

[0034] 1) Represent the text of the tweet as a sequence of words {w1,w2,…,w n};

[0035] 2) Map the text into d through the word encoding layer w dimensional vector sequence

[0036] 3) Use the bidirectional long short-term memory network layer according to the vector sequence H w Extract the intermediate sequence representation H of the text bls ;

[0037] The formula is described as: Hbls =BiLSTM(H w );

[0038] 4) The intermediate sequence is represented by H bls Perform maximum pooling to obtain text-based tweet representation

[0039] Preferably, the hashtag-based tweet representation is generated by the following steps:

[0040] 1) The hashtag of the tweet is represented as {#t1,#t2,…,#t m};

[0041] 2) Use the tag encoding layer to encode each Hashtag#t i Mapping is performed to obtain d t Dense vector sequence of dimension

[0042] 3) Use the self-attention mechanism to determine the dense vector sequence H t The contribution of each vector in the prediction task is used to generate the vector representation of the Hashtag;

[0043] The formula is described as:

[0044] First, the Query, Key, and Value in attention are calculated using the following formula:

[0045]

[0046]

[0047]

[0048] Where: W q ,W k ,W v ,b q ,b k ,b v is a learnable parameter, relu is an activation function;

[0049] Then, the attention weight is calculated by the following formula:

[0050]

[0051] Where: represents the query vector;

[0052] Finally, the vector representation of Hashtag is calculated by the following formula:

[0053]

[0054] Where: Represents the vector representation of Hashtag;

[0055] 4) Vector representation of hashtag Perform maximum pooling to obtain hashtag-based tweet representation

[0056] Preferably, the pseudo labels of unlabeled tweets are generated by the following formula:

[0057]

[0058] Where: Pseudo labels representing unlabeled tweets; s i represents the input unlabeled tweets; F represents the pre-trained stance detection model; θ represents the learnable parameters of the stance detection model.

[0059] Preferably, based on the curriculum learning framework, corresponding unlabeled tweets are selected from a set of unlabeled tweets with pseudo labels according to the category ratio of the set of labeled tweets, and then a set of pseudo-labeled tweets is generated by combining the pseudo-labels of the selected unlabeled tweets.

[0060] Preferably, the pseudo-labeled tweet set is generated by the following steps:

[0061] S2041: Calculate the entropy value of each unlabeled tweet in the set of unlabeled tweets with pseudo labels;

[0062] The formula is described as:

[0063]

[0064] Where: H i Represents unlabeled tweets i The entropy value of Represents unlabeled tweets i Pseudo-labels; |C| represents the number of unlabeled tweet label categories;

[0065] S2042: for each label category in the set of unlabeled tweets with pseudo labels, sort the unlabeled tweets belonging to the label category in ascending order of entropy value;

[0066] S2043: Calculate the weight of each label category in the set of labeled tweets;

[0067] The formula is described as:

[0068]

[0069] Where: Indicates the jth label category c j The weight of the proportion; i Indicates tweets with labels i The true position label category; D l Represents the number of tweets in the set of labeled tweets;

[0070] S2044: Calculate the number of selected unlabeled tweets of each label category according to the weight of each label category;

[0071] The formula is described as:

[0072]

[0073] Where: Indicates the jth label category c j The number of unlabeled tweets to be selected; q represents the total number of unlabeled tweets to be selected;

[0074] S2045: For each label category, select the first unlabeled tweets with pseudo labels sorted by entropy value. pseudo-labeled tweets;

[0075] The formula is described as:

[0076]

[0077] Where: Represents the selection of unlabeled tweets from the set of pseudo-labeled tweets The jth label category c j The set of unlabeled tweets; Indicates that the label category in the set of unlabeled tweets with pseudo labels is c j The collection of unlabeled tweets;

[0078] S2046: Merge all selected unlabeled tweets and reorder them in ascending order of entropy value to generate a set of pseudo-labeled tweets.

[0079] Preferably, the optimization training of the pre-trained stance detection model is achieved through the following steps:

[0080] S2051: Get the text and hashtag of unlabeled tweets from the pseudo-labeled tweet collection;

[0081] S2052: Extract text feature representation through bidirectional long short-term memory network to obtain text-based tweet representation;

[0082] S2053: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags;

[0083] S2054: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate a final tweet representation;

[0084] S2055: Input the final tweet representation into a multi-layer perceptron, and output a pseudo-predicted stance label corresponding to the unlabeled tweet;

[0085] S2056: Calculate the unlabeled loss based on the pseudo-predicted stance labels and pseudo-labels of unlabeled tweets, and optimize the model parameters;

[0086] S2057: Repeat steps S2051 to S2056 until the stance detection model converges.

[0087] Preferably, the label loss is calculated by the following formula:

[0088]

[0089]

[0090] Where: Indicates label loss; represents the predicted stance label; Indicates the true position label; β i is the loss discount weight of the tweets of the jth label category; ε is a minimum value; Indicates the jth label category c j The proportion weight of .

[0091] Preferably, the unlabeled loss is calculated by the following formula:

[0092]

[0093] Where: represents the unlabeled loss; represents the pseudo-predicted stance label; represents a pseudo label; Indicates the jth label category c j The proportion weight of .

[0094] Compared with the prior art, the semi-supervised stance detection method based on category-aware curriculum learning in the present invention has the following beneficial effects:

[0095] The present invention uses a set of labeled tweets and a set of unlabeled tweets as training data to realize the training of a semi-supervised stance detection model. Semi-supervised learning can reduce the demand for labeled data in model training, save data labeling costs and labor costs, thereby reducing the difficulty and cost of model training and ensuring the effect of semi-supervised stance detection.

[0096] When training a semi-supervised stance detection model, the present invention first uses labeled tweets to train a pre-trained model, then generates pseudo-labels for unlabeled tweets through the pre-trained model, and finally selects unlabeled tweets with pseudo-labels through course learning to achieve optimization training of the pre-trained model. On the one hand, the present invention selects pseudo-labeled tweets through course learning. Compared with the prior art of generating pseudo-labels for tweets by voting based on social network homogeneity, the present invention considers the confidence (entropy value) of the pseudo-labels, pays more attention to the quality of the pseudo-labels, and can better optimize the model through pseudo-labeled tweets, thereby improving the training effect of the semi-supervised stance detection model and ensuring the effectiveness of semi-supervised stance detection. On the other hand, the present invention adopts a method of selecting tweets according to the proportion of label categories for course learning. Compared with the existing course learning method, the present invention fully considers the problem of label category balance and improves the problem of selecting pseudo-labeled tweets biased towards strong categories in the prior art, thereby further improving the training effect and detection performance of the semi-supervised stance detection model and better ensuring the accuracy and effectiveness of semi-supervised stance detection.

[0097] The semi-supervised stance detection method and stance detection model based on the curriculum learning framework in the present invention were experimented with public data sets. The experimental results show that the model trained by the method of the present invention has better effect than the optimal model SANDS in the prior art. At the same time, by controlling the participation of different proportions of unlabeled data in training, it is found that the training method of the present invention can achieve the effect of the optimal model SANDS by using only half of the unlabeled data. Finally, through different ablation experiments, the effectiveness of the present invention in semi-supervised stance detection based on the curriculum learning framework is verified. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0099] Figure 1 The logical block diagram of the semi-supervised stance detection method based on category-aware curriculum learning;

[0100] Figure 2 This is the network structure diagram of the stance detection model;

[0101] Figure 3 Comparison of F1 scores of SDCL and the best baseline SANDS in each category;

[0102] Figure 4 is the hyperparameter N c Performance on different splits of the two datasets. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0104] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0105] The following is a further detailed description through specific implementation methods:

[0106] Example:

[0107] This embodiment discloses a semi-supervised stance detection method based on category-aware curriculum learning.

[0108] like Figure 1 As shown, the semi-supervised stance detection method based on category-aware curriculum learning is characterized by comprising:

[0109] S1: Get the tweet to be predicted;

[0110] S2: Input the tweet to be predicted into the trained stance detection model (hereinafter referred to as SDCL), and output the predicted stance label of the corresponding tweet;

[0111] In semi-supervised stance detection, the training data consists of labeled tweets. Tweets with no hashtags (0 <N l < <N u ), the test set includes the tweets to be predicted s i represents the i-th sentence, y i represents the stance label of the i-th tweet, N l 、N u Respectively represent the number of labeled tweets and the number of unlabeled tweets in the training set, N t represents the number of tweets in the test set. Let C represent the label category set, C={c1,c2,…,c n}, n represents the number of label categories, c i Represents the i-th label category. The purpose of semi-supervised stance detection is to learn a stance detection model F to predict the stance label of the input tweet, that is, Where θ is the learnable parameter of the model.

[0112] Combination Figure 2 As shown, the stance detection model is trained through the following steps:

[0113] S201: Obtain a set of labeled tweets and a set of unlabeled tweets as training data;

[0114] S202: pre-training a stance detection model using a set of labeled tweets to obtain a pre-trained stance detection model;

[0115] S203: Input the unlabeled tweet set into the pre-trained stance detection model, output the predicted stance label of each unlabeled tweet as its pseudo label, and then generate a set of unlabeled tweets with pseudo labels;

[0116] S204: performing course learning by selecting tweets according to the label category ratio, and then selecting corresponding unlabeled tweets from the unlabeled tweet set with pseudo labels to form a pseudo-labeled tweet set;

[0117] S205: Optimizing the pre-trained stance detection model through the pseudo-labeled tweet set until the model converges to obtain a trained stance detection model;

[0118] S3: The predicted stance label of the corresponding tweet is used as its stance detection result.

[0119] The present invention uses a set of labeled tweets and a set of unlabeled tweets as training data to realize the training of a semi-supervised stance detection model. Semi-supervised learning can reduce the demand for labeled data in model training, save data labeling costs and labor costs, thereby reducing the difficulty and cost of model training and ensuring the effect of semi-supervised stance detection.

[0120] When training a semi-supervised stance detection model, the present invention first uses labeled tweets to train a pre-trained model, then generates pseudo-labels for unlabeled tweets through the pre-trained model, and finally selects unlabeled tweets with pseudo-labels through course learning to achieve optimization training of the pre-trained model. On the one hand, the present invention selects pseudo-labeled tweets through course learning. Compared with the prior art of generating pseudo-labels for tweets by voting based on social network homogeneity, the present invention considers the confidence (entropy value) of the pseudo-labels, pays more attention to the quality of the pseudo-labels, and can better optimize the model through pseudo-labeled tweets, thereby improving the training effect of the semi-supervised stance detection model and ensuring the effectiveness of semi-supervised stance detection. On the other hand, the present invention adopts a method of selecting tweets according to the proportion of label categories for course learning. Compared with the existing course learning method, the present invention fully considers the problem of label category balance and improves the problem of selecting pseudo-labeled tweets biased towards strong categories in the prior art, thereby further improving the training effect and detection performance of the semi-supervised stance detection model and better ensuring the accuracy and effectiveness of semi-supervised stance detection.

[0121] The semi-supervised stance detection method and stance detection model based on the curriculum learning framework in the present invention were experimented with public data sets. The experimental results show that the model trained by the method of the present invention has better effect than the optimal model SANDS in the prior art. At the same time, by controlling the participation of different proportions of unlabeled data in training, it is found that the training method of the present invention can achieve the effect of the optimal model SANDS by using only half of the unlabeled data. Finally, through different ablation experiments, the effectiveness of the present invention in semi-supervised stance detection based on the curriculum learning framework is verified.

[0122] In the specific implementation process, a given tweet s consists of a piece of text and one or more hashtags (Hashtag is an existing concept in the form of #PelosiLiar, similar to Weibo). In the initialization stage, we process the text and the real stance label separately and fuse them after processing. Specifically, the pre-training of the stance detection model is achieved through the following steps:

[0123] S2021: Get the text and hashtag of the labeled tweets from the collection of labeled tweets;

[0124] S2022: Extract text feature representation through Bi-directional Long Short Term Memory Network (Bi-LSTM) to obtain text-based tweet representation;

[0125] In this embodiment, a text-based tweet representation is generated through the following steps:

[0126] 1) Represent the text of the tweet as a sequence of words {w1,w2,…,w n};

[0127] 2) The one-hot representation of the text is mapped into d through the word encoding layer (Word Embedding layer) w dimensional vector sequence

[0128] 3) Use the bidirectional long short-term memory network layer according to the vector sequence H w Extract the intermediate sequence representation H of the text bls ;

[0129] The formula is described as: H bls =BiLSTM(H w );

[0130] 4) The intermediate sequence is represented by H bls Perform maximum pooling to obtain text-based tweet representation

[0131] S2023: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags;

[0132] In this embodiment, a hashtag-based tweet representation is generated through the following steps:

[0133] 1) The hashtag of the tweet is represented as {#t1,#t2,…,#t m};

[0134] 2) Through the tag encoding layer (Hashtag Embedding layer), each Hashtag#t i The one-hot representation is mapped to obtain d t Dense vector sequence of dimension

[0135] 3) Use the self-attention mechanism to determine the dense vector sequence H t The contribution of each vector in the prediction task is used to generate the vector representation of the Hashtag;

[0136] The formula is described as:

[0137] First, the Query, Key, and Value in attention are calculated using the following formula:

[0138]

[0139]

[0140]

[0141] Where: W q ,W k ,W v ,b q ,b k ,b v is a learnable parameter, relu is an activation function;

[0142] Then, the attention weight is calculated by the following formula:

[0143]

[0144] Where: represents the query vector;

[0145] Finally, the vector representation of Hashtag is calculated by the following formula:

[0146]

[0147] Where: Represents the vector representation of Hashtag;

[0148] 4) Vector representation of hashtag Perform maximum pooling to obtain hashtag-based tweet representation

[0149] S2024: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate the final tweet representation;

[0150] The formula is described as:

[0151]

[0152] Where: The final tweet indicates; represents a text-based representation of tweets; represents the tweet representation based on hashtag; (·;·) represents the concatenation operation;

[0153] S2025: Input the final tweet representation into a multi-layer perceptron and output the predicted stance label of the corresponding tweet;

[0154] The formula is described as:

[0155]

[0156] Where: Represents the predicted stance label; MLP represents multi-layer perceptron;

[0157] It should be noted that, like SANDS, we also use the dropout layer to deal with overfitting and noise.

[0158] S2026: Calculate the label loss based on the predicted stance label and the true stance label of the tweet, and optimize the model parameters;

[0159] S2027: Repeat steps S2021 to S2026 until the stance detection model converges.

[0160] In the specific implementation process, inspired by the human learning process, curriculum learning was proposed as a training strategy. The essence of curriculum learning is to train the machine learning model from simple samples and gradually increase the training difficulty of the samples. Bengio et al. (reference Bengio Y, Louradour J, Collobert R, et al. Curriculum learning) showed that curriculum learning improves the generalization ability of the model and guides the model to a better parameter space. Jiang et al. (reference Jiang L, Meng D, Zhao Q, et al. Self-paced curriculum learning) managed curriculum learning as an optimization problem. Sachan and Xing (reference Sachan M, Xing E. Easy questions first? a casestudy on curriculum learning for question answering) applied self-paced learning to question answering research. Tay et al. (reference Tay Y, Wang S, Tuan LA, et al. Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives) proposed a curriculum pointer generator network for long narrative reading comprehension. Curriculum learning has been widely used by scholars in computer vision, natural language processing, and graph classification. However, no worker has used curriculum learning to deal with the problem of class imbalance.

[0161] The course learning framework of the present invention is built based on the generated pseudo labels, and we propose a “difficulty measurer” to measure the quality of the generated pseudo labels. Specifically, for the i-th unlabeled tweet s i We use the pre-trained model F to predict its category (pseudo label) and get the predicted category distribution of unlabeled tweets In order to measure the quality of the generated pseudo-labels, we calculate the entropy of each predicted category distribution. The smaller the entropy value of a tweet, the easier it is to learn, that is, it belongs to a simple sample. On the contrary, the larger the entropy value, the more difficult it is to learn, that is, it belongs to a difficult sample.

[0162] In the prior art, after calculating the entropy value of the category distribution for unlabeled tweets, the unlabeled tweets are directly sorted according to the entropy value. Although this method achieves good results, it still has limitations and cannot effectively deal with the problem of category imbalance in the data set. Therefore, the present invention assumes that the category distribution of labeled tweets is basically consistent with the category distribution of unlabeled tweets. To this end, the present invention is based on the course learning framework, selects corresponding unlabeled tweets from the set of unlabeled tweets with pseudo-labels through the category ratio of the set of labeled tweets, and then generates a set of pseudo-labeled tweets in combination with the pseudo-labels of the selected unlabeled tweets. The specific process is as follows:

[0163] First, generate pseudo labels for unlabeled tweets using the following formula:

[0164]

[0165] Where: Pseudo labels representing unlabeled tweets; s i represents the input unlabeled tweets; F represents the pre-trained stance detection model; θ represents the learnable parameters of the stance detection model.

[0166] S2041: Calculate the entropy value of each unlabeled tweet in the set of unlabeled tweets with pseudo labels;

[0167] The formula is described as:

[0168]

[0169] Where: H i Represents unlabeled tweets i The entropy value of Represents unlabeled tweets i Pseudo-labels; |C| represents the number of unlabeled tweet label categories;

[0170] It should be noted that since tweets with too high entropy values ​​are difficult for the model to learn, we filter out tweets that are difficult for the model to classify (those with larger entropy values ​​in each category).

[0171] S2042: for each label category in the set of unlabeled tweets with pseudo labels, sort the unlabeled tweets belonging to the label category in ascending order of entropy value;

[0172] S2043: Calculate the weight of each label category in the set of labeled tweets;

[0173] The formula is described as:

[0174]

[0175] Where: Indicates the jth label category c j The weight of the proportion; i Indicates tweets with labels i The true position label category; D l Represents the number of tweets in the set of labeled tweets;

[0176] S2044: Calculate the number of selected unlabeled tweets of each label category according to the weight of each label category;

[0177] The formula is described as:

[0178]

[0179] Where: Indicates the jth label category c j The number of unlabeled tweets to be selected; q represents the total number of unlabeled tweets to be selected;

[0180] S2045: For each label category, select the first unlabeled tweets with pseudo labels sorted by entropy value. pseudo-labeled tweets;

[0181] The formula is described as:

[0182]

[0183] Where: Represents the selection of unlabeled tweets from the set of pseudo-labeled tweets The jth label category c j The set of unlabeled tweets; Indicates that the label category in the set of unlabeled tweets with pseudo labels is c j The collection of unlabeled tweets;

[0184] S2046: Merge all selected unlabeled tweets and reorder them in ascending order of entropy value to generate a set of pseudo-labeled tweets.

[0185] The present invention selects pseudo-labeled tweets through course learning. Compared with the prior art that generates pseudo-labels for tweets by voting based on social network homogeneity, the present invention considers the confidence (entropy value) of pseudo-labels and pays more attention to the quality of pseudo-labels, thereby being able to better optimize the model through pseudo-labeled tweets, thereby improving the training effect of the semi-supervised stance detection model and ensuring the effectiveness of semi-supervised stance detection. At the same time, the present invention adopts the method of selecting tweets according to the proportion of label categories for course learning. Compared with the existing course learning method, the present invention fully considers the problem of label category balance and improves the problem of biased strong categories in selecting pseudo-labeled tweets in the prior art, thereby further improving the training effect and detection performance of the semi-supervised stance detection model and better ensuring the accuracy and effectiveness of semi-supervised stance detection.

[0186] In the specific implementation process, the optimization training of the pre-trained stance detection model is achieved through the following steps:

[0187] S2051: Get the text and hashtag of unlabeled tweets from the pseudo-labeled tweet collection;

[0188] S2052: Extract text feature representation through bidirectional long short-term memory network to obtain text-based tweet representation;

[0189] S2053: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags;

[0190] S2054: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate a final tweet representation;

[0191] S2055: Input the final tweet representation into a multi-layer perceptron, and output a pseudo-predicted stance label corresponding to the unlabeled tweet;

[0192] S2056: Calculate the unlabeled loss based on the pseudo-predicted stance labels and pseudo-labels of unlabeled tweets, and optimize the model parameters;

[0193] S2057: Repeat steps S2051 to S2056 until the stance detection model converges.

[0194] It should be noted that the relevant formulas and logic involved in the unlabeled tweet training model are consistent with the aforementioned labeled tweet training model.

[0195] In the specific implementation process, the present invention uses real label supervision and pseudo label supervision to optimize the model. In the pre-training stage, the real label supervision process is used to pre-train the classification model, and then the trained model is used to classify and predict the unlabeled tweets to obtain pseudo labels. Then, the sorted tweets with pseudo labels are obtained through course learning, and pseudo label supervised classification is performed.

[0196] 1) In the true label supervision stage, the model is trained using a labeled dataset and cross entropy is used as the loss function.

[0197] The formula is described as:

[0198]

[0199]

[0200] Where: Indicates label loss; represents the predicted stance label; Indicates the true position label; β i is the loss discount weight of the tweets of the jth label category, which is used to deal with the category imbalance problem. ε is a minimum value to avoid the denominator being 0; Indicates the jth label category c j The proportion weight of .

[0201] 2) In the pseudo-label supervision stage, the training data consists of tweets with pseudo labels sorted by the course learning module, and cross entropy is also used to optimize the model.

[0202] The formula is described as:

[0203] The unlabeled loss is calculated by the following formula:

[0204]

[0205] Where: represents the unlabeled loss; represents the pseudo-predicted stance label; represents a pseudo label; Indicates the jth label category c j The proportion weight of .

[0206] 3) The total loss function of the final stance detection model is:

[0207]

[0208] In order to better illustrate the advantages of the technical solution of the present invention, this embodiment discloses the following experiment.

[0209] 1. Experimental setup

[0210] 1.1 Dataset

[0211] To facilitate fair comparison with other models, we use two datasets provided by SANDS. The dataset statistics are shown in Table 1, where 500, 1000, and 1500 represent the number of labeled tweets during training. StanceUS is a dataset used in this paper, which collects 59,684 tweets from 24,490 users in the United States, with an average of 2.44 tweets per user. StanceIN is another dataset used in this paper, which collects 176,613 tweets from 63,230 users in India, with an average of 2.79 tweets per user.

[0212] Both datasets are politically related tweets. For StanceUS, the tweets’ stance labels can be divided into Pro-Dem, Anti-Dem, Pro-Rep, Anti-Rep and others. For StanceIN, the tweets’ stance labels can be divided into Pro-BJP, Anti-BJP, Pro-INC, Anti-INC, Pro-AAP, Anti-AAP and others. Finally, 3822 and 4185 labeled tweets were collected from the two datasets StanceUS and StanceIN, respectively. The distribution of labeled tweets in different categories is shown in Table 1. This paper uses macro-averaged F1 (Mac-F1) to evaluate the model effect.

[0213] Table 1 Distribution of training and test samples in different partitions of the annotated dataset

[0214]

[0215] 1.2 Baseline Model

[0216] In order to verify the effectiveness of the model framework proposed in this paper, the experimental results of this paper are compared with the most advanced methods. The baseline results used in this paper are all from SANDS (reference Dutta S, Caur S, Chakrabarti S, et al. Semi-supervised stance detection of tweets via distant network supervision). The following are some supervised baselines:

[0217] SVM (reference Mohammad SM, Sobhani P, Kiritchenko S.Stance and sentiment intweets) proposed a linear kernel support vector machine framework with different text features.

[0218] TAN (reference Du J, Xu R, He Y, et al.Stance classification with target-specific neural attention networks) proposed a target-specific attention

[0219] Bi-LSTM framework of the mechanism.

[0220] SiamNet, a Siamese adaptation framework of LSTM proposed based on the work of Conforti et al. (Reference Conforti C, Berndt J, Pilehvar MT, et al. Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter).

[0221] BICE (reference Augenstein I, T, Vlachos A, et al. Stance detection with bidirectional conditional encoding) This model uses Bi-LSTM to compute tweet representations conditioned on the target.

[0222] BERT (base, uncased) pre-trained by BERT (see Devlin J, Chang MW, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding) and then passes through a linear layer for stance prediction.

[0223] ConvNet: A convolutional model trained in a purely supervised manner using only labeled data.

[0224] BLSTM: A bidirectional LSTM model trained in a purely supervised manner using only labeled data.

[0225] Here are several semi-supervised baselines:

[0226] LP-SVM (from Zhou D, Bousquet O, Lal T, et al. Learning with local and global consistency) is a semi-supervised method of label diffusion using the SVM model as the base classifier.

[0227] ST-ConvNet is a semi-supervised method for self-learning using ConvNet as a base classifier. This framework selects the best k samples from an unlabeled dataset (based on the predicted class probabilities) and augments the labeled data at each iteration.

[0228] ST-BLSTM is similar to ST-ConvNet, which replaces the base classifier with a BLSTM model.

[0229] UST (reference Mukherjee S, Awadallah A. Uncertainty-aware self-training for few-shot text classification) proposed a semi-supervised method that used a self-training framework and BERT for stance classification.

[0230] GCN-ConvNet is a semi-supervised node classification method similar to (see Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks), which uses ConvNet as a node feature extractor and then uses two graph convolution layers on the subsequent network.

[0231] GCN-BLSTM is similar to GCN-ConvNet, replacing the ConvNet with BiLSTM layers.

[0232] SANDS (reference Dutta S, Caur S, Chakrabarti S, et al. Semi-supervised stance detection of tweets via distant network supervision) adopts a classification model with two functional views and uses distant supervision signals from social networks to optimize the model.

[0233] 4.1.3 Parameter settings

[0234] In this experiment, the word embedding layer is initialized by 200-dimensional Glove, and the tags in the text (#t i ) is initialized to 128 dimensions, the early stop is set to 5, the training epoch is set to 15, and the Adam optimizer learning rate is set to 0.0001. The multilayer perceptron (MLP) used in this paper has layer ∈ {1, 2} layers and the dimension of the hidden layer of the multilayer perceptron hd ∈ {200, 300, 400, 500, 600}.

[0235] 2. Overall experiment

[0236] The overall experimental results are shown in Table 2, where the bold results are the best results. Compared with previous supervised methods and semi-supervised methods, the model proposed in this paper shows the best results on both datasets. The experimental results in this paper are all averaged over five experiments. Compared with the purely supervised classifier, the Mac-F1 scores of the StanceUS dataset on three different partitions of 500, 1000, and 1500 are improved by 0.115, 0.041, and 0.065, respectively. The Mac-F1 scores of the StanceIN dataset are improved by 0.082, 0.061, and 0.073, respectively. Compared with the latest semi-supervised method SANDS, the three different partitions of the StanceUS dataset are improved by 0.015, 0.011, and 0.025, respectively. StanceIN is improved by 0.012, 0.011, and 0.013, respectively. By comparing with various models, the effectiveness of the model in this paper is more comprehensively verified.

[0237] Table 2 F1 scores of all models under different partitions of labeled training data on StanceUS and StanceIN

[0238]

[0239] Under the same data partitioning, the model works better in StanceUS than in StanceIN. One important reason for this difference is the different quality of tweets sampled from the two countries. StanceUS tweets are pure English, while StanceIN tweets usually include some special symbols and local Indian language, and the latter dataset is mixed with a lot of noise. In contrast, there are too many categories in StanceIN, which is also the reason for the difficulty in classification.

[0240] SANDS uses a homogeneous network and the followers’ tweets to vote for positions to generate pseudo labels, but the quality of its generation has not been measured. Secondly, the requirements for the dataset are relatively high. The followers’ tweets must be counted, and the tweets must discuss the same position. To a certain extent, SANDS is not universal. In comparison, the model proposed in this paper is more generalizable.

[0241] 3. Ablation experiment

[0242] In order to verify the SDCL model proposed in this paper, a series of ablation experiments were conducted. The experimental results are shown in Table 3. w / ocurriculum means removing curriculum learning and disrupting the final sorted samples with pseudo labels, that is, the samples are disordered during training. After removing curriculum learning, the model effect on StanceIN has a more obvious decline, and the decline on StanceUS is not significant. Through the analysis of the data, the StanceUS dataset has less noise, and the StanceIN dataset has more noise. On the noisy dataset, curriculum learning can better optimize the model. w / o pretrain means removing model pretraining, and the model effect is significantly reduced. This paper is based on generating pseudo labels to solve the problem of lack of labels. In addition, the category distribution of the dataset in this paper is extremely unbalanced. Without pretraining, directly generating pseudo labels for unlabeled tweets will result in poor quality of generated pseudo labels, which will lead to a decrease in the overall Mac-F1 score. w / o category-aware means removing category awareness, using the traditional curriculum learning method, and performing analogy-free differentiation selection on pseudo labels. Through experiments, the model effect has a significant decline. Ablation experiments are conducted on each module, and it can be seen that each module helps to improve the model effect.

[0243] Table 3 Ablation experiments of SDCL on two datasets

[0244]

[0245] 4. Analysis of the impact of the proportion of unlabeled data

[0246] In order to explore how the number of unlabeled tweets affects the model, we used 80%, 50%, 30%, and 10% of the original unlabeled data for the two datasets to train the model, and the experimental results are shown in Table 4. The results of the entire experiment show that as the proportion of unlabeled datasets increases, the classification effect of the model improves. When the data is divided into 1.5k, the proportion of unlabeled data in the two datasets is 10%, and the model can also achieve good classification results. At the same time, this paper found that when the proportion of unlabeled datasets is 50%, the model achieves the best SANDS effect in the three divisions of the two datasets, proving the effectiveness of the model in this paper.

[0247] Table 4 Mac-F1 trained by SDCL using different proportions of unlabeled data

[0248]

[0249] 5. Handling of category imbalance

[0250] Most real-world datasets have the problem of imbalanced categories, and the model tends to favor the dominant category during training. This scenario also applies to the dataset used in this article, as shown in Table 1. In order to explore the problem of model handling imbalanced categories, this article reproduces the code provided by SANDS and compares the classification results of each category with the experimental results of this article. Figure 3 As shown in the figure, we plotted the F1 scores of the models in the two datasets divided into 1500. The experiment found that in the StanceUS advantage category, the model proposed in this paper is slightly better than the SANDS model, and in most of the weak category classifications, the model proposed in this paper has a significant improvement over the model proposed by SANDS, such as Anti-Dem and Anti-Rep. Even in the other category, the model effect is comparable to that of SANDS. On the StanceIN dataset, the two models perform equally in the advantage category, and in the weak category Pro-INC classification, the model proposed in this paper is much better than

[0251] SANDS, when classifying the Anti-INC category, we can see that neither the SANDS reproduced in this article nor the model proposed in this article can effectively classify the data in this category. By observing the data set, it is not difficult to find that the number of data in this category is the least, and it is difficult for the model to classify it.

[0252] 6. Parameter sensitivity experiment

[0253] To explore the hyperparameter N c The influence of N is verified by three different data partitions in StanceUS. c Classification performance of the model from 10000 to 50000, with validation every 5000. Similar to the StanceIN dataset, validation N c The classification performance of the model from 60,000 to 160,000 is verified every 10,000. The experimental results are as follows Figure 4 The experimental results show that the overall performance of both datasets increases first and then decreases. For example, for StanceUS-500, when N c When N is 4500, Mac-F1 reaches its maximum value. This phenomenon shows that within a certain range, as N c The growth of is beneficial to the model for position classification. c When it grows to a certain value, noise will be introduced, resulting in a decrease in the classification performance of the model.

[0254] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A semi-supervised stance detection method based on category-aware curriculum learning, characterized in that include: S1: Get the tweet to be predicted; S2: Input the tweet to be predicted into the trained stance detection model and output the predicted stance label of the corresponding tweet; The stance detection model is built based on a bidirectional long short-term memory network. The stance detection model is trained through the following steps: S201: Obtain a set of labeled tweets and a set of unlabeled tweets as training data; S202: pre-training a stance detection model using a set of labeled tweets to obtain a pre-trained stance detection model; S203: Input the unlabeled tweet set into the pre-trained stance detection model, output the predicted stance label of each unlabeled tweet as its pseudo label, and then generate a set of unlabeled tweets with pseudo labels; S204: performing course learning by selecting tweets according to the label category ratio, and then selecting corresponding unlabeled tweets from the unlabeled tweet set with pseudo labels to form a pseudo-labeled tweet set; In step S204, a pseudo label of the unlabeled tweet is generated by the following formula: Where: Pseudo labels representing unlabeled tweets; s i represents the input unlabeled tweets; F represents the pre-trained stance detection model; θ represents the learnable parameters of the stance detection model; In step S204, based on the course learning framework, the corresponding unlabeled tweets are selected from the unlabeled tweet set with pseudo labels according to the category ratio of the labeled tweet set, and then the pseudo-labeled tweet set is generated by combining the pseudo-labels of the selected unlabeled tweets; Generate a collection of pseudo-labeled tweets by following the steps below: S2041: Calculate the entropy value of each unlabeled tweet in the set of unlabeled tweets with pseudo labels; The formula is described as: Where: H i Represents unlabeled tweets i The entropy value of Represents unlabeled tweets i Pseudo-labels; |C| represents the number of unlabeled tweet label categories; S2042: for each label category in the set of unlabeled tweets with pseudo labels, sort the unlabeled tweets belonging to the label category in ascending order of entropy value; S2043: Calculate the weight of each label category in the set of labeled tweets; The formula is described as: Where: Indicates the jth label category c j The weight of the proportion; i Indicates tweets with labels i The true position label category; D l Represents the number of tweets in the set of labeled tweets; S2044: Calculate the number of selected unlabeled tweets of each label category according to the weight of each label category; The formula is described as: Where: Indicates the jth label category c j The number of unlabeled tweets to be selected; q represents the total number of unlabeled tweets to be selected; S2045: For each label category, select the first unlabeled tweets with pseudo labels sorted by entropy value. pseudo-labeled tweets; The formula is described as: Where: Represents the selection of unlabeled tweets from the set of pseudo-labeled tweets The jth label category c j The set of unlabeled tweets; Indicates that the label category in the set of unlabeled tweets with pseudo labels is c j The collection of unlabeled tweets; S2046: merge all selected unlabeled tweets and re-sort them in order of entropy value from small to large to generate a set of pseudo-labeled tweets; S205: Optimizing the pre-trained stance detection model through the pseudo-labeled tweet set until the model converges to obtain a trained stance detection model; In step S205, the optimization training of the pre-trained stance detection model is implemented through the following steps: S2051: Get the text and hashtag of unlabeled tweets from the pseudo-labeled tweet collection; S2052: Extract text feature representation through bidirectional long short-term memory network to obtain text-based tweet representation; S2053: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags; S2054: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate a final tweet representation; S2055: Input the final tweet representation into a multi-layer perceptron, and output a pseudo-predicted stance label corresponding to the unlabeled tweet; S2056: Calculate the unlabeled loss based on the pseudo-predicted stance labels and pseudo-labels of unlabeled tweets, and optimize the model parameters; S2057: repeating steps S2051 to S2056 until the stance detection model converges; S3: The predicted stance label of the corresponding tweet is used as its stance detection result.

2. The semi-supervised stance detection method based on category-aware curriculum learning as claimed in claim 1, characterized in that: In step S202, the pre-training of the stance detection model is implemented through the following steps: S2021: Get the text and hashtag of the labeled tweets from the collection of labeled tweets; S2022: Extract text feature representation through bidirectional long short-term memory network to obtain text-based tweet representation; S2023: Learn the feature representation of hashtags through the self-attention mechanism and generate tweet representation based on hashtags; S2024: concatenate and normalize the text-based tweet representation and the hashtag-based tweet representation to generate the final tweet representation; The formula is described as: Where: The final tweet indicates; represents a text-based representation of tweets; represents the tweet representation based on hashtag; (·;·) represents the concatenation operation; S2025: Input the final tweet representation into a multi-layer perceptron and output the predicted stance label of the corresponding tweet; The formula is described as: Where: Represents the predicted stance label; MLP represents multi-layer perceptron; S2026: Calculate the label loss based on the predicted stance label and the true stance label of the tweet, and optimize the model parameters; S2027: Repeat steps S2021 to S2026 until the stance detection model converges.

3. The semi-supervised stance detection method based on category-aware curriculum learning as claimed in claim 2, characterized in that: In step S2022, a text-based tweet representation is generated by the following steps: 1) Represent the text of the tweet as a sequence of words {w1,w2,…,w n }; 2) Map the text into d through the word encoding layer w dimensional vector sequence 3) Use the bidirectional long short-term memory network layer according to the vector sequence H w Extract the intermediate sequence representation H of the text bls ; The formula is described as: H bls =BiLSTM(H w ); 4) The intermediate sequence is represented by H bls Perform maximum pooling to obtain text-based tweet representation 4. The semi-supervised stance detection method based on category-aware curriculum learning as claimed in claim 2, characterized in that: In step S2023, a hashtag-based tweet representation is generated through the following steps: 1) The hashtag of the tweet is represented as {#t1,#t2,…,#t m }; 2) Use the tag encoding layer to encode each Hashtag#t i Mapping is performed to obtain d t Dense vector sequence of dimension 3) Use the self-attention mechanism to determine the dense vector sequence H t The contribution of each vector in the prediction task is used to generate the vector representation of the Hashtag; The formula is described as: First, the Query, Key, and Value in attention are calculated using the following formula: Where: W q ,W k ,W v ,b q ,b k ,b v is a learnable parameter, relu is an activation function; Then, the attention weight is calculated by the following formula: Where: represents the query vector; Finally, the vector representation of Hashtag is calculated by the following formula: Where: Represents the vector representation of Hashtag; 4) Vector representation of hashtag Perform maximum pooling to obtain hashtag-based tweet representation 5. The semi-supervised stance detection method based on category-aware curriculum learning as claimed in claim 2, characterized in that: In step S2026, the label loss is calculated by the following formula: Where: Indicates that there is label loss; represents the predicted stance label; Indicates the true position label; β i is the loss discount weight of the tweets of the jth label category; ε is a minimum value; Indicates the jth label category c j The proportion weight of .

6. The semi-supervised stance detection method based on category-aware curriculum learning as claimed in claim 5, characterized in that: In step S2056, the unlabeled loss is calculated by the following formula: Where: represents the unlabeled loss; represents the pseudo-predicted stance label; represents a pseudo label; Indicates the jth label category c j The proportion weight of .

Images