A zero-shot stance detection device and method based on multi-level knowledge selection

Abstract

This invention discloses a zero-shot stance detection device and method based on multi-level knowledge selection, including a multi-level knowledge selection module and a zero-shot stance detection module. The multi-level knowledge selection module contains a knowledge selection model, which consists of an input layer, an encoding layer, a context layer, an attention layer, and an output layer. The knowledge selection model is used to match deep and shallow semantic information to obtain the deep knowledge required for zero-shot stance detection by extracting the interaction relationships between sentences and the interaction relationships between words in a single sentence. Knowledge, objectives, and arguments are input into the zero-shot stance detection module to complete the zero-shot stance detection, and the obtained deep knowledge is used as auxiliary information for zero-shot stance detection.

Description

Technical Field

[0001] This invention relates to the field of semantic matching and zero-shot learning for stance detection in the field of text opinion mining, and in particular to a zero-shot stance detection method based on multi-level knowledge selection. Background Technology

[0002] With the rapid development of the Internet, users are increasingly engaging in discussions about public events or opinions. A textual stance is the attitude that the author expresses towards a specific goal or when dealing with a problem. Goals mainly include people, organizations, and policies. Stance detection refers to judging a textual stance towards a pre-given goal, which can be a single goal, multiple goals, or cross-goals.

[0003] From a research methodology perspective, the methods can be mainly divided into those based on traditional machine learning, those based on topic models, and those based on deep learning.

[0004] In traditional machine learning methods, logistic regression, Naive Bayes, decision tree, and SVM are the most commonly used machine learning methods. The key is how to select appropriate feature representations. Xu et al. [1] used semantic features such as para2vec, LDA, and LSA to represent semantic information in Twitter text. They analyzed the effect of stance detection when using different semantic features. Sun et al. [2] analyzed the role of various language features such as semantic features, lexical features, morphological features, and syntactic features in the Chinese microblog stance detection task. Experiments showed that the above features all help to improve the performance of stance detection. Mohammad et al. [3] released a labeled dataset that integrates sentiment information and stance labels to study the mutual influence between sentiment and stance. Based on this dataset, Ebrahimi et al. [4] incorporated sentiment polarity into the target and stance, and modeled the interaction between the target, stance labels, and sentiment words in the text through a graph model, and established a log-linear sentiment-target-stance joint model.

[0005] In the topic model-based approach, Ahmed et al. [5] proposed a multi-perspective latent Dirichlet assignment model, which treats each word in the text as a combination of stance and topic, and uses factor topic modeling to solve the problem of modeling stance at the topic level, and performs posterior inference through folded Gibbs sampling. Gpttipati et al. [6] proposed a latent topic model, which integrates information extraction and sentiment analysis into the model, and uses the hierarchical structure of arguments in the debate encyclopedia dataset to construct topics to infer topics and stances in the field of socio-political debate. Finally, the advantages and disadvantages of the model are shown through qualitative and quantitative evaluation.

[0006] Among the deep learning-based methods, Sun et al. [7] proposed using a shared LSTM to learn a deep shared representation between stance and sentiment information, and to achieve mutual semantic representation through parameter sharing; then, the shared representation information is superimposed, and the hidden layer output of sentiment detection is used as an additional input for stance detection for joint learning, using sentiment information to promote the effect of stance detection. Sun et al. [8] pointed out that the importance of different linguistic information should be weighed, and proposed a multi-layer attention network model, which includes two parts: a linguistic attention part and a super attention part. The former is used to learn representations with different linguistic features (including sentiment words, dependencies and targets), and the latter adjusts the weights of different feature sets. Allway et al. [9] applied the Sem16 dataset, which is used for target internal stance detection, to zero-shot stance detection, and used adversarial learning to extract sample-independent transferable features to complete stance detection. In addition, Liu et al.

[10] introduced external knowledge from the structural and semantic levels and proposed a common sense augmented graph model based on BERT to solve the zero-shot stance detection task. Hardalov et al.

[11] proposed a few-shot cross-linguistic stance detection model based on sentiment, conducted experiments on 15 different datasets from 12 languages ​​in 6 language families, and proposed a few-shot pre-trained model based on pattern development training (PET).

[0007] [References]

[0008] [1]Xu J, Zheng S, Shi J, et al.Ensemble of feature sets and classification methods for stance detection[C] / / Natural LanguageUnderstanding and Intelligent Applications:5th CCF Conference on NaturalLanguage Processing and Chinese Computing,NLPCC 2016,and 24th InternationalConference on Computer Processing of Oriental Languages,ICCPOL 2016,Kunming,China,December 2–6, 2016, Proceedings 24. Springer International Publishing, 2016: 679-688.

[0009] [2]Sun Q,Wang Z,Zhu Q,et al.Exploring various linguistic features forstance detection[C] / / Natural Language Understanding and IntelligentApplications:5th CCF Conference on Natural Language Processing and ChineseComputing,NLPCC 2016,and 24th International Conference on Computer Processingof Oriental Languages,ICCPOL 2016,Kunming,China,December 2–6,2016,Proceedings24.Springer International Publishing,2016:840-847.

[0010] [3]Mohammad S M,Sobhani P,Kiritchenko S.Stance and sentiment intweets[J].ACM Transactions on Internet Technology(TOIT),2017,17(3):1-23

[0011] [4]Ebrahimi J,Dou D,Lowd D.A joint sentiment-target-stance model forstance classification in tweets[C] / / Proceedings of COLING 2016,the 26thinternational conference on computational linguistics:Technical papers.2016:2656-2665.

[0012] [5]Ahmed A,Xing E.Staying informed:supervised and semi-supervisedmulti-view topical analysis of ideological perspective[C] / / Proceedings of the2010Conference on Empirical Methods in Natural Language Processing.2010:1140-1150.

[0013] [6]Gottopati S,Qiu M,Sim Y,et al.Learning topics and positions fromdebatepedia[C].ACL,2013.

[0014] [7]Sun Q,Wang Z,Li S,et al.Stance detection via sentiment informationand neural network model[J].Frontiers of Computer Science,2019,13:127-138.

[0015] [8]Sun Q,Wang Z,Zhu Q,et al.Stance detection with hierarchicalattention network[C] / / Proceedings of the 27th international conference oncomputational linguistics.2018:2399-2409.

[0016] [9]Allaway E,Srikanth M,McKeown K.Adversarial learning for zero-shotstance detection on social media[J].arXiv preprint arXiv:2105.06603,2021.

[0017]

[10] Liu R, Lin Z, Tan Y, et al.Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph[C] / / Findings of the Association for Computational Linguistics:ACL-IJCNLP 2021.2021:3152-3157.

[0018]

[11] Hardalov M, Arora A, Nakov P, et al. Few-shot cross-lingual stance detection with sentiment-based pre-training[C] / / Proceedings of the AAAIConference on Artificial Intelligence.2022,36(10):10729-10737. Summary of the Invention

[0019] The purpose of this invention is to overcome the shortcomings of the prior art and provide a zero-sample position detection device and method based on multi-level knowledge selection to solve the position detection of invisible targets.

[0020] The objective of this invention is achieved through the following technical solution:

[0021] A zero-shot stance detection device based on multi-level knowledge selection includes a multi-level knowledge selection module and a zero-shot stance detection module;

[0022] The multi-layered knowledge selection module contains a knowledge selection model, which consists of an input layer, an encoding layer, a context layer, an attention layer, and an output layer. The knowledge selection model is used to match deep and shallow semantic information to obtain the deep knowledge required for zero-shot stance detection by extracting the interaction relationships between sentences and the interaction relationships between words in a single sentence.

[0023] Specifically, based on the target, articles from Wikipedia are selected, and sentences semantically related to the argument are chosen from these articles. A knowledge selection model learns deep and shallow semantic representations of the argument sentence P and the sentence Q to be selected from Wikipedia to perform sentence selection. In the encoding layer, BERT model A is used for encoding. BERT model A employs a multi-head attention mechanism, and the [CLS] marker in BERT model A can represent the semantic relationships of the input sequence X. In the context layer, a bidirectional LSTM is used for context-level representation learning, and a self-attention mechanism is used to obtain information about key words in the context. The semantic information of the [CLS] marker and the semantic information with key word information extracted by the bidirectional LSTM and attention mechanism are concatenated to jointly complete the selection of deep knowledge from both deep and shallow semantic perspectives.

[0024] Knowledge, objectives, and arguments are input into the zero-shot stance detection module to complete zero-shot stance detection. Acquired deep knowledge is used as auxiliary information for zero-shot stance detection. The class labels for zero-shot stance detection are support, opposition, and neutral. During the training phase, zero-shot stance detection maps stance features to a feature space for each class label, meaning each visible class label can be represented by a vector. A mapping function is learned to measure stance. During the testing phase, given a vector requiring category prediction, the mapping function finds the best-matching category in the category description feature vector library, thus completing the stance detection and determining the stance.

[0025] Furthermore, this includes the following steps:

[0026] In the input layer, the argument sentence P and the sentence Q to be selected from Wikipedia are separated using the [SEP] marker, and the [CLS] marker is inserted before the input argument sentence P. The output vector corresponding to the [CLS] marker is regarded as the semantic representation of the entire input sequence X.

[0027] The input vector corresponding to the [CLS] marker is processed by the BERT model A in the encoding layer to obtain output vectors at different levels. The final embedding vector is obtained by adding the different output vectors. The embedding representation of the input sequence X is then input into the BERT model A for encoding to obtain the context encoding vector representation.

[0028] In the context layer, a bidirectional LSTM is used to fuse contextual information into each vector of the argument sentence P and the sentence Q to be selected from Wikipedia, so as to obtain the corresponding context vector;

[0029] The self-attention layer captures key information from the context layer by analyzing the semantic information of the argument sentence P and the Wikipedia sentence Q to be selected.

[0030] The output layer concatenates the argument sentence P obtained from the self-attention layer, the sentence vector of the Wikipedia sentence to be selected Q, and the [CLS] semantic vector obtained through the BERT model A to obtain the concatenated vector A. Finally, vector A is input into the linear layer, and the output result is normalized by softmax.

[0031] The knowledge, objectives, and arguments are input into the zero-shot position detection module to complete the zero-shot position detection, and the acquired knowledge is used as auxiliary information for zero-shot position detection.

[0032] Furthermore, in the input layer, the argument sentence P = {p1, ... p} is... m} and each sentence to be selected in Wikipedia W is Q = {q1, ..., q} n}, its p i Let q be the i-th word in the argument sentence P. i For the i-th word of the sentence Q to be selected, the argument sentence P and the sentence Q to be selected from Wikipedia are separated using the [SEP] marker. The argument sentence P in BERT model A is then prepended with the [CLS] marker. The output vector corresponding to the [CLS] marker is considered as the common semantic representation of P and Q. The input format of BERT model A is as follows:

[0033] X = {[CLS], p1, ..., p} m [SEP], q1, ..., q n [SEP]};

[0034] In the encoding layer, different levels of output vectors are obtained through Token Embedding, Segment Embedding, and Position Embedding of BERT model A. The final output vector is obtained by adding the different output vectors. Then, the embedding representation of the input sequence X is input into BERT model A for encoding to obtain the context encoding vector representation h of the input sequence X.

[0035] h = {h0, h1, ..., h} l-1} = BERT(X)

[0036] Where h i Let h0 be the context representation of the i-th character in the input sequence X, and h0 be the vector representation of the [CLS] marker symbol;

[0037] In the context layer, two independent bidirectional LSTMs are used to compute the context vectors of P and Q, respectively. and

[0038] The self-attention layer uses a self-attention mechanism to extract important information from P and Q respectively;

[0039] The output layer consists of three parts. First, it processes the sentence vectors mp and m obtained through the self-attention mechanism P and Q. q Concatenate with the h0 vector of [CLS] semantics obtained through BERT model A

[0040] A = [h0; m p ;m q ]

[0041] Then vector A is input into the linear layer, and the output is normalized by softmax.

[0042] r = softmax(WA + b)

[0043] Where W represents the weight of the linear layer and b represents the bias of the linear layer, Wikipedia sentences with a relevance greater than 0.5 are retained.

[0044] Furthermore, in the zero-sample position detection process, the claim, target, and knowledge are combined and recorded as a single sample x. i The pre-trained BERT model B is used as the encoder, and the input format is redesigned to "[CLS]target:claim[SEP]knowledge[SEP]". For each sample x i After being input into the BERT model B, a hidden layer H with dimension D is obtained;

[0045] H=BERT([CLS]target:claim[SEP]knowledge[SEP])

[0046] Finally, the stance detection labels from BERT model B are input into a fully connected network, and the stance labels are predicted using softmax. The hidden variable layer is then input into a classifier with a softmax function to predict the stance distribution.

[0047] p i =softmax(WH i +b)

[0048] in This refers to the input sample x i The probability distribution of the position, d p W represents the dimension of the position detection label, and W and b are hyperparameters used in training.

[0049] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0050] This invention differs from current traditional stance detection methods, which often focus on single-target, multi-target, and cross-target stance detection tasks. However, such tasks require large labeled datasets, which is impractical in reality. The zero-shot stance detection method presented in this invention effectively addresses these issues. The zero-shot learning mechanism simulates human problem-solving; when encountering an unseen new target, it connects the unseen target with relevant knowledge learned from known targets. This knowledge is considered a bridge between known and unseen targets, enhancing the model's reasoning ability in stance detection and ultimately enabling it to detect stances of unseen targets.

[0051] To address the issues of high target dependence and insufficient knowledge selection in stance detection, this invention designs a zero-shot stance detection model based on multi-level knowledge selection. This model utilizes BERT, BiLSTM, and Self-Attention mechanisms to construct a semantic matching module from multiple perspectives of deep and shallow semantics, performing multi-level semantic modeling of knowledge to solve the problems of knowledge sparsity and noise. Considering the flexible expression and diverse topics of internet text, it is impossible to rely entirely on labeled datasets for stance detection. Therefore, a knowledge-enhanced zero-shot stance detection model is constructed. The purpose of zero-shot stance detection is to predict stances for unknown target data, and the stance expression in the text is closely related to the target topic of discussion. For stance detection of unknown targets, it is crucial to use additional knowledge to capture transferable stance features between known and unknown targets. To better extend the target-related stance features learned from known targets to unknown targets, this invention uses a multi-level knowledge selection module to match the deep and shallow multi-level semantic information of the arguments and the sentences to be selected in Wikipedia articles to capture the deep knowledge required for zero-shot stance detection. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall framework of the zero-sample position detection device.

[0053] Figure 2 This is a schematic diagram of the knowledge selection module.

[0054] Figure 3 This is a schematic diagram of the zero-sample stance detection module. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0056] This embodiment proposes a zero-sample stance detection device based on multi-level knowledge selection, including a multi-level knowledge selection module and a zero-sample stance detection module;

[0057] The multi-layered knowledge selection module contains a knowledge selection model, which consists of an input layer, an encoding layer, a context layer, an attention layer, and an output layer. The knowledge selection model is used to match deep and shallow semantic information to obtain the deep knowledge required for zero-shot stance detection by extracting the interaction relationships between sentences and the interaction relationships between words in a single sentence.

[0058] The zero-shot stance detection module is used to perform zero-shot stance detection on the input knowledge, objectives, and arguments, and uses the acquired deep knowledge as auxiliary information for zero-shot stance detection.

[0059] Specifically, the zero-shot stance detection method based on multi-level knowledge selection includes two stages: multi-level knowledge selection and zero-shot stance detection.

[0060] 1. First Stage: Multi-level Knowledge Selection

[0061] 1.1 Input Layer

[0062] For the BERT model, the input consists of two sentences. In this embodiment, the argument sentence P = {p1, ... p2} is used. m The sentence to be selected in} and Wikipedia w is Q={q1,...,q n} is separated using the [SEP] notation, where pi is the i-th word in the argument sentence P, and q i For the i-th word of the sentence Q to be selected, a [CLS] marker is inserted before the BERT model. The output vector corresponding to this marker can be regarded as the semantic representation of P and Q. Therefore, the input format of the model is as follows:

[0063] X = {[CLS], p1, ..., p} m [SEP], q1, ..., q n [SEP]

[0064] 1.2 Coding Layer

[0065] In the encoding layer, different levels of output vectors are obtained through Token Embedding, Segment Embedding, and Position Embedding of the BERT model. The final output vector is obtained by adding the different output vectors. Then, the embedding representation of the input sequence X is input into the BERT model for encoding to obtain the context encoding vector representation h of the input sequence X.

[0066] h = {h0, h1, ..., h} l-1} = BERT(X)

[0067] Where h i Let h0 be the context representation of the i-th character in the input sequence X, and h0 be the vector representation of the [CLS] marker symbol;

[0068] 1.3 Context Layer

[0069] To obtain deep semantics from the context representation of a single sentence, a bidirectional LSTM is used to fuse contextual information into each vector of the argument sentence P and the Wikipedia sentence Q to obtain the corresponding context vector. Compared with a unidirectional LSTM, a bidirectional LSTM has the advantage of capturing more semantic information. Therefore, this paper uses two independent bidirectional LSTMs to compute the context vectors of the argument sentence P and the Wikipedia sentence Q, respectively. Taking the argument sentence P as an example, the context vector obtained for each word vector is calculated as follows:

[0070]

[0071]

[0072] By concatenating the two, we obtain the final context vector.

[0073]

[0074] in Let P be the context vector of the i-th step of the argument sentence. Similarly, we can obtain the context vector of the i-th step of the Wikipedia sentence Q to be selected.

[0075] 1.4 Self-Attention Layer

[0076] The self-attention layer captures key information from the context of the sentence's semantics, thereby improving the model's ability to extract semantic features. After bidirectional LSTM fusion, a self-attention mechanism is used to extract key information from the argument sentence P and the candidate Wikipedia sentence Q, respectively. This layer is based on context vectors. and Where 1 represents the maximum input length of the bidirectional LSTM. First, hh p and h q The input is fed into a linear layer, and self-attention is calculated using the following formula.

[0077] m p = self-attention(h p )

[0078] m q = self-attention(h q )

[0079] Self-attention is a special type of attention that performs attention calculations within the same sentence. The resulting query searches for key-value pairs at various positions within the same sentence. Therefore, the query and key-value pairs for self-attention are generated within a single sentence.

[0080] 1.5 Output Layer

[0081] This section consists of three parts. First, the sentence vector obtained after the self-attention layer and the semantic vector h0 of the [CLS] marker obtained through the BERT model are concatenated.

[0082] A = [h0; m p ;m q ]

[0083] Then, vector A is input into the linear layer, and the output is normalized by softmax.

[0084] r = softmax(WA + b)

[0085] Where W represents the weight of the linear layer, b represents the linear layer bias, and r represents the text matching result. Wikipedia sentences with a relevance greater than 0.5 are retained as auxiliary knowledge information for the second-stage zero-sample stance detection.

[0086] 2. Second Phase: Zero-Sample Stance Testing

[0087] 2.1 Coding Layer

[0088] Given an argument claim, a target, and knowledge based on each target, this embodiment uses the pre-trained BERT model as the encoder. Considering that BERT was originally designed to process at most two sequences, and that this task requires encoding Wikipedia knowledge in addition to the argument and target, the argument claim and target are merged into a single sequence. The input format is redesigned as "[CLS]target:claim[SEP]knowledge[SEP]", as shown in the figure. This input format allows the target, claim, and knowledge to interact during the encoding process. The final layer's [CLS] symbol embeddings are merged and the output is used as the final representation of the input. Therefore, the target and textual knowledge are concatenated. For each sample X, the input to the BERT model yields a D-dimensional hidden layer H.

[0089] H=BERT([CLS]target:claim[SEP]knowledge[SEP])

[0090] 2.2 Position Detection Layer

[0091] Finally, the stance detection labels from BERT are input into a fully connected network, and the labels are predicted using sofftmax. The hidden variable layer is then fed into a classifier with a softmax function to predict the stance distribution.

[0092] p i =softmax(WH i +b)

[0093] in This refers to the input sample x i The probability distribution of the position, d p W represents the dimension of the position label, while W and b are hyperparameters used in training.

[0094] Specifically, experiments were conducted to verify the effectiveness of the present invention based on the above-described technical solution and other models. This mainly includes two parts: first, verifying the effectiveness of the multi-level knowledge selection model; and second, verifying the effect of the knowledge-enhanced zero-shot stance detection model. Therefore, the experimental results are explained separately for each part. This embodiment verifies multi-level knowledge selection on the WikiQA and SNLI datasets, and verifies zero-shot stance detection on the VAST, SEM16, and WT-WT datasets.

[0095] In terms of evaluation metrics, the evaluation metric in SNLI is Acc (accuracy), as shown in Table 1;

[0096] In WikiQA, the evaluation metrics are MAP (mean average precision) and MRR (mean reciprocal ranking), as shown in Table 2;

[0097] The evaluation metric on the VAST, SEM16, and WT-WT datasets is F1 (accuracy), as shown in Table 3.

[0098] Table 1. Acc performance metrics based on the SNLI dataset.

[0099] Model Acc% BIMPM 86.9 eSIM 88.0 MwAN 88.3 DIIN 88.4 RE2 88.7 DEIM 88.9 This invention 89.4

[0100] Table 2. MAP and MRR performance metrics based on the WikiQA dataset.

[0101] Model MAP% MRR% ABCNN 69.2 71.1 BiMPM 71.8 73.1 HCRN 74.3 75.6 RE2 74.5 76.2 DEIM 75.5 77.5 This invention 76.2 78.4

[0102] Table 3. F1 performance metrics based on VAST, SEM16, and WT-WT

[0103]

[0104] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A zero-shot stance detection device based on multi-level knowledge selection, characterized by, The multi-level knowledge selection module and the zero-shot stance detection module are included. The knowledge selection model is arranged in the multi-level knowledge selection module, and the knowledge selection model is sequentially composed of an input layer, an encoding layer, a context layer, an attention layer and an output layer; the knowledge selection model is used to match deep and shallow semantic information, so as to obtain deep knowledge required by zero-shot stance detection by extracting the interaction relationship between sentences and the interaction relationship between words in a single sentence; Specifically, according to the target, an article of Wikipedia is selected, and a sentence related to the semantic of the argument is selected from the article, the semantic representation of the argument sentence P and the selected sentence Q in the Wikipedia is learned by the knowledge selection model to select the sentence, the BERT model A is used for encoding in the encoding layer, the BERT model A adopts a multi-head attention mechanism, and the [CLS] symbol in the BERT model A can represent the semantic relationship of the input sequence X; the bidirectional LSTM is used for context-level representation learning in the context layer, and the information of key words in the context is obtained through the self-attention mechanism; the semantic information of the [CLS] symbol and the semantic information of the key words extracted by the bidirectional LSTM and the attention mechanism are spliced, and the selection of the deep knowledge is completed from the deep and shallow semantic angles; The knowledge, the target and the argument are input into the zero-shot stance detection module to complete the zero-shot stance detection, and the obtained deep knowledge is used as auxiliary information for the zero-shot stance detection; The class label of the zero-shot stance detection is support, opposition and neutrality, the stance feature is mapped into a feature space for each class label in the training stage of the zero-shot stance detection, that is, each visible class label can be represented by a vector, a mapping function is learned, and the mapping function can be used to measure the stance; in the test stage, for a given vector requiring class prediction, the most matching class in the class description feature vector library is found through the mapping function, and then the stance detection is completed to judge the stance.

2. A zero-shot stance detection method based on multi-level knowledge selection, based on the zero-shot stance detection device of claim 1, characterized in that, The following steps are included: The argument sentence P and the selected sentence Q in the Wikipedia are segmented by using the [SEP] symbol in the input layer, and the [CLS] symbol is inserted before the input argument sentence P, and the output vector corresponding to the [CLS] symbol is regarded as the semantic representation of the entire input sequence X; The input vector corresponding to the [CLS] symbol is obtained by the BERT model A in the encoding layer to obtain different levels of output vectors, and the final embedding vector is obtained by adding different output vectors; then the embedding representation of the input sequence X is input into the BERT model A for encoding to obtain the context encoding vector representation; The context information is fused into each vector of the argument sentence P and the selected sentence Q in the Wikipedia by the bidirectional LSTM in the context layer to obtain the corresponding context vector; The semantic information of the argument sentence P and the selected sentence Q in the Wikipedia is captured in the self-attention layer, and the key information in the context layer is captured; The output layer splices the argument sentence P obtained through the self-attention layer, the sentence vector of the Wikipedia to-be-selected sentence Q, and the [CLS] semantic vector obtained through the BERT model A to obtain a spliced vector , and finally inputs the vector into a linear layer, and the output result is normalized through softmax. The knowledge, the target and the argument are input into the zero-shot stance detection module to complete the zero-shot stance detection, and the obtained knowledge is used as auxiliary information for the zero-shot stance detection.

3. The zero-shot stance detection method based on multi-level knowledge selection according to claim 2, characterized in that, In the input layer, the argument sentence P is and each sentence Q to be selected in Wikipedia W wherein is the i-th word in the argument sentence P, i = 1, 2, 3…m; is the i-th word in the sentence Q to be selected, i = 1, 2, 3…n; the argument sentence P and the sentence Q to be selected in Wikipedia are separated by the [SEP] marker, and the [CLS] marker is inserted before the argument sentence P of the BERT model A, and the output vector corresponding to the [CLS] marker is regarded as the semantic representation common to P and Q. The input format of the BERT model A is as follows: ； In the encoding layer, different levels of output vectors are obtained through Token Embedding, Segment Embedding and Position Embedding of the BERT model A respectively, and the final output vector is obtained by adding different output vectors; then the embedding representation of the input sequence X is input into the BERT model A for encoding to obtain the context encoding vector representation of the input sequence X h ; ； wherein is a context representation for the i-th character in the input sequence X, i = 1, 2, 3...l; is a vector representation for the [CLS] token. In the context layer, two independent bidirectional LSTMs are used to compute the context vectors for P and Q, respectively and ; The self-attention mechanism is used in the self-attention layer to extract important information of P and Q respectively; The output layer consists of three parts. First, it takes the sentence vectors m obtained through the self-attention mechanism (P and Q) and... p m q And the [CLS] semantics obtained through BERT model A Concatenate vectors ； The vector is then input to a linear layer, and the output is normalized by the softmax criterion. ； wherein denotes the weights of the linear layer, denotes the linear layer bias, and the Wikipedia sentences to be selected are kept if the correlation is above 0.

5.

4. The zero-shot stance detection method based on multi-level knowledge selection according to claim 2, wherein, In the zero-shot stance detection process, the argument claim and the target and knowledge are combined as an example , a pre-trained BERT model B is used as an encoder, and the input format is redesigned as "[CLS]target:claim[SEP]knowledge[SEP]" , after inputting each example into the BERT model B, a hidden layer of dimension D is obtained H ; ； Finally, the stance detection label from the BERT model B is input into a fully connected network, and a stance label is predicted through softmax. The hidden variable layer is input into a classifier with a softmax function to predict the stance distribution, wherein ； where refers to the input example the probability distribution of the stance, is the dimension of the stance detection label, and W and b are hyperparameters in training.

Images