Zero-shot stance detection method and system based on multi-expert collaborative learning
By employing a multi-expert collaborative learning method based on the BERT model and utilizing a gating mechanism to filter and fuse low-level semantic features of the text, the problem of insufficient accuracy in zero-shot stance detection is solved, achieving more efficient unknown target stance detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
- Filing Date
- 2024-03-08
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies fail to fully utilize the multifaceted underlying features of textual semantics in zero-shot position detection, resulting in insufficient classification accuracy for unknown targets.
We employ a multi-expert collaborative learning method based on the BERT model, which captures different aspects of text semantics through different experts and uses a gating mechanism for feature selection and fusion to enhance the ability to detect stance.
It improves the accuracy of zero-sample position classification for unknown targets and enhances the model's generalization ability across different feature spaces.
Smart Images
Figure CN118211151B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a zero-shot position detection method and system based on multi-expert collaborative learning. Background Technology
[0002] Stance detection aims to uncover users' stances or attitudes toward specific goals or topics (e.g., events, products, policies, people, or services) (i.e., support, opposition, or neutrality). Traditional stance detection training and test sets belong to the same goal, but obtaining datasets for all goals is difficult to achieve, and obtaining effective labels for a new, unseen goal is often costly. Therefore, zero-shot stance detection, which can quickly adapt to new, unseen goals, has attracted much attention.
[0003] Zero-shot stance detection enhances a model's learning and generalization capabilities by learning and transferring features across different targets, enabling the model to generalize well across various feature spaces. New issues and perspectives constantly emerge in the real world, and retraining a model for each new issue is neither practical nor economical; therefore, zero-shot stance detection presents a new challenge. Furthermore, it faces challenges in handling the diversity and complexity of language, such as the differences in opinion expression across different cultural and linguistic backgrounds. Zero-shot stance detection primarily learns deep semantic features of text, aiming to use these transferable and invariant features to predict the stance of unseen targets.
[0004] The key to zero-shot stance detection is learning transferable features from known labeled samples and generalizing them to unseen target data, enabling stance detection on a constantly evolving array of new target data. Existing research has attempted to learn stance representations from existing target data and transfer them to achieve zero-shot stance detection in unseen target domains through attention mechanisms, adversarial learning, sentiment or common-sense feature enhancement, and statistical pragmatic feature enhancement. However, these methods primarily attempt feature transfer between different target domains from the global feature space of text semantics, without fully considering the more fundamental and multifaceted features that constitute semantics. In terms of text semantics, a sentence is a complex concept containing various elements, and its semantics is usually considered to be composed of multiple aspects such as vocabulary and grammar. Therefore, text can be represented by various features, which are more fundamental and transferable.
[0005] Therefore, how to learn the underlying multifaceted features of text semantic expression and improve the accuracy of zero-sample position classification of unknown targets is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] In view of this, the present invention provides a zero-shot stance detection method and system based on multi-expert collaborative learning that can at least solve some of the above-mentioned technical problems. The method constructs a multi-expert collaborative semantic decoupling learning model based on the BERT model. Different experts focus on different aspects of the underlying semantic features of the text, and the decoupled features are filtered and aggregated through gating units, which enhances the ability to capture deep features of different aspects of language, obtains more effective features, and improves the accuracy of zero-shot stance classification for unknown targets.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0008] In a first aspect, embodiments of the present invention provide a zero-shot stance detection method based on multi-expert collaborative learning, the method comprising the following steps:
[0009] The method includes the following steps:
[0010] S1. Construct a multi-expert collaborative semantic decoupling learning model based on the BERT model. The outputs of different experts use different BERT output layers as text encoders. Different experts obtain different aspects of the underlying semantic features of the text input.
[0011] S2. The underlying semantic features of the text obtained by the multi-expert collaborative semantic decoupling learning model are concatenated and input into the feature filtering and fusion module based on the gating mechanism. The gating mechanism is used to filter and fuse the underlying semantic features of the text from multiple aspects to obtain a text underlying semantic feature representation that is effective for the position.
[0012] S3. Input the same batch of text into the semantic feature learning module and obtain high-level semantic features of the text through the BERT encoder;
[0013] S4. The obtained high-level semantic features of the text are concatenated with the low-level semantic features of the text that have been screened and fused through a gating mechanism, and then input into the stance classifier to obtain the text stance classification.
[0014] Further, in step S1, firstly, BERT is used as the encoder f = BERT(·) to encode the content of the target t, text x, and target-text pair (x,t). The position label is regarded as a relationship between the target and the text, and its input forms are as follows:
[0015] [CLS]t[SEP]
[0016] [CLS]x[SEP]
[0017] [CLS]t[SEP]x[SEP]
[0018] Where [CLS] indicates the start of the classification task, and [SEP] indicates the separation or end;
[0019] Secondly, different experts capture different grammatical features, and fine-grained mining yields target-irrelevant features that are easy to transfer; feature experts for different functions are:
[0020] H (k,l) =f (k,l) =BERT(·)[l]
[0021] in, The feature extracted by the k-th expert is taken from the l-th layer of the encoder output, d m is the dimension of the feature; f is the encoder;
[0022] The text feature matrix consisting of n words is obtained by average pooling:
[0023]
[0024] Among them, h (k,l) H represents the pooled feature. (k,l) [i] is the feature vector of the i-th word.
[0025] Further, in step S2, firstly, the output features of multiple experts are subjected to feature aggregation based on concatenation, which is used as the features to be screened. The expression is:
[0026]
[0027] in, This is the aggregation result of k expert features. This is for splicing operations.
[0028] Secondly, the gate function is defined as the neural network mapping g. φ (·), by learning the gating weights, it is used to control the selection of aggregated features; the encoding of the target t is used as the input of the gating function to select the features with the greatest correlation to the target stance judgment;
[0029] z = f([CLS]t[SEP])[CLS]
[0030] m = g φ (z)=W b z
[0031]
[0032] in, Let be the feature vector of target t. For learnable parameters, For the gated mask vector, These are the features after filtering.
[0033] Furthermore, in step S3, the highest-level feature encoded by the encoder represents the target-dependent feature, expressed as:
[0034] h = f([CLS]t[SEP]x[SEP]) [-1,CLS]
[0035] in, High-level semantic features that the text target depends on.
[0036] Furthermore, in step S4, the probability distribution of the predicted position is adopted using a fully connected layer with softmax normalization:
[0037]
[0038] in, For input sample x i Predicted position probability distribution, d p As a dimension of stance labeling, and These are learnable parameters;
[0039] Based on the predicted position probability distribution The classifier is trained using the cross-entropy loss with the true label y.
[0040]
[0041] in, For cross-entropy loss, N b This is a sample of the entire dataset.
[0042] Secondly, embodiments of the present invention also provide a zero-shot stance detection system based on multi-expert collaborative learning. The system applies the aforementioned zero-shot stance detection method based on multi-expert collaborative learning to perform stance detection. The system includes:
[0043] The multi-expert collaborative semantic decoupling learning module is used to build a multi-expert collaborative semantic decoupling learning model based on the BERT model. The outputs of different experts use different BERT output layers as text encoders, and different experts obtain different aspects of the underlying semantic feature representation of the text input.
[0044] The feature filtering and fusion module based on the gating mechanism is used to filter and fuse multiple aspects of the underlying semantic features of the text to obtain a text underlying semantic feature representation that is effective for the position.
[0045] The semantic feature learning module is used to obtain high-level semantic features of text through the BERT encoder;
[0046] A classifier is used to classify the text stance based on the semantic features of the concatenated text, thereby enabling the detection of the text stance.
[0047] Compared with the prior art, the present invention has at least the following beneficial effects:
[0048] This invention provides a zero-shot stance detection method and system based on multi-expert collaborative learning. It constructs a multi-expert collaborative semantic decoupling learning model based on the BERT model. By mining multiple low-level features that affect semantic expression through different experts, it learns multiple target-independent transferable features in a fine-grained manner. The decoupled features are filtered and aggregated through gating units, which enhances the ability to capture deep features of different aspects of language, obtains more effective features, and improves the accuracy of zero-shot stance classification for unknown targets.
[0049] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.
[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0053] Figure 1 This is a schematic diagram of a zero-sample stance detection method based on multi-expert collaborative learning, provided for an embodiment of the present invention.
[0054] Figure 2 This is a schematic diagram of a zero-sample stance detection system based on multi-expert collaborative learning, provided as an embodiment of the present invention. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.
[0056] In the description of this invention, it should be noted that some processes described in this application specification and drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may be performed in any order or in parallel. Furthermore, various numbers are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0057] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0058] See Figure 1 As shown, this invention provides a zero-shot stance detection method based on multi-expert collaborative learning, which mainly includes the following steps:
[0059] S1. Construct a multi-expert collaborative semantic decoupling learning model based on the BERT model. In the model, the outputs of different experts use different BERT output layers as text encoders, and different experts obtain different aspects of the underlying semantic features of the text input.
[0060] S2. The underlying semantic features of the text obtained by the multi-expert collaborative semantic decoupling learning model are concatenated and input into the feature filtering and fusion module based on the gating mechanism. The gating mechanism is used to filter and fuse the underlying semantic features of the text from multiple aspects to obtain the underlying semantic feature representation of the text that is effective for the position.
[0061] S3. Input the same batch of text into the semantic feature learning module and obtain high-level semantic features of the text through the BERT encoder;
[0062] S4. The obtained high-level semantic features of the text are concatenated with the low-level semantic features of the text that have been screened and fused through a gating mechanism, and then input into the stance classifier to obtain the text stance classification.
[0063] Combination Figures 1-2 As shown below, the relevant theories involved in the method of the present invention and the specific implementation methods of the present invention will be described in detail:
[0064] I. Multi-expert collaborative semantic decoupling learning
[0065] In this embodiment, this module focuses on learning target-independent features. To better understand semantic stance expressions, it is necessary not only to mine the global semantic features of the text but also to pay attention to the more fundamental and multifaceted grammatical features. This is because the underlying features constituting text semantics are more basic target-independent features, which are easier to transfer across targets. The pre-trained language model BERT can capture grammatical-level features at the lower level, while the higher level learns more abstract semantic features. Inspired by this, this invention constructs a multi-expert collaborative semantic decoupling learning model based on a pre-trained language model. Different experts capture different grammatical features, and fine-grained mining of target-independent features that are easy to transfer is achieved.
[0066] In one specific embodiment, BERT is first used as the encoder f = BERT(·) to encode the content of the target t, text x, and target-text pair (x,t). The position label is regarded as a relationship between the target and the text, and its input forms are: "[CLS]t[SEP]", "[CLS]x[SEP]", and "[CLS]t[SEP]x[SEP]", where [CLS] indicates the start of the classification task. This label is used to summarize the semantic representation. During training, BERT learns the overall semantic information of the text sequence by processing the output of this label. [SEP] indicates the separation or end. This label is used to separate different sentences or paragraphs.
[0067] Secondly, the difference between different experts lies in their expertise in capturing different syntactic features (i.e., using different layers of the encoder f's output); in this embodiment, feature experts with different functions are defined as follows:
[0068] H (k,l) =f (k,l) =BERT(·)[l]
[0069] in, The feature extracted by the k-th expert is taken from the l-th layer of the encoder output, d m The dimension of the feature;
[0070] Furthermore, the text feature matrix composed of n words is obtained by average pooling:
[0071]
[0072] Among them, h (k,l) H represents the pooled feature. (k,l) [i] is the feature vector of the i-th word.
[0073] II. Feature Filtering and Fusion Based on Gating Mechanism
[0074] The features obtained from different experts represent the different underlying grammatical features used in the semantic expression of the text. However, not all underlying grammatical features necessarily have a positive effect on stance judgment. Therefore, in order to further select features that are valuable to the target stance in a more targeted manner, this embodiment uses a gating mechanism to filter the obtained multi-expert features.
[0075] First, the output features from multiple experts are combined using a concatenation-based feature aggregation method, which serves as the features to be selected. The expression is as follows:
[0076]
[0077] in, For splicing operations, This is the aggregation result of k expert features.
[0078] Secondly, the gate function is defined as the neural network mapping g. φ (·), by learning the gating weights, the selection of aggregated features is controlled; at the same time, since the ultimate goal of feature learning is to predict the stance of text x on target t, this embodiment uses the encoding of target t as the input of the gating function to select the features most correlated with the target stance judgment.
[0079] z = f([CLS]t[SEP])[CLS]
[0080] m = g φ (z)=W b z
[0081]
[0082] in, Let be the feature vector of target t. For learnable parameters, For the gated mask vector, These are the features after filtering.
[0083] III. Semantic Feature Learning
[0084] Goal-independent features are easily transferable, while goal-dependent features also contribute to the final stance expression. The combined effect of goal-independent and goal-dependent features determines the expression of stance inclination. Since higher-level semantic features are more abstract and goal-dependent, this embodiment uses the highest-level features encoded by the encoder to represent goal-dependent features.
[0085] h = f([CLS]t[SEP]x[SEP]) [-1,CLS]
[0086] in, The sentence vector corresponding to the last layer CLS tag of the encoder is the global semantic feature (high-level semantic feature of the text) that the text target depends on.
[0087] IV. Classifier
[0088] To more comprehensively utilize the multifaceted characteristics of text, this embodiment fuses target-dependent high-dimensional semantic features with target-independent grammatical features learned by multiple experts, enabling the classifier to better adapt to zero-shot stance detection tasks. Specifically, this embodiment integrates the selected multifaceted features. The feature h is concatenated with the target-dependent feature. Furthermore, a fully connected layer with softmax normalization is used to predict the probability distribution of the position:
[0089]
[0090] in, For input sample x i Predicted position probability distribution, d p As a dimension of stance labeling, and These are learnable parameters;
[0091] Based on the predicted position probability distribution The classifier is trained using the cross-entropy loss with the true label y.
[0092]
[0093] in, For cross-entropy loss, N b This is a sample of the entire dataset.
[0094] In another embodiment, the present invention also provides a zero-shot stance detection system based on multi-expert collaborative learning, applied to a zero-shot stance detection method based on multi-expert collaborative learning in the above embodiments, to achieve high-quality zero-shot stance detection; see also Figure 2 As shown, the system includes:
[0095] The multi-expert collaborative semantic decoupling learning module is used to build a multi-expert collaborative semantic decoupling learning model based on the BERT model. The outputs of different experts use different BERT output layers as text encoders, and different experts obtain different aspects of the underlying semantic feature representation of the text input.
[0096] The feature filtering and fusion module based on the gating mechanism is used to filter and fuse multiple aspects of the underlying semantic features of the text to obtain a text underlying semantic feature representation that is effective for the position.
[0097] The semantic feature learning module is used to obtain high-level semantic features of text through the BERT encoder;
[0098] A classifier is used to classify the text stance based on the semantic features of the concatenated text, thereby enabling the detection of the text stance.
[0099] The specific working principle of this system is as follows:
[0100] First, multiple expert encoders are used to perform multi-expert collaborative semantic decoupling learning on the same batch of text input. This system uses different output layers of BERT as the experts' outputs to obtain different low-level semantic feature representations (for the same sentence). Then, these are concatenated and input into a feature filtering and fusion module based on a gating mechanism. The gating mechanism filters the features, selecting those valuable to the target stance. Next, the same batch of text is input into a semantic feature learning module, where the BERT encoder obtains high-level semantic features. These high-level features are then concatenated with the gating-filtered and fused multi-expert low-level features. This concatenation is then input into a stance classifier to obtain the stance representation of the text.
[0101] In one specific embodiment, PyTorch 1.8.1 is used to develop the system model of the present invention. Preferably, the pre-trained language model adopts the BERT-base-uncased model from huggingface Transformers. Specific parameters are set as follows: 768-dimensional word embedding vectors, learning rate 2e-5, Adam optimizer, mini-batch size set to 64, and number of expert encoders K=3. For the learning objective, the hyperparameter L2 regularization coefficient λ is preferably set to 1e-5.
[0102] The invention is experimentally verified using the two public datasets WT-WT and VAST:
[0103] WT-WT is a large-scale dataset for stance detection in the financial domain, containing 51,284 tweets. The dataset is constructed based on several company mergers and acquisitions. Each text is labeled with a stance (Refute, Comment, Support, and Unrelated), where Refute corresponds to opposition, Comment to neutrality, and Support to support. The experiments used four companies' actions against CVS_AET (CA), CI_ESRX (CE), ANTM_CI (AC), and AET_HUM (AH) as targets. To maintain consistency with other datasets, instances labeled Unrelated were ignored. The evaluation used a "leave-one-target-out" setup. Similarly, 15% of the training set was randomly selected as the validation set.
[0104] The VAST dataset is designed for zero-shot and few-shot stance detection. Built from New York Times article comments, it contains 18,548 examples from the "Room for Debate" section, with 5,630 different targets for zero-shot and few-shot pose detection. Each document has a stance label defined as {"Pro", "Con", "Neutral"} for a specific target (Pro for approval, Con for disapproval, Neutral for neutral). In the zero-shot stance detection task, the training and validation sets do not contain data with the same target as the test set.
[0105] The evaluation metrics used in the experiments followed those of previous position detection work. In WT-WT, the average of the macro F1 score and micro F1 score was calculated for each target to evaluate the model performance on the WT-WT dataset. For the VAST dataset, the macro F1 score for each label in the VAST dataset was used to evaluate the model's classification performance.
[0106] In this experiment, a comparative experiment was conducted between the present invention and the following model: the experimental results are shown in Table 1.
[0107] Table 1. Experimental Comparison Results
[0108]
[0109] In Table 1 above, bold text indicates the best result for each row, and underlined text indicates the second best result; the model is described below:
[0110] BiCond: Uses an LSTM-conditional attention model and a bidirectional conditional coding mechanism to achieve zero-shot stance detection.
[0111] CrossNet: Proposes a cross-target position classification model based on a self-attention network, which predicts the positions between targets by learning knowledge from the source target.
[0112] BERT model: A pre-trained language model that takes the state of the last hidden layer after adding [CLS] tags and inputs it into a linear layer for classification.
[0113] TOAD: A target adversarial neural network was established, which uses a bidirectional conditional coding stance detection model combined with an additional target-specific attention layer to generate target-invariant representations.
[0114] TPDG: By constructing a target-adaptive heterogeneous graph and leveraging word-level pragmatic dependencies, it improves the performance of stance detection for unknown targets. It distinguishes between target-related and target-independent words and constructs corresponding graph models to capture stance information and adapt to the contextual representation of different targets.
[0115] PT-HCL: A hierarchical contrastive learning framework consisting of a data augmentation module for a pre-training task and the hierarchical contrastive learning framework itself. It uses stance detection as a pre-task and employs the hierarchical contrastive learning framework to distinguish the correlations between features.
[0116] JointCL is a contrastive learning-based position detection strategy that captures the relationships between known and unknown targets using a target-oriented prototype graph contrastive learning strategy. It constructs a prototype graph to connect prototypes with target-based representations. Then, a contrastive learning strategy is used to learn the correlations and differences between target representations.
[0117] Table 1 presents the experimental results using the average of the model's three highest scores for each dataset's corresponding evaluation metric. On the WT-WT dataset, this invention achieved relatively superior results compared to others. Furthermore, we also obtained excellent or comparable results on the VAST dataset.
[0118] The results show that directly learning target-invariant features and applying them to zero-shot position detection does not yield ideal results. This limitation arises because semantic representation is multifaceted, and traditional methods for extracting coupled features neglect complex deep semantic features. In contrast, this invention achieves good results by fine-grainedly mining multifaceted features. By employing multi-expert feature mining, this invention obtains deep textual semantic features in a fine-grained manner.
[0119] In the experiment, the number of experts was fixed at K=3. On the WT-WT dataset, the performance of this invention outperformed other competing models. Compared with the two-stage JoinCL, this invention, by adopting a fine-grained approach to mining multiple low-level features, achieved better results than the two-stage strategy of clustering before prediction. This invention allows the model to obtain features constituting semantics at different levels, and can effectively combine specific target features to achieve zero-shot stance detection. Compared with contrastive learning models such as PT-HCL, because multiple experts obtain more diverse low-level transferable features of the text, it can fully capture the correlation between targets, thus achieving better performance.
[0120] On the VAST dataset, where the target data is unevenly distributed, more target-invariant features need to be acquired. Compared to JoinCL, our model obtains more low-level features through the selection and fusion of multiple features, which contain more target-invariant transferable features. Therefore, this invention is superior to JoinCL. Regarding PT-HCL, due to the uneven distribution of target data in VAST, it learns effective representations of unseen targets through a prototype graph learning strategy, thus slightly outperforming this invention in terms of Con.
[0121] As described in the above embodiments, those skilled in the art will understand that the present invention provides a zero-shot stance detection method and system based on multi-expert collaborative learning. This method improves the accuracy of zero-shot stance classification for unknown targets by introducing multiple experts to learn various low-level features in the semantic representation of text. First, an expert model is built based on BERT, with different experts using different low-level outputs of BERT as text encoding to obtain different aspects of the text's semantic representation. Second, a gating mechanism is used to filter effective features related to stance, thereby improving the feature quality of stance prediction. Finally, the selected multi-faceted semantic deconstruction features and high-level semantic features are comprehensively utilized to detect the text's stance. Extensive experiments on multiple benchmark datasets demonstrate that the present invention significantly improves accuracy in zero-shot stance detection tasks, exhibiting excellent performance and outperforming competing baseline models.
[0122] The system provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment, and will not be repeated here.
[0123] In addition, embodiments of the present invention also provide a storage medium storing one or more programs readable by a computing device. The one or more programs include instructions that, when executed by the computing device, cause the computing device to perform a zero-sample position detection method based on multi-expert collaborative learning as described in the above embodiments.
[0124] In embodiments of the present invention, the storage medium may be, for example, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, and any suitable combination thereof.
[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products, etc. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.
[0126] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The words "a" or "an" preceding a component do not exclude the presence of a plurality of such components. This invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer.
[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0128] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A zero-shot stance detection method based on multi-expert collaborative learning, characterized in that, The method includes the following steps: S1. Construct a multi-expert collaborative semantic decoupling learning model based on the BERT model. The outputs of different experts use different BERT output layers as text encoders. Different experts obtain different aspects of the underlying semantic features of the text input. S2. The underlying semantic features of the text obtained by the multi-expert collaborative semantic decoupling learning model are concatenated and input into the feature filtering and fusion module based on the gating mechanism. The gating mechanism is used to filter and fuse the underlying semantic features of the text from multiple aspects to obtain a text underlying semantic feature representation that is effective for the position. S3. Input the same batch of text into the semantic feature learning module and obtain high-level semantic features of the text through the BERT encoder; S4. The obtained high-level semantic features of the text are concatenated with the low-level semantic features of the text that have been screened and fused through a gating mechanism, and then input into the stance classifier to obtain the text stance classification.
2. The zero-shot stance detection method based on multi-expert collaborative learning according to claim 1, characterized in that, In step S1, firstly, BERT is used as the encoder f = BERT(·) to encode the content of the target t, text x, and target-text pair (x,t). The position label is regarded as a relationship between the target and the text, and its input forms are as follows: [CLS]t[SEP] [CLS]x[SEP] [CLS]t[SEP]x[SEP] Where [CLS] indicates the start of the classification task, and [SEP] indicates the separation or end; Secondly, different experts capture different grammatical features, and fine-grained mining yields target-irrelevant features that are easy to transfer; feature experts for different functions are: H (k，l) =f (k,l) =BERT(·)[l] in, The feature extracted by the k-th expert is taken from the l-th layer of the encoder output, d m is the dimension of the feature; f is the encoder; The text feature matrix consisting of n words is obtained by average pooling: Among them, h (k,l) H represents the pooled feature. (k,l) [i] is the feature vector of the i-th word.
3. The zero-shot stance detection method based on multi-expert collaborative learning according to claim 2, characterized in that, In step S2, firstly, the output features of multiple experts are aggregated based on concatenation, and these aggregated features are used as the features to be screened. The expression is: in, This is the aggregation result of k expert features. For splicing operations; Secondly, the gate function is defined as the neural network mapping g. φ (·), by learning the gating weights, it is used to control the selection of aggregated features; the encoding of the target t is used as the input of the gating function to select the features with the greatest correlation to the target stance judgment; z = f([CLS]t[SEP])[CLS] m=g φ (z)=W b z in, Let be the feature vector of target t. For learnable parameters, For the gated mask vector, These are the features after filtering.
4. The zero-shot stance detection method based on multi-expert collaborative learning according to claim 3, characterized in that, In step S3, the highest-level feature encoded by the encoder represents the target-dependent feature, expressed as: h=f([CLS]t[SEP]x[SEP]) [-1,CLS] in, High-level semantic features that the text target depends on.
5. The zero-shot stance detection method based on multi-expert collaborative learning according to claim 4, characterized in that, In step S4, the probability distribution of the predicted position is adopted using a fully connected layer with softmax normalization: in, For input sample x i Predicted position probability distribution, d p As a dimension of stance labeling, and These are learnable parameters; Based on the predicted position probability distribution The classifier is trained using the cross-entropy loss with the true label y. in, For cross-entropy loss, N b This is a sample of the entire dataset.
6. A zero-shot stance detection system based on multi-expert collaborative learning, characterized in that, The system employs a zero-shot stance detection method based on multi-expert collaborative learning as described in any one of claims 1-5 to perform stance detection. The system comprises: The multi-expert collaborative semantic decoupling learning module is used to build a multi-expert collaborative semantic decoupling learning model based on the BERT model. The outputs of different experts use different BERT output layers as text encoders, and different experts obtain different aspects of the underlying semantic feature representation of the text input. The feature filtering and fusion module based on the gating mechanism is used to filter and fuse multiple aspects of the underlying semantic features of the text to obtain a text underlying semantic feature representation that is effective for the position. The semantic feature learning module is used to obtain high-level semantic features of text through the BERT encoder; A classifier is used to classify the text stance based on the semantic features of the concatenated text, thereby enabling the detection of the text stance.