Zero or few sample vertical field detection method based on comparative learning and prompt learning

Abstract

The invention provides a zero or few sample stand detection method based on comparative learning and prompt learning. The method comprises the following steps: performing word segmentation and coding on an input sentence, and then performing instance generation and prompt word embedding; performing prompt learning by utilizing a BERT model, so that a standing detection task is converted into a gap filling test task, and standing inference is realized; constructing a gating multi-layer sensor in the BERT model, and realizing a vertical field detection task under the condition of zero or few samples; in the BERT model, utilizing a contrast learning framework to learn difference of vertical texts, strengthening similarity of samples of the same kind and improving distinction degree of samples of different kinds; and training and optimizing the BERT model to realize the vertical field detection under the condition of zero or few samples. According to the method, the powerful representation capability of comparative learning and the flexibility of prompt learning are fully utilized, the overall performance and effect of zero / few sample standing field detection are improved, the implementation cost of a standing field detection task is reduced, and the application range of the method is further widened.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a zero or few-sample stance detection method based on contrastive learning and prompt learning. Background Art

[0002] Stance detection is an important task in natural language processing, which aims to determine the stance of a text on a certain topic (support, oppose or neutral). Implementing zero / few-shot stance detection means that the detection model can still correctly classify the stance of a text without labeled data of a specific stance.

[0003] Existing zero / few-shot stance detection technologies include: 1) Attention mechanism, but its computational resources are large, and the target-specific attention mechanism is highly dependent on the dataset used, and the generalization ability on other datasets is insufficient. 2) Graph structure, but its algorithm has high computational complexity, and there are some cases where it does not work well under certain targets and cannot be effectively adapted under certain conditions. 3) Models for sentiment analysis and semantic understanding, but in actual applications, there are cases where knowledge transfer is incomplete or not applicable to specific scenarios, which requires further research and optimization to improve the transfer effect, and complex knowledge transfer mechanisms and models will reduce the interpretability of the model, making it difficult to understand the specific basis for the model to make decisions; this poses a challenge to credibility and acceptance in certain application scenarios. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a zero or few-shot stance detection method based on contrastive learning and prompt learning, which fully utilizes the powerful representation ability of contrastive learning and the flexibility of prompt learning, improves the overall performance and effect of zero / few-shot stance detection, reduces the implementation cost of stance detection tasks, and further broadens its application scope.

[0005] To achieve the above object, the present invention provides the following scheme: a zero or few-sample stance detection method based on contrastive learning and prompt learning, comprising the following steps:

[0006] Segment and encode the input sentence to obtain encoding information, and input the encoding information into the BERT model for instance generation and prompt word embedding to obtain a model representation for detecting meaning and stance judgment;

[0007] Using the BERT model for prompt learning, the stance detection task is transformed into a fill-in-the-blank test task, achieving stance inference under zero or few-sample conditions;

[0008] Based on the gating mechanism, a gated multilayer perceptron is constructed in the BERT model to capture the semantic relationship between text and prompts, so as to achieve stance detection tasks under zero or few-shot conditions;

[0009] In the BERT model, a contrastive learning framework is used to learn the differences in stance texts to strengthen the similarity of samples of the same category and improve the distinction of samples of different categories;

[0010] The BERT model is trained and optimized to achieve stance detection under zero or few-shot conditions.

[0011] Optionally, the input sentence is segmented and encoded to obtain encoding information, and the encoding information is input into the BERT model for instance generation and prompt word embedding to obtain a model representation for detecting meaning and stance determination, including:

[0012] Select a tokenizer to divide the input sentence into multiple basic elements to obtain a list of fragments, and add special tags before and after the list of fragments to adapt to the input format of the BERT model;

[0013] Mapping the fragment list with the vocabulary in the BERT model to obtain an ID sequence of the fragment list, and generating an input vector from the ID sequence;

[0014] The ID sequence is input into the BERT model to generate an instance embedding of each input sentence, and the prompt word embedding of each input sentence is generated by combining the prompt template and using the BERT model.

[0015] Optionally, the BERT model is used for prompt learning to transform the stance detection task into a fill-in-the-blank test task, thus achieving stance inference under zero or few-sample conditions, including:

[0016] Creating a prompt template, using marking symbols to transform the stance detection task into a fill-in-the-blank test task, and designing a variety of prompt forms based on the fill-in-the-blank test task;

[0017] The actual input text is embedded into the prompt template to generate a specific prompt text, the specific prompt text is encoded by the BERT model and special tags are added, and the stance label is obtained by using the prediction function of the tag symbol;

[0018] Filling placeholders with the marking symbols in the prompt template to predict stance labels at corresponding positions, and predefining the predicted stance labels to obtain a candidate vocabulary list;

[0019] Using the BERT model to predict the words on the marked symbols of the input text to obtain the filling-in-the-blank result of the filling-in-the-blank test task, and using the cross entropy loss to calculate the difference between the predicted probability and the true label to obtain the classification loss for measuring the prediction quality and accuracy;

[0020] The filling-in-the-blank result is matched with a predefined stance label to complete the mapping of candidate words, generate a final stance label, and realize stance inference under zero or few sample conditions.

[0021] Optionally, the gated multilayer perceptron includes an input layer for text embedding and prompt embedding, a hidden layer for feature extraction and capturing semantic relationships between text and prompts, a gating mechanism for dynamically adjusting prompt weights, and an output layer for stance detection and generating a final stance prediction result.

[0022] Optionally, based on the gating mechanism, a gated multilayer perceptron is constructed in the BERT model to capture the semantic relationship between the text and the prompt, so as to achieve stance detection tasks under zero or few-shot conditions, including:

[0023] Generate text embedding and prompt embedding using the input layer, and concatenate the text embedding and prompt embedding in the feature dimension to form an input vector;

[0024] Mapping the input vector to a new hidden dimension using the hidden layer and the fully connected network, and extracting semantic features between text and prompts;

[0025] Using the gating mechanism to linearly transform the hint embedding separately and map it to the new hidden dimension to obtain a linear transformation result, and using the Sigmoid activation function to activate the linear transformation result to generate a hint weight;

[0026] Multiplying the output of the hidden layer by the prompt weight element-wise to dynamically adjust the prompt influence;

[0027] The feature representation adjusted by the gating mechanism is mapped to the dimension of the stance category through a linear layer, and then the mapping result of the stance dimension is converted into a probability distribution using a softmax activation function, and the category with the highest probability is selected as the final stance prediction result.

[0028] Optionally, concatenate the text embedding and the prompt embedding in the feature dimension to form an input vector, including:

[0029] When the vector dimensions of text embedding and hint embedding are the same, the vector dimensions are aligned directly. When the vector dimensions of text embedding and hint embedding are different, a linear transformation or zero padding is used to align the vector dimensions.

[0030] The aligned text embedding and prompt embedding are concatenated through a concatenation operation to obtain the input vector.

[0031] Optionally, mapping the input vector to a new hidden dimension using the hidden layer and the fully connected network, and extracting semantic features between the text and the prompt, including:

[0032] Identify the composition of the input vector and select the output dimension of the hidden layer, and then initialize the weight matrix and bias vector;

[0033] Multiplying the input vector by a weight matrix and adding the weight matrix to a bias vector to map the input vector to a new hidden dimension of the hidden layer;

[0034] An activation function is selected to learn and represent nonlinear features, and then the weights and biases in the hidden layer are updated through back propagation to achieve feature extraction.

[0035] Optionally, the linear layer is implemented by initializing the size of the weight matrix and the bias vector. The initialized weight matrix and the bias vector compress and transform the feature representation adjusted by the gating mechanism from the high-dimensional space to the space of stance judgment, and the transformed feature representation is multiplied by the initialized weight matrix and then added to the initialized bias vector to complete the linear transformation.

[0036] Optionally, in the BERT model, a contrastive learning framework is used to learn the differences in stance texts to strengthen the similarity of samples of the same type and improve the distinction of samples of different types, including:

[0037] Generating positive and negative sample pairs in training data with stance labels, and using the BERT model to generate embedded representations of the positive and negative sample pairs to obtain embedded vectors;

[0038] Calculating the similarity between the embedding vectors using cosine similarity or similarity metric, and calculating the contrast loss of the positive and negative sample pairs;

[0039] A comprehensive loss function is obtained by combining the contrast loss and the classification loss of the fill-in-the-blank test task, and the comprehensive loss function is trained and optimized to optimize the joint process of contrastive learning and cue learning.

[0040] The present invention discloses the following technical effects by providing a zero or few-sample stance detection method based on contrastive learning and prompt learning:

[0041] 1. In the implementation of zero or few-shot stance detection, word segmentation and encoding are key steps to convert text into a format that the model can understand and process. They also generate representations in the BERT model system for actual input sentences and preset prompts. These representations (especially the embedding of the [MASK] marker) will be used to detect meaning and stance judgment, and assist in the implementation of zero-shot stance detection. This process makes full use of the language understanding ability of the pre-trained model to help the model infer and generate more accurate outputs.

[0042] 2. Prompted learning is very suitable for zero or few-shot learning, that is, it can complete specific tasks without explicitly labeled data. This prompt learning method takes advantage of the powerful ability of language models in fill-in-the-blank test tasks, transforming complex natural language understanding tasks into simple fill-in-the-blank tasks that they are good at, thereby achieving efficient and accurate stance detection.

[0043] 3. Contrastive learning can enhance the model's ability to distinguish texts with different stances. Using the contrastive learning framework, the model can effectively learn the differences in stance texts, making it more accurate and robust when predicting stances in the fill-in-the-blank task learned through prompts. The model performs well in detecting different stances by continuously strengthening the similarity of samples of the same type and improving the distinction between samples of different types.

[0044] 4. By designing a gated multilayer perceptron, the semantic relationship between text and prompts can be effectively captured, and the weight of prompts can be dynamically adjusted to achieve fine-grained zero or few-shot stance detection tasks. By combining prompt learning and contrastive learning methods, the reliance on a large amount of labeled data is reduced, the implementation cost of stance detection tasks is effectively reduced, and the effect of stance detection is improved, which will further broaden its application scope and provide more accurate decision support.

[0045] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A schematic diagram of a method flow chart provided by an embodiment of the present invention;

[0048] Figure 2 A structural diagram of a gated multilayer perceptron provided in an embodiment of the present invention;

[0049] Figure 3A flowchart of the final stance prediction result provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1-3 As shown, the present invention provides a zero or few-sample stance detection method based on contrastive learning and prompt learning, comprising the following steps:

[0053] 1. Segment and encode the input sentence to obtain encoding information, and input the encoding information into the BERT model for instance generation and prompt word embedding to obtain a model representation for detecting meaning and stance judgment.

[0054] In implementing zero-shot stance detection, word segmentation and encoding are key steps to convert text into a format that the model can understand and process. Specifically, they include:

[0055] 1.1 Select a tokenizer to divide the input sentence into multiple basic elements to obtain a list of fragments, and add special tags before and after the list of fragments to adapt to the input format of the BERT model.

[0056] Before selecting a tokenizer, tokenization is first performed, which is to convert a text string into a series of smaller units, usually words or subwords, so that the model can process it. This is the first step in processing natural language input. BERT uses a word-based segmentation method called WordPiece. It can handle a large number of variations and combinations with a smaller vocabulary.

[0057] Word segmentation process, for example, first, the input sentence "The government should increase education funding" is divided into several basic elements. For simplified Chinese or other languages ​​that do not use spaces to segment words, the word segmenter will identify common subwords or characters. The WordPiece word segmenter looks for the longest matching subword unit, which means that if a word does not exist in the dictionary, it will be split into known subwords. For example, the word "unbelievable" may be split into ["un", "##believ", "##able"].

[0058] In order to adapt to the input format of the BERT model, special tags are usually added before and after the segmented fragment list: [CLS] (classification tag) is used for the beginning of the sentence, and [SEP] (separation tag) is used for the end of the sentence. In the case of multiple sentences, [SEP] is inserted between sentences.

[0059] 1.2 Mapping the fragment list with the vocabulary in the BERT model to obtain an ID sequence of the fragment list, and generating an input vector from the ID sequence;

[0060] For example, the input vector includes:

[0061] input_ids: the ID sequence mentioned above;

[0062] token_type_ids: used to indicate the case of two sentences (if there is only one sentence, all values ​​are 0);

[0063] attention_mask: used to indicate which positions are filled and do not need to be paid attention to (in actual input, this vector is usually not used because there is no necessary filling value).

[0064] 1.3 Input the ID sequence into the BERT model to generate instance embedding of each input sentence, combine the prompt template and use the BERT model to generate prompt word embedding of each input sentence.

[0065] Among them, the prompt template, such as "The stance' supports 'on the topic is [MASK].", needs to generate the embedding of this sentence through the BERT model. The embedding vector of [MASK] is the part that the model needs to predict based on the context. After passing this prompt sentence through BERT, we are concerned about the vector representation of the corresponding position of [MASK] provided by BERT after calculation, which indicates the expected prediction tag in the task.

[0066] Through the above processing, we generate representations in the BERT system for the actual input sentences and the preset prompt sentences. These representations (especially the embedding of [MASK]) will be used to detect meaning and stance judgment, and assist in achieving zero-shot stance detection. This process fully utilizes the language understanding ability of the pre-trained model to help the model infer and generate more accurate outputs.

[0067] 2. Use the BERT model for prompt learning to transform the stance detection task into a fill-in-the-blank test task, and achieve stance inference under zero or few sample conditions; specifically:

[0068] 2.1 Create a prompt template, use the marker symbol [MASK] to transform the stance detection task into a fill-in-the-blank test task, and design a variety of prompt forms based on the fill-in-the-blank test task;

[0069] Design a prompt template. For example, for the Chinese case, convert "The government should increase education funding" into: "The stance on the sentence 'The government should increase education funding' is [MASK]", and the English version can be "[MASK] is the stance of'The government should increase education funding.'".

[0070] Design multiple prompt forms to cover more expressions. For example:

[0071] Template 1: "Regarding 'the government should increase education funding', their opinion is [MASK]."

[0072] Template 2: "The position expressed in the article 'The government should increase education funding' is [MASK]."

[0073] 2.2 The actual input text is embedded into the prompt template to generate a specific prompt text, the specific prompt text is encoded using the BERT model and special tags (such as `[CLS]`, `[SEP]`, etc.) are added, and the stance label is obtained using the prediction function of the tag symbol [MASK].

[0074] 2.3 Using the marking symbol to fill the placeholder in the prompt template to predict the stance label of the corresponding position, and predefine the predicted stance label to obtain a candidate vocabulary list;

[0075] The [MASK] tag is used for placeholders, and the model will predict possible words for this position. Since the positions we want to predict are specific labels (such as "support", "oppose", "neutral"), these labels are defined in advance and a list of candidate words is provided.

[0076] Candidate words: A list of candidate words that define the position labels. For example, "support" for support, "oppose" for opposition, and "neutral" for neutrality;

[0077] English: support: "support", opposition: "oppose", neutral: "neutral";

[0078] Chinese: Support: “support”, Oppose: “oppose”, Neutral: “neutral”.

[0079] 2.4 Use the BERT model to predict the words on the marked symbols of the input text to obtain the fill-in-the-blank result of the fill-in-the-blank test task, and use the cross entropy loss to calculate the difference between the predicted probability and the true label to obtain the classification loss used to measure the prediction quality and accuracy.

[0080] 2.5 Match the fill-in-the-blank result with the predefined stance label to complete the mapping of the candidate words (e.g., if the predicted word is "support", the stance is "support"), generate the final stance label, and realize stance inference under zero or few sample conditions.

[0081] Through the above steps, the zero or few-shot stance detection task is converted into a fill-in-the-blank test task. Pre-trained language models such as BERT can use their powerful language understanding and generation capabilities to infer the stance of the sentence by predicting the word at the position of the `[MASK]` tag. Finally, the prediction results are mapped to specific stance labels, achieving stance detection under zero-shot conditions. This prompt learning method utilizes the powerful capabilities of the language model in the fill-in-the-blank test task, converting the complex natural language understanding task into a simple fill-in-the-blank task that it is good at, thereby achieving efficient and accurate stance detection.

[0082] 3. Based on the gating mechanism, a gated multilayer perceptron is constructed in the BERT model to capture the semantic relationship between text and prompts, so as to achieve stance detection tasks under zero or few-sample conditions.

[0083] 3.1 As Figure 2 As shown, the gated multilayer perceptron includes an input layer for text embedding and prompt embedding, a hidden layer for feature extraction and capturing the semantic relationship between text and prompt, a gating mechanism for dynamically adjusting prompt weights, and an output layer for stance detection and generating the final stance prediction result.

[0084] 3.2 As Figure 3 As shown in Figure 1, the process of using a gated multilayer perceptron to perform stance detection tasks under zero or few-shot conditions is as follows:

[0085] 3.21 Generate text embedding and prompt embedding using the input layer, and concatenate the text embedding and prompt embedding in the feature dimension to form an input vector.

[0086] When the vector dimensions of the text embedding and the hint embedding are the same, the vector dimensions are directly aligned. When the vector dimensions of the text embedding and the hint embedding are different, a linear transformation or zero padding method (but linear transformation is usually more flexible and accurate) is used to align the vector dimensions. The aligned text embedding and hint embedding are concatenated through a concatenation operation to obtain the input vector.

[0087] 3.22 Mapping the input vector to a new hidden dimension using the hidden layer and the fully connected network, and extracting semantic features between the text and the prompt; specifically including:

[0088] Identify the composition of the input vector and select the output dimension of the hidden layer (i.e., `hidden_dim`). This dimension should be selected based on the complexity of the task, the amount of data, and the computing resources. Choose a value that balances the network computing power and the actual application requirements.

[0089] Initialize the weight matrix and bias vector: the size of the weight matrix is ​​(input vector dimension x hidden_dim), and the bias vector is a vector of the hidden layer dimension size; weight initialization has a great influence on the training process and convergence speed of the network. Commonly used methods include random initialization, Gaussian distribution initialization or Xavier initialization;

[0090] When passing through the hidden layer, the input vector is multiplied by the weight matrix and then added to the bias vector to map the input vector to the new hidden dimension of the hidden layer; the mathematical expression of this step is: `hidden_output = activation_function(W*input_vector+b)`, where \(W\) is the weight matrix, \(b\) is the bias, and `activation_function` is the activation function;

[0091] An activation function is selected to learn and represent nonlinear features, and then the weights and biases in the hidden layer are updated through back propagation to achieve feature extraction, so that the model can more accurately reflect the true structure of the data.

[0092] 3.23 Use the gating mechanism to perform linear transformation on the prompt embedding separately, and use the fully connected network to map it to the new hidden dimension (`hidden_dim` dimension) to obtain the linear transformation result, use the Sigmoid activation function to activate the linear transformation result, compress the output value to the [0,1] interval, and generate the prompt weight.

[0093] 3.24 The output of the hidden layer is multiplied element-wise by the prompt weight to dynamically adjust the prompt influence; this ensures that different prompts act on different instances with different weights, so that the influence of the prompt is closely aligned with the semantic context.

[0094] 3.25 Mapping the feature representation adjusted by the gating mechanism to the dimension representing the stance category (such as support, opposition or neutrality) of the original text through a linear layer, and then using the softmax activation function to convert the mapping result of the stance dimension into a probability distribution, and selecting the category with the largest probability as the final stance prediction result;

[0095] The linear layer is implemented by initializing the size of the weight matrix and the bias vector. The initialized weight matrix and the bias vector compress and transform the feature representation adjusted by the gating mechanism from the high-dimensional space to the space of stance judgment, and multiply the transformed feature representation with the initialized weight matrix and then add it to the initialized bias vector to complete the linear transformation.

[0096] 4. In the BERT model, the contrastive learning framework is used to learn the differences in stance texts to strengthen the similarity of samples of the same category and improve the distinction of samples of different categories; specifically, the following steps are performed:

[0097] 4.1 generating positive and negative sample pairs in the training data with stance labels, and using the BERT model to generate embedded representations of the positive and negative sample pairs to obtain embedded vectors;

[0098] Positive pairs: consist of texts with the same stance. These texts share similar opinions or stances and constitute positive pairs. For example, two articles both support the government to increase education funding.

[0099] Negative pairs: consist of texts with different positions. These texts are opposed to each other in terms of their positions. For example, one article supports and one article opposes the government to increase education funding.

[0100] Methods for generating positive and negative sample pairs: 1) Selection from training data: Select texts with the same stance label from the labeled training data as positive sample pairs; select texts with different stance labels as negative sample pairs. 2) Use domain knowledge to construct pairs. For example, support and opposition to education funding can be used as different stance pairs in multiple fields. 3) Data enhancement: Generate multiple different versions of the same text as positive sample pairs through methods such as synonym replacement or sentence rewriting; generate texts with opposing stances for a given text through opposing viewpoint generation technology to form negative sample pairs.

[0101] 4.2 Calculate the similarity between the embedding vectors using cosine similarity or similarity measurement, and design a loss function to calculate the contrast loss of the positive and negative sample pairs so that the distance between the positive sample pairs is minimized and the distance between the negative sample pairs is maximized;

[0102] For example:

[0103] SimCLR loss: A commonly used contrastive loss, defined as:

[0104] Positive sample pair: The similarity obtained through model mapping is high (cosine similarity is close to 1).

[0105] Negative sample pairs: The similarity obtained by model mapping is low (cosine similarity is close to 0 or negative).

[0106] Margin-based loss: Another contrast loss, by defining a threshold (margin). If the distance between the negative sample pair is less than the margin, additional loss will be incurred;

[0107] For positive sample pairs, we hope that their cosine distance is as close to 0 as possible;

[0108] For negative sample pairs, we hope that their cosine distance is greater than margin.

[0109] 4.3 Combining the contrast loss and the classification loss of the fill-in-the-blank test task to obtain a comprehensive loss function, training and optimizing the comprehensive loss function to optimize the joint process of contrastive learning and cue learning.

[0110] 5. Train and optimize the BERT model to achieve stance detection under zero or few-shot conditions. Finally, the model performance can be evaluated using an unlabeled test dataset to identify the model's zero-shot stance detection capabilities on new data.

[0111] Therefore, the present invention provides a zero or few-sample stance detection method based on contrastive learning and prompt learning, which can achieve:

[0112] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0113] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A zero or few-shot stance detection method based on contrastive learning and cue learning, characterized in that: The following steps are involved: Segment and encode the input sentence to obtain encoding information, and input the encoding information into the BERT model for instance generation and prompt word embedding to obtain a model representation for detecting meaning and stance judgment; Using the BERT model for prompt learning, the stance detection task is transformed into a fill-in-the-blank test task, achieving stance inference under zero or few-sample conditions; Based on the gating mechanism, a gated multilayer perceptron is constructed in the BERT model to capture the semantic relationship between text and prompts, so as to achieve stance detection tasks under zero or few-shot conditions; In the BERT model, a contrastive learning framework is used to learn the differences in stance texts to strengthen the similarity of samples of the same category and improve the distinction of samples of different categories; The BERT model is trained and optimized to achieve stance detection under zero or few-shot conditions.

2. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 1, characterized in that: The input sentence is segmented and encoded to obtain encoding information, and the encoding information is input into the BERT model for instance generation and prompt word embedding to obtain a model representation for detecting meaning and position determination, including: Select a tokenizer to divide the input sentence into multiple basic elements to obtain a list of fragments, and add special tags before and after the list of fragments to adapt to the input format of the BERT model; Mapping the fragment list with the vocabulary in the BERT model to obtain an ID sequence of the fragment list, and generating an input vector from the ID sequence; The ID sequence is input into the BERT model to generate an instance embedding of each input sentence, and the prompt word embedding of each input sentence is generated by combining the prompt template and using the BERT model.

3. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 2, characterized in that: The BERT model is used for prompt learning to transform the stance detection task into a fill-in-the-blank test task, and to achieve stance inference under zero or few-sample conditions, including: Creating a prompt template, using marking symbols to transform the stance detection task into a fill-in-the-blank test task, and designing a variety of prompt forms based on the fill-in-the-blank test task; The actual input text is embedded into the prompt template to generate a specific prompt text, the specific prompt text is encoded by the BERT model and special tags are added, and the stance label is obtained by using the prediction function of the tag symbol; Filling placeholders with the marking symbols in the prompt template to predict stance labels at corresponding positions, and predefining the predicted stance labels to obtain a candidate vocabulary list; Using the BERT model to predict the words on the marked symbols of the input text to obtain the filling-in-the-blank result of the filling-in-the-blank test task, and using the cross entropy loss to calculate the difference between the predicted probability and the true label to obtain the classification loss for measuring the prediction quality and accuracy; The filling-in-the-blank result is matched with a predefined stance label to complete the mapping of candidate words, generate a final stance label, and realize stance inference under zero or few sample conditions.

4. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 3, characterized in that: The gated multilayer perceptron includes an input layer for text embedding and prompt embedding, a hidden layer for feature extraction and capturing the semantic relationship between text and prompt, a gating mechanism for dynamically adjusting prompt weights, and an output layer for stance detection and generating the final stance prediction results.

5. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 4, characterized in that: Based on the gating mechanism, a gated multilayer perceptron is constructed in the BERT model to capture the semantic relationship between text and prompts, so as to achieve stance detection tasks under zero or few-shot conditions, including: Generate text embedding and prompt embedding using the input layer, and concatenate the text embedding and prompt embedding in the feature dimension to form an input vector; Mapping the input vector to a new hidden dimension using the hidden layer and the fully connected network, and extracting semantic features between text and prompts; Using the gating mechanism to linearly transform the hint embedding separately and map it to the new hidden dimension to obtain a linear transformation result, and using the Sigmoid activation function to activate the linear transformation result to generate a hint weight; Multiplying the output of the hidden layer by the prompt weight element-wise to dynamically adjust the prompt influence; The feature representation adjusted by the gating mechanism is mapped to the dimension of the stance category through a linear layer, and then the mapping result of the stance dimension is converted into a probability distribution using a softmax activation function, and the category with the highest probability is selected as the final stance prediction result.

6. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 5, characterized in that: Concatenate text embedding and prompt embedding in the feature dimension to form an input vector, including: When the vector dimensions of text embedding and hint embedding are the same, the vector dimensions are aligned directly. When the vector dimensions of text embedding and hint embedding are different, a linear transformation or zero padding is used to align the vector dimensions. The aligned text embedding and prompt embedding are concatenated through a concatenation operation to obtain the input vector.

7. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 6, characterized in that: The input vector is mapped to a new hidden dimension using the hidden layer and the fully connected network, and semantic features between the text and the prompt are extracted, including: Identify the composition of the input vector and select the output dimension of the hidden layer, and then initialize the weight matrix and bias vector; Multiplying the input vector by a weight matrix and adding the weight matrix to a bias vector to map the input vector to a new hidden dimension of the hidden layer; An activation function is selected to learn and represent nonlinear features, and then the weights and biases in the hidden layer are updated through back propagation to achieve feature extraction.

8. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 7, characterized in that: The linear layer is implemented by initializing the size of the weight matrix and the bias vector. The initialized weight matrix and the bias vector compress and transform the feature representation adjusted by the gating mechanism from the high-dimensional space to the space of stance judgment, and multiply the transformed feature representation with the initialized weight matrix and then add it to the initialized bias vector to complete the linear transformation.

9. The zero or few-shot stance detection method based on contrastive learning and cue learning according to claim 8, characterized in that: In the BERT model, the contrastive learning framework is used to learn the differences in stance texts to strengthen the similarity of samples of the same type and improve the distinction of samples of different types, including: Generating positive and negative sample pairs in training data with stance labels, and using the BERT model to generate embedded representations of the positive and negative sample pairs to obtain embedded vectors; Calculating the similarity between the embedding vectors using cosine similarity or similarity metric, and calculating the contrast loss of the positive and negative sample pairs; A comprehensive loss function is obtained by combining the contrast loss and the classification loss of the fill-in-the-blank test task, and the comprehensive loss function is trained and optimized to optimize the joint process of contrastive learning and cue learning.

Images