Zero sample standing field detection method and system based on discussion data multi-task fine tuning and knowledge enhancement

Abstract

The invention relates to the technical field of field detection, in particular to a zero-sample field detection method and system based on multi-task fine tuning and knowledge enhancement of argument data, and the method comprises the steps: firstly, searching related knowledge between a text and an object from an external knowledge base through a prompt project, and carrying out the knowledge enhancement of fine tuning data and test data; then analyzing and discussing that a plurality of tasks are included in the corpus data, and meanwhile, preprocessing data of each task; and finally, performing multi-task fine tuning on the open-source large model by using the discussion corpus data and external knowledge, and performing field reasoning by using the model after fine tuning. According to the method, multi-task fine tuning is carried out on the open-source large model by utilizing discussion corpus data with rigorous logic and relatively low noise, so that the performance of model site detection is improved; and meanwhile, the model takes related knowledge returned by an external knowledge base as a context, so that the accuracy of the model in text understanding and analysis aspects is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of stance detection technology, and in particular to a zero-sample stance detection method and system based on multi-task fine-tuning and knowledge enhancement of debate data. Background Art

[0002] Stance Detection is an important task in the field of natural language processing, which aims to automatically determine the stance or attitude of a text towards a specific object, including support, opposition or neutrality.

[0003] Zero-shot stance detection aims to determine the stance or attitude of a text towards a specific topic or object that was not covered during the training phase. In the real world, text expressions and target objects are often complex and changeable, not limited to a specific existing field or topic, and are usually difficult to predict. Zero-shot stance detection aims to meet the challenges in similar complex scenarios. Although there have been many achievements in the existing zero-shot stance detection research, its performance is not ideal and it is still a difficult problem in the field of stance detection research, which is roughly reflected in the following two aspects:

[0004] On the one hand, the scarcity of data annotation and the limitations of social media data limit the generalization ability of stance detection models. Mainstream social media stance detection datasets tend to focus on specific topics, involve a small number of objects and are highly domain-specific. At the same time, the way social media users express their opinions and stances is often informal, colloquial, short text, and semantically obscure, resulting in weak logic and spelling, grammatical and other errors, which increases the difficulty of semantic understanding and processing analysis. These problems also make deep learning models with data-driven as the core often only able to learn surface and domain-specific features, and it is difficult to obtain the stance association between text and objects. In addition, manual stance annotation is costly and highly subjective, making it very difficult to annotate high-quality stance datasets. On the other hand, the complexity of the stance detection task itself and the lack of contextual information exacerbate the difficulty of model learning. The stance detection task is essentially a natural language reasoning task, which requires the model to have relevant professional knowledge and a deep understanding of the semantics of the text in order to make accurate judgments, such as Figure 1 As shown in Figure 2.2, since texts usually do not contain explicit knowledge related to objects, and implicit information is often hidden in short content, relevant knowledge needs to be incorporated into stance learning as context to enhance the model’s comprehensive understanding of texts and objects, thereby improving the model’s ability to solve stance detection tasks. Summary of the invention

[0005] The present invention aims to solve the challenges faced by zero-sample stance detection in terms of scarce data annotation and the complexity of the task itself. A zero-sample stance detection method and system based on multi-task fine-tuning and knowledge enhancement of argumentation data is proposed. The method and system significantly improve the effect of stance detection by combining multi-task fine-tuning of argumentation data with knowledge enhancement technology that retrieves relevant knowledge between text and objects from an external database.

[0006] To achieve the above purpose, the technical solution adopted is:

[0007] A zero-shot stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data, including:

[0008] Firstly, we use hint engineering to retrieve relevant knowledge between text and objects from an external knowledge base to enhance the knowledge of fine-tuning data and test data.

[0009] Then, the debate corpus data contains multiple tasks, and the data of each task is preprocessed separately;

[0010] Finally, we use debate corpus data and external knowledge to fine-tune the open source large model for multiple tasks, and use the fine-tuned model to perform position reasoning.

[0011] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, knowledge enhancement of fine-tuning data and test data includes:

[0012] A large language model is selected as the external knowledge base, and knowledge related to texts and objects is retrieved from the external knowledge base by designing prompts, filling in texts and objects; if the external knowledge base returns relevant knowledge, this knowledge will serve as a supplement to the relevant knowledge in the fine-tuning data and test data.

[0013] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, a filtering mechanism is used to filter the retrieved relevant knowledge. If the external knowledge base provides abnormal or incorrect information, this information will be filtered and the returned result will be empty.

[0014] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, the debate corpus data includes four tasks, namely:

[0015] ①Stance detection: determine the stance of the premise sentence on the target object; the stance detection task acquires external knowledge;

[0016] ② Opinion extraction: extracting opinions, views or attitudes on specific themes or topics from text information;

[0017] ③ Argument generation: information and arguments that support or refute a specific position or viewpoint;

[0018] ④ Relational reasoning: Analyze the information and context in the text to infer the relationship between the premise sentence and the hypothesis sentence.

[0019] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, the data of each task are preprocessed and constructed to meet the format requirements of the fine-tuning data of the corresponding task.

[0020] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, the open source large model adopts LLAMA3-8b-Instruct and Mistral-7b-Instruct.

[0021] According to the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data of the present invention, further, the stance reasoning process is: first, use the knowledge enhancement method to retrieve the relevant knowledge of the text and the object in the test data; then, combine the retrieved relevant knowledge with the text and the object, and perform prompt engineering according to the fine-tuning data format; finally, use the fine-tuned model to infer the stance of the text in the test data on the target object.

[0022] Furthermore, the present invention also proposes a zero-sample stance detection system based on multi-task fine-tuning of argument data and knowledge enhancement, which is characterized by comprising a knowledge enhancement module, a multi-task prompt construction module of argument data and a fine-tuning reasoning module, wherein:

[0023] The knowledge enhancement module is used to use hint engineering to retrieve relevant knowledge between text and objects from an external knowledge base to perform knowledge enhancement on fine-tuning data and test data;

[0024] The debate data multi-task prompt construction module is used to analyze the debate corpus data containing multiple tasks and pre-process the data of each task separately;

[0025] The fine-tuning reasoning module is used to perform multi-task fine-tuning on the open source large model using debate corpus data and external knowledge, and use the fine-tuned model to perform position reasoning.

[0026] The beneficial effects achieved by adopting the above technical solution are:

[0027] 1. In view of the scarcity of data annotation and the limitations of social media data, the present invention uses debate corpus data related to the stance detection task on the open source large model to perform supervised multi-task fine-tuning, better learn and transfer zero-sample stance detection and reasoning analysis capabilities, and improve the performance of model stance detection.

[0028] 2. In view of the complexity of the stance detection task itself and the lack of contextual information, the present invention uses a large language model as an external knowledge base, detects relevant knowledge of the text and object to be tested from the external knowledge base as context information, and enhances the model's ability to understand the semantics of the text and object to be tested.

[0029] 3. The method of the present invention significantly improves the accuracy of zero-sample stance detection. Compared with ChatGPT, the average F1 value of the MDKA model framework of the present invention on the public datasets SemEval 2016task A and P-Stance is improved by 15.74% and 3.55% respectively, verifying the effectiveness and superiority of the zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0031] Figure 1 It is an example of the relevant knowledge of a text and an object;

[0032] Figure 2 It is a framework diagram of a zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to an embodiment of the present invention;

[0033] Figure 3 It is a comparison of the influence of the relevant knowledge of the embodiment of the present invention on long and short texts. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0035] In the real world, stance detection tasks usually require evaluating the stance or attitude of the text to be tested towards a new object that has never appeared before, i.e., zero-shot stance detection. This is one of the key points and difficulties in stance detection research and is of great significance for real-world applications. However, zero-shot stance detection usually has the characteristics of scarce data annotation and complex tasks, which often leads to low detection performance.

[0036] Stance detection aims to determine the stance or attitude of a text towards a specific object. Define a dataset where k i is to obtain text x through an external knowledge base i and object t iThe goal of this case is to use the relevant knowledge k i As context, infer the text x i For target object i Position i , including support, opposition or neutral.

[0037] From the perspective of relevant task data and external knowledge enhancement, this paper proposes a zero-shot stance detection method (i.e., MDKA model) based on multi-task fine-tuning and knowledge enhancement of debate data. Figure 2 As shown, it contains the following contents:

[0038] Step S101: Retrieve relevant knowledge between text and objects from an external knowledge base using prompt engineering to perform knowledge enhancement on fine-tuning data and test data.

[0039] In this paper, a prompt-driven knowledge enhancement method is adopted, which uses large language models (LLMs) as external knowledge bases and retrieves knowledge from external knowledge bases through prompt engineering to solve the problem of lack of highly specialized and domain-specific knowledge in specific tasks.

[0040] Large language models have been shown to store a large amount of factual knowledge in their parameters and have achieved state-of-the-art results on multiple natural language processing tasks after fine-tuning. Therefore, this case selected the Qwen large model as an external knowledge base, and retrieved knowledge related to text and objects from the external knowledge base by designing prompts, filling in text and objects. Once the external knowledge base returns relevant knowledge, this knowledge will be integrated into the data of the stance detection task (fine-tuning data and test data) to enhance the model's comprehensive understanding of text and objects and stance reasoning ability, thereby significantly improving the performance of the model and generating more accurate output results.

[0041] Using large language models (LLMs) such as Qwen and ChatGPT as external knowledge bases has many advantages: they can not only accurately acquire knowledge, but also respond to queries quickly and provide extensive and accurate information. These models are easy to use and resource-efficient, and effectively support knowledge enhancement tasks in various fields. In addition, the knowledge they provide usually has a certain degree of interpretability, which helps to ensure the credibility and quality of the output knowledge. However, potential problems include hallucinations or the generation of lengthy and irrelevant information. In order to ensure the accuracy of information retrieval, a filtering mechanism is used in this case to standardize and filter the retrieved knowledge to ensure that only concise and effective information is returned. If the external knowledge base provides abnormal or incorrect information, such as irrelevant answers, no solutions or neutral replies, this information will be filtered out, thereby ensuring that the returned results are empty. This approach ensures that only information that meets the standards is returned, ensuring the high quality and reliability of the retrieval results, and providing accurate and valuable content.

[0042] Step S102: Analyze the debate corpus data to include multiple tasks, and pre-process the data of each task separately to ensure that they meet the format requirements and specifications of the corresponding task fine-tuning data.

[0043] Disagreements between people on a topic may lead to a heated debate, where opposing parties exchange arguments on the topic to strengthen their positions. Inspired by the human debate process, compared with the mainstream social media corpus data in existing research, debate corpus data is exactly the data most relevant to the stance detection task that this case is looking for. The debate corpus data has the characteristics of wide coverage of topics, high structural quality, standardized style, rigorous logical structure, clear stance expression and less noise. In computational argumentation, argumentation is the process of supporting a certain point of view, which should include premises and conclusions. Argumentation is subjective and is a declarative summary and induction of the stance of a specific object. The process of generating conclusions from premises is similar to the process of stance detection. The two promote each other and enhance the model's understanding of controversial texts and stance reasoning ability. Argumentation analysis is the core of computational argumentation. The proponent supports the point of view, and the refutator tests the robustness of the argument, reveals logical loopholes and proposes counterexamples or alternative explanations. Generating arguments or refutations is the reverse process of stance detection. The two reinforce each other, allowing the model to examine controversial points from multiple angles and learn strategies for identifying and dealing with both sides of the debate, thereby improving the ability to distinguish different stances, evaluate the strength of evidence, and evaluate logical structure in complex texts.

[0044] Therefore, the debate corpus data contains four tasks: ① stance detection: judging the stance of the premise sentence on the target object. It is worth noting that for the stance detection task, the knowledge enhancement method in step S101 is used to obtain relevant knowledge of the text and the object, see Figure 2 Step a: External knowledge can be acquired through this stance detection task; ② Opinion extraction: extracting opinions, views or attitudes under a specific theme or topic from text and other information; ③ Argument generation: information and arguments that support or refute a specific stance or viewpoint, used to prove the validity and correctness of the argument / reveal the errors, loopholes or deficiencies of the other party's argument; ④ Relational reasoning: analyzing the information and context in the text to infer the relationship between the premise sentence and the assumption sentence.

[0045] In this case, we designed corresponding fine-tuning prompts for each task through prompt engineering, aiming to help the model improve its understanding ability, better learn and transfer the ability of each task, and thus improve the performance of the model's stance detection. Subsequently, 1,000 data were randomly selected from each task and preprocessed to ensure that the data met the format requirements of the fine-tuning data of each task.

[0046] Step S103: Use debate corpus data and external knowledge to perform multi-task fine-tuning on the open source large model, and use the fine-tuned model to perform stance reasoning.

[0047] During the model fine-tuning stage, LLAMA3-8b-Instruct and Mistral-7b-Instruct performed outstandingly in natural language processing tasks. Their 7b and 8b parameter scales have relatively low hardware requirements and operating costs, and are cost-effective. They also support frameworks such as TensorFlow and PyTorch, and users can easily obtain, deploy and adjust them. They also have rich documentation and community support to help users get started quickly. Therefore, after comprehensively considering performance, cost, and ease of use, this case selected these two open source large models for semi-parametric supervised fine-tuning based on LORA (Low-Rank Adaptation). Specifically, the open source large model was supervised for multi-task fine-tuning using debate corpus data, and finally the MDKA model was obtained by combining multi-task fine-tuning of debate data and external knowledge, see Figure 2 Step b.

[0048] In the stance reasoning stage, the knowledge enhancement method of step S101 is first used to retrieve the relevant knowledge of the text and objects in the test data. Then, the retrieved knowledge is combined with the text and objects, and the prompt engineering is performed according to the fine-tuning data format. For details, see Figure 2 Step c. Finally, the fine-tuned MDKA model is used to infer the stance of the text in the test data towards the target object, see Figure 2 Step d.

[0049] Corresponding to the above method, this embodiment also proposes a zero-sample stance detection system based on multi-task fine-tuning of argument data and knowledge enhancement, including a knowledge enhancement module, a multi-task prompt construction module of argument data and a fine-tuning reasoning module, wherein:

[0050] The knowledge enhancement module is used to use hint engineering to retrieve relevant knowledge between text and objects from an external knowledge base to perform knowledge enhancement on fine-tuning data and test data.

[0051] The debate data multi-task prompt construction module is used to analyze the debate corpus data containing multiple tasks and pre-process the data of each task separately.

[0052] The fine-tuning reasoning module is used to perform multi-task fine-tuning on the open source large model using debate corpus data and external knowledge, and use the fine-tuned model to perform position reasoning.

[0053] In order to verify the effectiveness of this solution, further explanation is given below in combination with experimental data.

[0054] (1) Experimental setup

[0055] 1.1) Fine-tune the data

[0056] The fine-tuning data used in this experiment is mainly based on the debate corpus dataset, supplemented by the natural language reasoning dataset, covering four tasks: stance detection, opinion extraction, argument / refutation generation, and relational reasoning.

[0057] The Webis dataset is derived from debatepedia.org and contains 465 controversial topics and 12,326 argument samples, covering multiple fields with high quality and diversity. As open source data, these semi-structured argument texts provide a wide range of topic coverage, high-quality structure and standardized corpus resources for research such as computational argumentation, and are of great significance for the study of persuasion strategies, position evolution and the role of language in debate.

[0058] The Change my view dataset comes from the / r / ChangeMyView community on Reddit, where users put forward their opinions and accept challenges. After successful persuasion, users will explicitly acknowledge the change of opinion, ensuring the authenticity and naturalness of the data. The data reflects rich interactive examples of position changes and refutation attempts, and provides accurate position change labels through direct user feedback, without the need for additional manual annotation, reducing subjective bias.

[0059] The MNLI dataset is a widely used dataset for natural language reasoning tasks, corresponding to relational reasoning tasks. In natural language reasoning tasks, given two sentences (premise and hypothesis), the model needs to judge whether the hypothesis can be inferred from the premise, which is divided into three categories: entailment, neutrality, and contradiction.

[0060] 1.2) Evaluation Data

[0061] This experiment uses two classic Twitter-based stance detection datasets, namely SemEval-16TASK6a and P-Stance. The SemEval-16 TASK6a (SemEval 2016task A) dataset aims to evaluate the performance of whether a text supports, opposes, or is neutral to a specific object. It contains 5 topics and is a dataset for 3-classification tasks. The P-Stance dataset focuses on the 2020 election cycle. It contains 3 topics and is a dataset for 2-classification tasks. This dataset contains two subtasks, which aim to determine whether the text supports or opposes a specific object. It should be noted that the samples marked as "none" in the P-Stance dataset have a low consistency problem, so this experiment adopts the same approach as previous studies and removes samples marked as "neither".

[0062] Table 1 Object and label statistics of SemEval 2016 task A and P-Stance datasets

[0063]

[0064] 1.3) Evaluation criteria

[0065] This experiment uses the macro average of the F1 score as the evaluation metric. For the three-classification task in SemEval 2016 task A, the following settings are used, that is, each sample is labeled as "support", "oppose" or "neutral". Therefore, the calculation formula of Favg is: avg =(F favor +F against +F neither ) / 3. For the binary classification task of P-Stance, each sample is labeled as “support” or “oppose”. Therefore, F avg The calculation formula is: avg =(F favor +F against ) / 2.

[0066] 1.4) Fine-tune the details

[0067] NVIDIA TESLA V100S 32G was used in all experiments. The AdamW optimizer was used for model fine-tuning, with the batch size set to 8 and the maximum text sequence length set to 1024. The model proposed in this paper was trained for 3 epochs, with a warmup step of 0.1 and a learning rate of 1e-5. For the knowledge enhancement module, Qwen-turbo was used. 1 The Qwen large model serves as an external knowledge base.

[0068] (2) Baseline Method / Comparison Model

[0069] In order to evaluate the effectiveness of the proposed model, a comprehensive evaluation was performed and compared with a series of established baselines. The following are the details of these baseline models:

[0070] Zero-shot based methods. BERT: pre-training deep bidirectional Transformer model, fine-tuning context-sensitive representation; TGA Net: combining text attention and graph neural network, extracting features across sentences to model text structure; BERT-GCN: using BERT to extract text features, combined with graph convolutional network to process dependencies, and integrating syntactic and semantic features; PT-HCL: hierarchical contrastive learning strategy enhances the model's ability to capture differences in texts with different positions.

[0071] Fine-tuning based methods. RoBERTa: Improves BERT training by removing the NSP task and adopting a dynamic masking strategy; BERTweet: Pre-trained specifically for social media texts such as Twitter; JointCL: Combining multi-task learning and contrastive learning; TarBK-BERT: Uses background knowledge to enhance stance detection capabilities for specific topics.

[0072] Methods based on large models. COLA uses three stages to collaboratively apply large language models: multi-dimensional text analysis, reasoning-enhanced debate, and stance conclusions to analyze text in a refined manner, identify potential stances, and make comprehensive judgments to accurately detect stances in texts. TR-Tweet+COT: The fusion of COT embedding strengthens the stance detection performance of RoBERTa, and uses LLMs to conduct in-depth multi-angle text analysis and argumentation to judge stances; ChatGPT: It is a dialogue generation model that can be indirectly applied to stance detection through its powerful understanding and generation capabilities; LLAMA and Mistral, as large-scale language models, can understand and generate diverse texts. Using prompt technology, they can guide the analysis of specific topic expressions to achieve stance recognition.

[0073] (3) Experimental results

[0074] Experiments on zero-shot stance detection were conducted on the SemEval 2016 task A and P-Stance datasets. The specific experimental results are shown in Table 2.

[0075] Table 2 Experimental results of zero-shot stance detection on SemEval 2016 task A and P-Stance datasets

[0076]

[0077] The results in Table 2 show that the MDKA framework proposed in this paper can significantly improve the performance of zero-shot stance detection tasks, and outperforms all baseline models on public datasets. It is worth noting that although the amount of fine-tuning data used by the model of this invention is small, with only 1,000 data for each task, the experimental results show that on the basis of a large model, a small amount of high-quality data is enough to provide sufficient information for the fine-tuning process, enabling the model to learn the professional capabilities of specific tasks.

[0078] Compared with the fine-tuning base models LLAMA3-8b-Instruct and Mistral-7b-Instruct, the present invention combines the MDKA framework with multi-task fine-tuning of argument data and knowledge enhancement technology, and has significant overall improvements on the SemEval 2016task A dataset and the P-Stance dataset. The results show that the model shows good generalization ability on social media data by using rigorous logic and less noise argument corpus data for multi-task fine-tuning, which enhances the model's understanding and stance reasoning ability. In addition, through multi-task fine-tuning of argument data and combining it with knowledge enhancement technology to retrieve relevant knowledge between text and objects from an external knowledge base, the lack of professional and domain-specific knowledge in the model can be more effectively solved, thereby significantly improving the effect of stance detection.

[0079] (4) Ablation experiment

[0080] In order to study the impact of each module and fine-tuning dataset in the design of the present invention, an ablation study was performed on the SemEval 2016task A dataset using LLAMA3-8b-Instruct as the fine-tuning base model to evaluate the performance of the MDKA model framework of the present invention when each module is removed. The first part is the ablation experiment of the model fine-tuning module and the knowledge enhancement module, and the second part is the ablation experiment of the fine-tuning dataset.

[0081] ① “w / o Fine-tuning” means no fine-tuning, and “w / o KA” means no knowledge augmentation technology;

[0082] ②“w / o C” means there is no Change My View dataset, “w / o M” means there is no MNLI dataset, and “w / o W” means there is no Webis dataset.

[0083] Table 3 Ablation experiment results

[0084]

[0085] In the first group of ablation experiments on model fine-tuning and knowledge enhancement modules, as shown in Table 3, relative to the MDKA model framework proposed in the present invention, removing the multi-task fine-tuning module results in the model lacking training to use relevant knowledge to reason about positions. At the same time, due to the lack of fine-tuning training for the specific task of position detection, the model's position reasoning and comprehension capabilities are weak, resulting in a 18.98% decrease in the F1 value. Removing the knowledge enhancement module results in the model lacking relevant knowledge between text and objects when reasoning about positions. Due to the lack of context, the model cannot understand the association between text and objects well, resulting in a 10.39% decrease in the F1 value. The above analysis shows that by using debate corpus data for multi-task fine-tuning, the model's comprehension ability and generalization can be significantly improved, enabling it to more effectively utilize the retrieved relevant knowledge. In the second group of ablation experiments on the fine-tuning dataset, it was found that no matter which dataset was removed, the F1 value dropped significantly, as shown in Table 3, further confirming the importance of each fine-tuning dataset.

[0086] like Figure 3 As shown in the figure, based on multi-task fine-tuning, combined with the relevant knowledge between text and objects, in long text tasks (such as the P-Stance dataset), due to its richer context, the improvement effect is relatively weak, and the F1 value is only improved by 4.06%. However, in short text tasks (such as the SemEval 2016task A dataset), due to the lack of context, the effect is significantly improved, and the F1 value is improved by 12.68%. This shows that knowledge enhancement by retrieving knowledge from external knowledge bases is of great significance in short text stance detection lacking context.

[0087] The present invention proposes an MDKA model framework, which combines supervised fine-tuning and knowledge enhancement technology based on a large model, aiming to enhance the model's stance detection ability in zero-shot scenarios. Zero-shot stance detection faces the challenges of scarce data annotation and complex tasks. The present invention retrieves text and object-related knowledge from an external knowledge base LLMs and integrates it into the data of the stance detection task, as a context-assisted model to learn stance features and reasoning capabilities. In addition, the present invention selects rigorous logic and low-noise debate corpus data as multi-task fine-tuning data, and uses an open source large model as a base model for fine-tuning to perform supervised multi-task fine-tuning. Experimental results show that the MDKA model framework performs well on public datasets for stance detection, demonstrating the great potential of model fine-tuning and knowledge enhancement in the field of zero-shot stance detection, and also verifies the effectiveness of using debate corpus data multi-task fine-tuning and external knowledge for zero-shot stance detection tasks. Since the large model shows strong general capabilities, the use of large models for data enhancement in the future, combined with fine-tuning solutions to solve a specific task, shows a promising development potential.

[0088] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0089] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A zero-shot stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data, characterized in that: include: Firstly, we use hint engineering to retrieve relevant knowledge between text and objects from an external knowledge base to enhance the knowledge of fine-tuning data and test data. Then, the debate corpus data contains multiple tasks, and the data of each task is preprocessed separately; Finally, we use debate corpus data and external knowledge to fine-tune the open source large model for multiple tasks, and use the fine-tuned model to perform position reasoning.

2. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 1 is characterized in that: Knowledge enhancement of fine-tuning data and test data includes: A large language model is selected as the external knowledge base, and knowledge related to texts and objects is retrieved from the external knowledge base by designing prompts, filling in texts and objects; if the external knowledge base returns relevant knowledge, this knowledge will serve as a supplement to the relevant knowledge in the fine-tuning data and test data.

3. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 2 is characterized in that: A filtering mechanism is used to filter the retrieved relevant knowledge. If the external knowledge base provides abnormal or incorrect information, this information will be filtered and the returned result will be empty.

4. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 1 is characterized in that: The debate corpus data contains four tasks: ①Stance detection: determine the stance of the premise sentence on the target object; the stance detection task acquires external knowledge; ② Opinion extraction: extracting opinions, views or attitudes on specific themes or topics from text information; ③ Argument generation: information and arguments that support or refute a specific position or viewpoint; ④ Relational reasoning: Analyze the information and context in the text to infer the relationship between the premise sentence and the hypothesis sentence.

5. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 1 is characterized in that: The data of each task are preprocessed and constructed into the format that meets the fine-tuning data requirements of the corresponding task.

6. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 1 is characterized in that: The open source large model uses LLAMA3-8b-Instruct and Mistral-7b-Instruct.

7. The zero-sample stance detection method based on multi-task fine-tuning and knowledge enhancement of debate data according to claim 1 is characterized in that: The stance reasoning process is as follows: first, the knowledge enhancement method is used to retrieve relevant knowledge about the text and objects in the test data; then, the retrieved relevant knowledge is combined with the text and objects, and prompt engineering is performed according to the fine-tuned data format; finally, the fine-tuned model is used to infer the stance of the text in the test data on the target object.

8. A zero-sample stance detection system based on multi-task fine-tuning and knowledge enhancement of debate data, characterized in that: It includes a knowledge enhancement module, a debate data multi-task prompt construction module, and a fine-tuning reasoning module, among which: The knowledge enhancement module is used to use hint engineering to retrieve relevant knowledge between text and objects from an external knowledge base to perform knowledge enhancement on fine-tuning data and test data; The debate data multi-task prompt construction module is used to analyze the debate corpus data containing multiple tasks and pre-process the data of each task separately; The fine-tuning reasoning module is used to perform multi-task fine-tuning on the open source large model using debate corpus data and external knowledge, and use the fine-tuned model to perform position reasoning.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images