Unified argument mining method based on instruction learning and mining system thereof

Abstract

The present application belongs to the technical field of computational argumentation and argument mining, and particularly relates to a unified argument mining method based on instruction learning and a mining system thereof. Step 1: three types of AM are unified to be modeled as an instruction learning task, that is, a framework of Uni-AM is constructed; Step 2: a generation model based on LLM is trained using the framework of Uni-AM constructed in Step 1; Step 3: a function of simultaneously processing multiple types of AM is realized using the generation model based on LLM in Step 2. The present application is used to solve the problem that all three types of argument mining and their respective argument mining sub-tasks cannot be simultaneously processed in the prior art.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computational argumentation and argument mining, and particularly relates to a unified argument mining method based on instruction learning and an argument mining system thereof. BACKGROUND

[0002] Argument Mining (AM) aims to analyze the logical structure of argumentative texts. According to the logical flow structure of argumentative texts, AM can be divided into three types: tree, generic, and argument pair. Existing methods usually handle the three types of AM independently, ignoring the fact that in real-life scenarios, the three types of AM are often mixed together and share the same context and background. This causes difficulties when applying existing methods to real-life scenarios and also causes potential performance loss. In the present application, a framework is proposed to unify the three types of AM into a self-guided learning task, called Uni-AM. To the best of our knowledge, Uni-AM is the first attempt to effectively solve all three types of AM on three sub-tasks in a unified way. Uni-AM explicitly enhances the focus on information gain for different AM types and has high scalability to adapt to a wide range of experiments on three widely used datasets. Experimental results show the effectiveness of Uni-AM on all AM types and sub-tasks. In addition, it is also found that joint learning of different AM types significantly improves performance, which further verifies the necessity of unified modeling.

[0003] From predicting financial market trends to public relations, the analysis of argumentative texts provides valuable insights across a wide range of fields, demonstrating the importance of automatic argumentative text analysis. Argument Mining is a text mining and understanding task that extracts and analyzes the logical structure of argumentative texts.

[0004] Given an argumentative text, such as a speech arguing for a certain viewpoint or a post expressing a particular opinion, argument mining (AM) first identifies, extracts, and classifies all argument components (ACs), then organizes the structure of the argumentative text and classifies the argument relations (ARs) between ACs. ACs are claims or points in the text (e.g., "Euthanasia should be legalized"), and ARs represent the relationships between ACs (e.g., support, attack).

[0005] According to the logical structure of argumentative text structure, AM can be divided into three categories: tree AM, generic AM, and argument pair AM. The core difference between different types of AM lies in the structure of argument relations in the text.

[0006] For tree AM, argument relations form a tree structure, while in generic AM, argument relations form a directed acyclic graph (DAG), and in argument pair AM, argument relations form a bipartite graph.

[0007] To explain the differences and connections between different types of AM more intuitively, take the social media scenario as an example.

[0008] In social media, a user can post a tweet with multiple claims and arguments, and the analysis of this tweet fits the characteristics of generic AM. The comments of other users under this tweet naturally form a tree, which can be regarded as a tree AM. If two users exchange views and debate on social media, the analysis of their interaction process is an argumentation pair AM. Figure 1 An example of three types of AM is shown, and the argumentation relationship corresponding to these examples is illustrated.

[0009] Previous work has made many excellent attempts. In these works, it is widely recognized that AM has the following three basic sub-tasks: argument component type classification (ACTC), which classifies the extracted arguments into categories (e.g., value, strategy). Argumentation relationship identification (ARI), determine whether two arguments constitute an argumentation relationship. Argumentation relationship type classification (ARTC), classify all identified inter-argument relationships (e.g., inference, evidence).

[0010] However, the current methods are usually task-specific and difficult to migrate to other types of AM. As can be seen from the above examples, in real-world scenarios such as social media discussions and television debates, different types of AM and different AM sub-tasks are not isolated. They tend to be mixed and share the same background and background knowledge. This close relationship means the need for the ability to handle multiple types of AM at the same time, and also shows the potential information gain between different types of AM and AM sub-tasks. However, there is still a lack of practical framework for unified modeling of the three types of AM. In addition, most of the current work cannot fully utilize the powerful text modeling capabilities and rich world knowledge of LLM. These methods tend to use the hidden state output by the model to generate the embedding of the argument component, which is neither efficient nor easy to train for current large language models. SUMMARY

[0011] The present application provides a unified argument mining method based on instruction learning and a unified argument mining system based on instruction learning to solve the problem that the prior art cannot simultaneously handle all three types of argument mining and their respective argument mining sub-tasks.

[0012] The present application is implemented by the following technical solutions:

[0013] A unified argument mining method based on instruction learning, the unified argument mining method comprising the following steps:

[0014] Step 1: unify the three types of AM into an instruction learning task, i.e., build a framework for Uni-AM;

[0015] Step 2: Train the LLM-based generative model using the framework constructed in Step 1 for Uni-AM;

[0016] Step 3: Implement the function of simultaneously processing multiple types of AM using the LLM-based generative model in Step 2.

[0017] Further, the step 1 is specifically, three types of AM are tree AM, generic AM and argument pair AM, and an instruction shared between tasks is constructed for these three types of AM to guide the model to understand argumentative texts.

[0018] Further, an n-input query is constructed to confirm the category of each AC, and the ACTC sub-task is completed.

[0019] Further, an input query is constructed to determine the relationship type between any two argument components AC_i and AC_j, to implement the ARI and ARTC sub-tasks.

[0020] Further, for the ACTC task, a query is constructed for each AC, as follows: instruction: read the following passage and answer the question [argument text] input: [AC] the type of argument is [AC type set list].

[0021] Further, for the ARI and ARTC tasks, consider the combination of any two ACs and use the query to solve ARI and ARTC.

[0022] Further, for each AC pair, the query is constructed as follows: instruction: read the following article and answer the question; [argument text], input: what is the relationship between arguments [AC1] and [AC2] [AR type set list]; if the model output has no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered as the result of ARTC.

[0023] A unified argument mining system based on instruction learning, which uses the unified argument mining method based on instruction learning as described above, the unified argument mining system comprises

[0024] A Uni-AM framework construction module for modeling three types of AM as instruction learning tasks;

[0025] A training module for training an LLM-based generative model using the Uni-AM framework constructed by the Uni-AM framework construction module;

[0026] Using the LLM-based generative model to implement the function of simultaneously processing multiple types of AM.

[0027] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method as described above when executing the computer program.

[0028] A computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described above.

[0029] The beneficial effects of the present application are:

[0030] The combination of ARI and ARTC queries in the present application can greatly improve the computing performance.

[0031] The present application can simultaneously process all three types of argument mining and their respective argument mining sub-tasks. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is the schematic diagram of the algorithm principle of the present application.

[0033] Figure 2 is the effect diagram of Uni-AM in the PE data set of the present application.

[0034] Figure 3 is the effect diagram of Uni-AM in the CDCP data set of the present application.

[0035] Figure 4 is the effect diagram of Uni-AM in the RR data set of the present application.

[0036] Figure 5 is the flow chart of the method of the present application. DETAILED DESCRIPTION

[0037] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons having ordinary skill in the art will readily recognize that embodiments of the application can be practiced without some or all of the specific details set forth herein. In other instances, well known structures have not been described in detail in order to avoid obscuring the application.

[0038] It should be understood that the term "comprising" as used in this specification and in the following claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0039] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting, as the scope of the present application will be defined by the claims. As used in this specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise.

[0040] The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the drawings in the present application specification. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0041] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0042] Embodiment one

[0043] As Figure 1 shown, a unified argument mining method based on instruction learning, the unified argument mining method comprises the following steps:

[0044] Step 1: model three types of AM as instruction learning tasks, i.e. build the framework of Uni-AM;

[0045] Step 2: train the LLM-based generation model using the framework of Uni-AM built in step 1; wherein the argumentative text is regarded as an instruction, and different types of AM subtasks are modeled as inputs;

[0046] Step 3: use the LLM-based generation model of step 2 to implement the function of simultaneously processing multiple types of AM.

[0047] Further, step 1 is specifically, three types of AM are tree AM, generic AM and argument pair AM, and an instruction shared among tasks is constructed for the three types of AM to guide the model to understand the argumentative text.

[0048] Further, an n-input query is constructed to confirm the category of each AC, and the ACTC subtask is completed.

[0049] Further, an input query is constructed to determine the relationship type between any two argument components AC_i and AC_j, to implement the ARI and ARTC subtasks.

[0050] Further, for the ACTC task, construct a query for each AC as follows: instruction: read the following passage and answer the question [argument text] input: [AC] the type of argument is [AC type set list].

[0051] Further, for the ARI and ARTC tasks, consider the combination of any two ACs and use the query to solve ARI and ARTC.

[0052] Further, for each AC pair, the query is constructed as follows: instruction read the following article and answer the question; [argument text], input: what is the relationship between [AC1] and [AC2] [AR type set list]; if the model output is no relationship, consider that there is no relationship between the two ACs, otherwise consider it as the result of ARTC.

[0053] There are two considerations for modeling ARI and ARTC as the same question-answer pair; one is to reduce the computing time of the model; since the number of queries between argument pairs is the square of the number of ACs, combining ARI and ARTC into one query to calculate long texts with a large number of ACs will greatly save computing overhead; second, ARI and ARTC are both pair-wise relationship learning processes between arguments, and their results should naturally have a great relevance to each other (for example, when the ARI result between two arguments is None, the ARTC result should also be None); Therefore, according to the experimental results, combining the two queries will greatly improve the computing performance.

[0054] Specifically, Uni-AM was experimented on PE, CDCP and RR datasets to analyze the impact of Uni-AM on tree, generic and argument pair AMs.

[0055] From Figure 2 , it is observed that Uni-AM achieves the best results in all subtasks. For the ACTC subtask, Uni-AM is 1.2 percentage points higher than the current best result in macro F1 socre. For the ARI subtask, it is found that Uni-AM is 1.2 percentage points better than SOTA in macro F1, and better results are achieved in Rel and No-Rel. For the ARTC subtask, it is found that the performance of Uni-AM is 1.7 percentage points better than SOTA. The results prove the effectiveness of Uni-AM, which is because the joint training method leads to the effect gain between different AM subtasks.

[0056] From Figure 3In the experiments, it is observed that Uni-AM outperforms the SOTA results in all sub-tasks. For the ACTC sub-task, it improves the current SOTA by 2.5% on the macro F1 metric. For the ARI sub-task, it outperforms T5-Trans by 5.0% on the macro F1 score, and achieves better results on both Rel and No-Rel metrics. For the ARTC sub-task, Uni-AM is the first work that reports the ARTC task metrics on the CDCP dataset. This is because the unified modeling scheme is highly scalable for all AM types and all AM sub-tasks.

[0057] From Figure 4 In the experiments, it is observed that for the argumentation pair task, although only ARI test is conducted due to data annotation limitation, the experimental results still show an improvement of 0.8 percentage points compared with the current Macro-F1 State-Of-Art MLMC. This is because the instruction learning method and the joint training framework can better understand the argumentation context, thus performing well on the ARI task which requires in-depth semantic understanding.

[0058] Overall, the experimental results demonstrate that some of the most advanced models cannot handle the ARTC sub-task, and some models cannot handle both tree-based AM and general AM at the same time. More importantly, no existing model can handle all three types of AM and their respective AM sub-tasks at the same time except for the model proposed in the present invention. This further proves the scalability of the conversion framework.

[0059] Embodiment Two

[0060] The embodiment provides a unified argument mining system based on instruction learning, which uses the unified argument mining method based on instruction learning as described in Embodiment One, and the unified argument mining system comprises a Uni-AM framework modeling module and a training module.

[0061] The Uni-AM framework modeling module is used to unify the three types of AM into an instruction learning task.

[0062] The training module trains the LLM-based generative model using the Uni-AM framework constructed by the Uni-AM framework modeling module; wherein the argumentative text is regarded as an instruction, and different types of AM sub-tasks are modeled as inputs.

[0063] The LLM-based generative model is used to realize the function of simultaneously handling multiple types of AM.

[0064] Further, the Uni-AM framework module is specifically three types of AMs, namely tree AM, generic AM and argument pair AM, and constructs an instruction shared between tasks to guide the model to understand argumentative texts.

[0065] Further, an n-input query is constructed to determine the category of each AC, and the ACTC subtask is completed.

[0066] Further, an input query is constructed to determine the relationship type between any two argument components AC_i and AC_j, to realize the ARI and ARTC subtasks.

[0067] Further, for the ACTC task, a query is constructed for each AC, as follows: instruction: read the following passage and answer the question [argument text] input: [AC type set list] the type of argument is [AC type set list].

[0068] Further, for the ARI and ARTC tasks, consider the combination of any two ACs, and use the query to solve ARI and ARTC.

[0069] Further, for each AC pair, the query is constructed as follows: instruction: read the following article and answer the question; [argument text], input: what is the relationship between arguments [AC1] and [AC2] [AR type set list]; if the model output has no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered as the result of ARTC.

[0070] There are two considerations for modeling ARI and ARTC as the same question-answer pair; one is to reduce the computing time of the model; since the number of queries between argument pairs is the square of the number of ACs, combining ARI and ARTC into one query to calculate long texts with a large number of ACs will greatly save computing overhead; second, ARI and ARTC are both pair-wise relationship learning processes between arguments, and their results should naturally have a great relevance to each other (for example, when the ARI result between two arguments is None, the ARTC result should also be None); therefore, according to the experimental results, combining the two queries will greatly improve the computing performance.

[0071] As can be seen from the above, the embodiment of the application unifies the three types of AMs into instruction learning tasks, trains a LLM-based generative model to learn all these self-instruction inputs, enables all AM subtasks to benefit from each other, and realizes the function of simultaneously processing multiple types of AMs. Experimental results show that the combination of the two queries of the method greatly improves the computing performance.

[0072] Embodiment three

[0073] The electronic device provided by the embodiment of the present application comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the memory is used for storing software programs and modules, and the processor executes various function applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected through a bus. Specifically, the processor realizes any step in the embodiment one by running the computer program stored in the memory.

[0074] It should be understood that, in the embodiment of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0075] The memory can include read-only memory, flash memory and random access memory, and provide instructions and data for the processor. Part or all of the memory can also include non-volatile random access memory.

[0076] As can be seen from the above, the electronic device provided by the embodiment of the present application can realize the unified argument mining method as described in the embodiment one by running the computer program, obtain a new framework called Uni-AM, and model three types of AM as instruction learning tasks. A LLM-based generation model is trained to jointly learn all these self-instruction inputs, so that all AM subtasks can benefit from each other. Combining two queries should greatly improve the computing performance.

[0077] It should be understood that the above-mentioned integrated modules / units, if implemented in the form of software function units and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by computer programs instructing related hardware, and the above-mentioned computer programs can be stored in a computer readable storage medium. The computer programs can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer programs include computer program codes, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the above-mentioned computer program codes, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the contents included in the above-mentioned computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0078] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the present application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown in the present application, but will conform to the widest scope consistent with the principles and novel features disclosed in the present application.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is taken as an example, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above-mentioned device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software function unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the above-mentioned method embodiments, which will not be repeated here.

[0080] It should be noted that the method and details provided by the above embodiments can be combined with the apparatus and device provided by the embodiments, and mutual reference is not repeated.

[0081] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0082] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the above-described apparatus / device embodiments are merely illustrative, and the division of the above-described modules or units is merely a logical function division, and an actual implementation can be divided into another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0083] The above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A unified argument mining method based on instruction learning, characterized by, The unified argument mining method comprises the following steps: Step 1: unify three types of AMs into a self-instruction learning task, and build an instruction shared among tasks to guide the model to understand the argumentative text, that is, form a framework of Uni-AM, the three types of AMs being a tree AM, a generic AM and an argument pair AM; Step 2: train an LLM-based generation model using the framework of Uni-AM built in step 1; Step 3: use the LLM-based generation model in step 2 to realize the function of simultaneously processing multiple types of AMs; n input queries are constructed to confirm the category of each AC, and the ACTC subtask is completed; An input query is constructed to determine the relationship type between any two argument components AC_i and AC_j, to realize the ARI and ARTC subtasks; For the ACTC task, a query is constructed for each AC, as follows: instruction: read the following passage and answer the question [argument text] input: [AC] the type of argument is [AC type set list]; For the ARI and ARTC tasks, consider the combination of any two ACs, and use the query to solve ARI and ARTC; For each AC pair, the query is constructed as follows: instruction: read the following article and answer the question; [argument text], input: what is the relationship between arguments [AC1] and [AC2] [AR type set list]; if the model output has no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered as the result of ARTC.

2. A unified argument mining system based on instruction learning, characterized by, The unified argument mining system uses the unified argument mining method based on instruction learning according to claim 1, and the unified argument mining system comprises A Uni-AM framework modeling module, which unifies three types of AMs into a self-instruction learning task, and builds an instruction shared among tasks to guide the model to understand the argumentative text, the three types of AMs being a tree AM, a generic AM and an argument pair AM; A training module, which trains an LLM-based generation model using the Uni-AM framework built by the Uni-AM framework modeling module; Use the LLM-based generation model to realize the function of simultaneously processing multiple types of AMs; n input queries are constructed to confirm the category of each AC, and the ACTC subtask is completed; An input query is constructed to determine the relationship type between any two argument components AC_i and AC_j, to realize the ARI and ARTC subtasks; For the ACTC task, a query is constructed for each AC, as follows: instruction: read the following passage and answer the question [argument text] input: [AC] the type of argument is [AC type set list]; For the ARI and ARTC tasks, consider the combination of any two ACs, and use the query to solve ARI and ARTC; For each AC pair, the query is constructed as follows: instruction: read the following article and answer the question; [argument text], input: what is the relationship between arguments [AC1] and [AC2] [AR type set list]; if the model output has no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered as the result of ARTC. For each AC pair, the query is constructed as follows: instruction read the following article and answer the question; [argument text], input: what is the relationship between [AC1] and [AC2] [ARTC list]; if the model output has no relationship, consider that there is no relationship between the two ACs, otherwise consider it as the result of ARTC.

3. A computer device, comprising: A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable into the processor, the processor implementing the method according to any one of claims 1 when executing the computer program.

4. A computer-readable storage medium, characterized in that, A computer program product comprising a memory, a processor and a computer program stored on the memory and loadable into the processor, the processor implementing the method according to any one of claims 1 when executing the computer program.

Images