Unified discussion mining method based on instruction learning and mining system thereof

Abstract

The invention belongs to the technical field of calculation discussion and discussion mining, and particularly relates to a unified discussion mining method based on instruction learning and a mining system thereof. The method comprises the following steps of: 1, uniformly modeling three types of AM as an instruction learning task, namely constructing a Unii-AM framework; 2, training an LLM-based generative model by using the Unii-AM framework constructed in the step 1; and 3, realizing a function of simultaneously processing various types of AM by using the LLM-based generative model in the step 2. The method is used for solving the problem that in the prior art, all three types of discussion mining and respective discussion mining subtasks cannot be processed at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of computational argumentation and argumentation mining, and specifically relates to a unified argumentation mining method based on instruction learning and a mining system thereof. Background Art

[0002] Argumentation mining (AM) aims to analyze the logical structure of argumentative texts. According to the logical flow structure of argumentative texts, AM can be divided into three types: tree type, generic type, and argument pair type. Existing methods usually deal with the three types of AM independently, ignoring that in real-life scenarios, the three types of AM are often mixed together and share the same context and background. This creates difficulties when applying existing methods to real-life scenarios and also causes potential performance losses. In this paper, a framework is proposed to unify the three types of AM into a self-guided learning task, called Uni-AM. To the best of our knowledge, Uni-AM is the first attempt to effectively solve all three types of AM on three subtasks in a unified manner. Uni-AM explicitly enhances the focus on information gain of different AM types and is highly scalable to adapt to a wide range of experimental results on three widely used datasets show the effectiveness of Uni-AM on all AM types and subtasks. In addition, it is found that joint learning of different AM types significantly improves the performance, which further verifies the necessity of unified modeling.

[0003] From predicting financial market trends to public relations, the analysis of argumentative texts provides valuable insights across a wide range of fields, showing the importance of automated argumentative text analysis. Argument mining is a text mining and understanding task that extracts and analyzes the logical structure of argumentative texts.

[0004] Given an argumentative text, such as a speech advocating a certain point of view or a post expressing a specific point of view, argument mining (AM) first identifies, extracts, and classifies all argument components (ACs), then organizes the structure of the argumentative text and classifies argument relations (ARs) between ACs. ACs are claims or arguments in the text (e.g., "euthanasia should be legalized"), and ARs represent the relations between ACs (e.g., support, attack).

[0005] According to the logical structure of argumentative text structure, AM can be divided into three categories: tree AM, generic AM and argument pair AM. The core difference between different types of AM lies in the structure of argument relations in the text.

[0006] For tree AM, argument relations form a tree structure, while in generic AM, argument relations form a directed acyclic graph (DAG), and in argument pair AM, argument relations form a bipartite graph.

[0007] In order to explain the differences and connections between different types of AM more intuitively, let’s take the social media scenario as an example.

[0008] In social media, a user may post a tweet with multiple claims and arguments, and the analysis of this tweet meets the characteristics of a generic AM. The comments of other users under the tweet naturally form a tree, which can be regarded as a tree AM. If two users exchange views and debate on social media, the analysis of their interaction process is an argument pair AM. Figure 1 An example of each of the three types of AM is presented, and the argumentation relationships corresponding to these examples are explained.

[0009] Many excellent attempts have been made in previous works. In these works, it is widely recognized that AM has the following three basic subtasks: Argument Component Type Classification (ACTC), which classifies the extracted arguments into categories (e.g., value, strategy). Argument Relation Identification (ARI), which determines whether two arguments form an argument relationship. Argument Relation Type Classification (ARTC), which classifies all identified inter-argument relations (e.g., derivation, evidence).

[0010] However, current methods are often task-dependent and difficult to migrate to other types of AM. As can be seen from the above examples, in real-world scenarios such as social media discussions and TV debates, different types of AM and different AM subtasks are not isolated. They are often mixed and share the same context and background knowledge. This close connection implies the need for the ability to handle multiple types of AM simultaneously, and also shows the potential information gain between different types of AM and AM subtasks. However, there is still a lack of a practical framework for unified modeling of these three AM types. In addition, most of the current work fails to fully utilize the powerful text modeling capabilities and rich world knowledge of LLM. These methods tend to use the hidden states of the model output to generate embeddings of the argument components, which is neither effective nor easy to train for current large language models. Summary of the invention

[0011] The present invention provides a unified argument mining method and a mining system based on instruction learning, so as to solve the problem that all three types of argument mining and their respective argument mining subtasks cannot be processed simultaneously in the prior art.

[0012] The present invention is achieved through the following technical solutions:

[0013] A unified argument mining method based on instruction learning, the unified argument mining method comprising the following steps:

[0014] Step 1: Unify the three types of AM into instruction learning tasks, that is, build the framework of Uni-AM;

[0015] Step 2: Use the Uni-AM framework built in step 1 to train the LLM-based generative model;

[0016] Step 3: Use the LLM-based generative model from step 2 to implement the capability of processing multiple types of AMs simultaneously.

[0017] Furthermore, the step 1 is specifically that the three types of AMs are tree AM, generic AM and argument pair AM, and these three types of AMs are constructed into an instruction shared between tasks to guide the model to understand the argumentative text.

[0018] Furthermore, n input queries are constructed to confirm the category of each AC and complete the ACTC subtask.

[0019] Furthermore, an input query is constructed to determine the type of relationship between any two argument components AC_i and AC_j to implement the ARI and ARTC subtasks.

[0020] Furthermore, for the ACTC task, a query is constructed for each AC as follows: instruction: Read the following paragraph and answer the question [argument text] input: [AC] The type of argument is [list of AC type sets].

[0021] Furthermore, for ARI and ARTC tasks, we consider the combination of any two ACs and use the query to solve ARI and ARTC.

[0022] Furthermore, for each AC pair, the query is constructed as follows: instruction: Read the following article and answer the questions; [argument text], input: What is the relationship between the arguments [AC1] and [AC2] [AR type set list]; If the model outputs no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered to be the result of ARTC.

[0023] A unified argument mining system based on instruction learning, the unified argument mining system uses the unified argument mining method based on instruction learning as described above, the unified argument mining system includes

[0024] Uni-AM framework building blocks for uniformly modeling the three types of AM as instruction learning tasks;

[0025] The training module trains the LLM-based generative model using the Uni-AM framework built by the Uni-AM framework building module;

[0026] The ability to handle multiple types of AM simultaneously is achieved using an LLM-based generative model.

[0027] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above method is implemented.

[0028] A computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0029] The beneficial effects of the present invention are:

[0030] The present invention combines the two queries of ARI and ARTC to greatly improve the computing performance.

[0031] The present invention can handle all three types of argument mining and their respective argument mining subtasks simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the algorithm principle of the present invention.

[0033] Figure 2 This is the effect diagram of Uni-AM of the present invention on the PE data set.

[0034] Figure 3 This is the effect diagram of Uni-AM of the present invention in CDCP data set.

[0035] Figure 4 This is the effect diagram of Uni-AM of the present invention on the RR data set.

[0036] Figure 5 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0038] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0039] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0040] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings of the present application specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0042] Embodiment 1

[0043] like Figure 1 As shown, a unified argument mining method based on instruction learning, the unified argument mining method comprises the following steps:

[0044] Step 1: Unify the three types of AM into instruction learning tasks, that is, build the framework of Uni-AM;

[0045] Step 2: Use the Uni-AM framework built in step 1 to train the LLM-based generative model; in which the argumentative text is regarded as an instruction and different types of AM subtasks are modeled as inputs;

[0046] Step 3: Use the LLM-based generative model from step 2 to implement the capability of processing multiple types of AMs simultaneously.

[0047] Furthermore, the step 1 is specifically that the three types of AMs are tree AM, generic AM and argument pair AM, and these three types of AMs are constructed into an instruction shared between tasks to guide the model to understand the argumentative text.

[0048] Furthermore, n input queries are constructed to confirm the category of each AC and complete the ACTC subtask.

[0049] Furthermore, an input query is constructed to determine the type of relationship between any two argument components AC_i and AC_j to implement the ARI and ARTC subtasks.

[0050] Furthermore, for the ACTC task, a query is constructed for each AC as follows: instruction: Read the following paragraph and answer the question [argument text] input: [AC] The type of argument is [list of AC type sets].

[0051] Furthermore, for ARI and ARTC tasks, we consider the combination of any two ACs and use the query to solve ARI and ARTC.

[0052] Furthermore, for each AC pair, the query is constructed as follows: instruction: Read the following article and answer the questions; [argument text], input: What is the relationship between the arguments [AC1] and [AC2] [AR type set list]; If the model outputs no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered to be the result of ARTC.

[0053] There are two considerations for modeling ARI and ARTC as the same question-answer pair; first, to reduce the computational time of the model; since the number of queries between argument pairs is the square of the number of ACs, combining ARI and ARTC into one query to compute long texts with a large number of ACs will greatly save computational overhead; second, both ARI and ARTC are pairwise relational learning processes between arguments, and their results should naturally have a large correlation with each other (for example, when the ARI result between two arguments is None, the ARTC result should also be None); therefore, combining the two queries will greatly improve computational performance based on experimental results.

[0054] Specifically, experiments are conducted on Uni-AM on PE, CDCP and RR datasets to analyze the impact of Uni-AM on tree, generic and argumentation on AM.

[0055] from Figure 2 In the paper, it is observed that Uni-AM achieves the best results in all subtasks. For the ACTC subtask, Uni-AM is 1.2 percentage points higher than the current best result in macro F1 socre. For the ARI subtask, Uni-AM is found to be 1.2 percentage points better than the SOTA in macro F1 and achieves better results in both Rel and No-Rel. For the ARTC subtask, Uni-AM is found to perform 1.7 percentage points better than the SOTA. The results demonstrate the effectiveness of Uni-AM, which is because the joint training method leads to effect gains between different AM subtasks.

[0056] from Figure 3In the paper, it is observed that Uni-AM outperforms SOTA results in all subtasks. For the ACTC subtask, it is 2.5% better than the current SOTA in the macro F1 metric. For the ARI subtask, it is 5.0% better than T5-Trans in the macro F1 score and achieves better results on both Rel and No-Rel metrics. For the ARTC subtask, Uni-AM is the first work to report ARTC task metrics on the CDCP dataset. This is because the unified modeling scheme of is highly scalable for all AM types and all AM subtasks.

[0057] from Figure 4 In the paper, it is observed that for the debate pair task, although only ARI testing was performed due to data annotation limitations, the experimental results still show an improvement of 0.8 percentage points compared to the current State-Of-Art MLMC of Macro-F1. This is because the instruction learning method and joint training framework can better understand the argument context, and thus perform well on the ARI task that requires deep semantic understanding.

[0058] In general, our experimental results show that some state-of-the-art models cannot handle the ARTC subtask, and some models cannot handle both tree-based AM and general AM at the same time. More importantly, except for the model proposed in this paper, no existing model can handle all three types of AM and their respective AM subtasks at the same time. This further demonstrates the scalability of our transformation framework.

[0059] Embodiment 2

[0060] This embodiment provides a unified argument mining system based on instruction learning, the unified argument mining system uses the unified argument mining method based on instruction learning as described in the first embodiment, and the unified argument mining system includes a Uni-AM framework building module and a training module;

[0061] Uni-AM framework building blocks for uniformly modeling the three types of AM as instruction learning tasks;

[0062] The training module uses the Uni-AM framework built by the Uni-AM framework building module to train the LLM-based generative model; in which the argumentative text is regarded as an instruction and different types of AM subtasks are modeled as inputs;

[0063] The ability to handle multiple types of AM simultaneously is achieved using an LLM-based generative model.

[0064] Furthermore, the Uni-AM framework building module is specifically composed of three types of AMs, namely tree AM, generic AM and argument pair AM. These three types of AMs are constructed into an instruction shared between tasks to guide the model to understand argumentative texts.

[0065] Furthermore, n input queries are constructed to confirm the category of each AC and complete the ACTC subtask.

[0066] Furthermore, an input query is constructed to determine the type of relationship between any two argument components AC_i and AC_j to implement the ARI and ARTC subtasks.

[0067] Furthermore, for the ACTC task, a query is constructed for each AC as follows: instruction: Read the following paragraph and answer the question [argument text] input: [AC] The type of argument is [list of AC type sets].

[0068] Furthermore, for ARI and ARTC tasks, we consider the combination of any two ACs and use the query to solve ARI and ARTC.

[0069] Furthermore, for each AC pair, the query is constructed as follows: instruction: Read the following article and answer the questions; [argument text], input: What is the relationship between the arguments [AC1] and [AC2] [AR type set list]; If the model outputs no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered to be the result of ARTC.

[0070] There are two considerations for modeling ARI and ARTC as the same question-answer pair; first, to reduce the computational time of the model; since the number of queries between argument pairs is the square of the number of ACs, combining ARI and ARTC into one query to compute long texts with a large number of ACs will greatly save computational overhead; second, both ARI and ARTC are pairwise relational learning processes between arguments, and their results should naturally have a large correlation with each other (for example, when the ARI result between two arguments is None, the ARTC result should also be None); therefore, combining the two queries will greatly improve computational performance based on experimental results.

[0071] As can be seen from the above, the embodiment of the present invention uniformly models the three types of AM as instruction learning tasks, trains an LLM-based generative model to jointly learn all these self-instruction inputs, and enables all AM subtasks to benefit from each other to achieve the function of processing multiple types of AMs at the same time. Experimental results show that the combination of two queries in this method can greatly improve the computing performance.

[0072] Embodiment 3

[0073] An embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory is used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected via a bus. Specifically, the processor implements any step in the first embodiment above by running the computer program stored in the memory.

[0074] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0075] The memory may include a read-only memory, a flash memory, and a random access memory, and provides instructions and data to the processor. A part or all of the memory may also include a nonvolatile random access memory.

[0076] As can be seen from the above, the electronic device provided by the embodiment of the present invention can implement the unified argument mining method as described in Example 1 by running a computer program, and obtain a new framework, called Uni-AM, which is used to uniformly model the three types of AM as instruction learning tasks. Train a LLM-based generative model to jointly learn all these self-instruction inputs so that all AM subtasks can benefit from each other. Combining the two queries should greatly improve the computing performance.

[0077] It should be understood that if the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0078] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in the present invention may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but will conform to the widest scope consistent with the principles and novel features disclosed in the present invention.

[0079] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0080] It should be noted that the methods and detailed examples provided in the above embodiments can be combined with the devices and equipment provided in the embodiments, and references can be made to each other, and no further details will be given.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0082] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0083] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A unified argument mining method based on instruction learning, characterized in that: The unified argument mining method comprises the following steps: Step 1: Unify the three types of AM into instruction learning tasks, that is, build the framework of Uni-AM; Step 2: Use the Uni-AM framework built in step 1 to train the LLM-based generative model; Step 3: Use the LLM-based generative model from step 2 to implement the capability of processing multiple types of AMs simultaneously.

2. The unified debate mining method according to claim 1, characterized in that: Specifically, the three types of AMs are tree AM, generic AM and argument pair AM, and these three types of AMs are used to construct an instruction shared between tasks to guide the model to understand argumentative texts.

3. The unified debate mining method according to claim 2, characterized in that: Construct n input queries to confirm the category of each AC and complete the ACTC subtask.

4. The unified debate mining method according to claim 3, characterized in that: Construct an input query to determine the type of relationship between any two argument components AC_i and AC_j to implement the ARI and ARTC subtasks.

5. The unified debate mining method according to claim 2, characterized in that: For the ACTC task, construct queries for each AC as follows: instruction: Read the following paragraph and answer the question [argument text] input: [AC] What is the type of argument? [a list of AC types].

6. The unified debate mining method according to claim 4, characterized in that: For ARI and ARTC tasks, we consider the combination of any two ACs and use the queries to solve ARI and ARTC.

7. The unified debate mining method according to claim 6, characterized in that: For each AC pair, the query is constructed as follows: instruction: Read the following article and answer the questions; [argument text], input: What is the relationship between [AC1] and [AC2]? [AR type set list]; If the model outputs no relationship, it is considered that there is no relationship between the two ACs, otherwise it is considered to be the result of ARTC.

8. A unified argument mining system based on instruction learning, characterized in that: The unified argument mining system uses the unified argument mining method based on instruction learning as described in claims 1-7, and the unified argument mining system includes Uni-AM framework building blocks for uniformly modeling the three types of AM as instruction learning tasks; The training module trains the LLM-based generative model using the Uni-AM framework built by the Uni-AM framework building module; The ability to handle multiple types of AM simultaneously is achieved using an LLM-based generative model.

9. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images