Methods for evaluating the effectiveness of learners' argumentation process under critical thinking training models

Abstract

This invention presents a method for evaluating the effectiveness of learners' argumentation process under a critical thinking training model. It involves the fields of natural language processing and knowledge graph technology, aiming to address the key problem of the difficulty in quantifying and tracking the argumentation process in traditional thinking assessments. The method involves extracting knowledge triples from argumentation texts to construct a knowledge graph, storing and visualizing the knowledge graph using a graph database to obtain the argumentation knowledge graph; generating existing argumentation paths to obtain a set of reference argumentation paths; determining the maximum intersection path between observed and reference argumentation paths; calculating the evaluation value of the maximum intersection path; and calculating the evaluation value of the critical thinking argumentation effectiveness. By establishing a bidirectional representation mechanism between observed and reference argumentation paths, and utilizing path intersection discovery and path evaluation calculation, this invention ultimately achieves refined tracking and quantitative evaluation of learners' argumentation process, providing data support for teaching interventions and thus promoting the systematic development of critical thinking skills.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and knowledge graph technology, specifically to a method for evaluating the effectiveness of learners' argumentation process under a critical thinking training model. Background Technology

[0002] Because of the implicit and complex nature of cognitive development, traditional assessment methods struggle to objectively and systematically measure the argumentation process of critical thinking. For example, existing technologies (such as keyword extraction and simple scoring) can only perform surface-level text analysis, failing to transform the argument content into structured entity-relational triples, making it difficult to accurately capture the logical structure. Traditional tools (such as the CCTDI scale) only provide general dimensional scores, unable to identify specific logical flaws in the argumentation process (such as insufficient premises or breaks in reasoning), making it difficult to provide precise guidance for improvement. Furthermore, existing methods rely on static rules or fixed scoring standards, making it difficult to adapt to dynamic changes in different disciplines or argumentation styles, and lacking the ability to automatically match standard reference paths.

[0003] Therefore, this invention provides a method for evaluating the effectiveness of learners' argumentation process under a critical thinking training model. Summary of the Invention

[0004] In order to address the key problem of the difficulty in objectively representing and quantifying the thinking process in the current teaching evaluation system, this invention proposes a method for evaluating the effectiveness of learners' argumentation process under a critical thinking training model.

[0005] The method for evaluating the effectiveness of learners' argumentation process under the critical thinking training model is implemented through the following steps:

[0006] Step 1: Extract knowledge triples from the argument text to construct a knowledge graph. Store and visualize the knowledge graph using a graph database to obtain the argument knowledge graph.

[0007] Step 2: Based on the argumentation knowledge graph obtained in Step 1, generate the argumentation paths that have occurred and obtain a set of reference argumentation paths;

[0008] Step 3: Determine the maximum intersection path between the observational argumentation path and the reference argumentation path;

[0009] Step 4: Calculate the evaluation value of the maximum intersection path, and calculate the evaluation value of the critical thinking argumentation effect based on the evaluation value.

[0010] Further, in step one, natural language processing tools are used to extract knowledge triples from the learner's argument text, and the set of triples TR = {(h, r, t)} is output, where h represents the head entity, r represents the relation, and t represents the tail entity; the entities in the triples are used as nodes, and the relations are used as edges to construct a directed graph. Based on the directed graph, a graph database is used to store and visualize the knowledge graph, and the argument knowledge graph CKG = (V, E) is output, where V is the set of nodes and E is the set of edges.

[0011] Furthermore, the specific process of step two is as follows:

[0012] Step 2.1 Initialize the observation and verification path set P o and temporary path list Tp=TR;

[0013] Step 22: Check if the temporary path list Tp is not empty. If it is, retrieve a path p from Tp. c Set the path p c The first node is Hp c The tail node is Tp c The set of remaining paths is Tp′=Tp-{p c If Tp is not updated, then Tp = Tp', and proceed to steps two and three; otherwise, proceed to step three.

[0014] Steps 2 and 3: Check if TR is not empty. If it is, take a triplet tr from TR. c Set the triple tr c The first node is Htr c The tail node is Ttr c The set of remaining triples is TR′=TR-{tr c If TR is not updated, TR = TR′ and proceed to step two or four; otherwise, return to step two or two.

[0015] Step 24: If pc == tr c or tr c ∈p c If not, proceed to steps two and three; otherwise, proceed to step two and five.

[0016] Step 25: If Htr c ==Tp c Then for path p c To expand, i.e.: p c =p c +tr c And update path P in Tp c If not, proceed to step 26; otherwise, proceed to step 23.

[0017] Step 26: If Ttr c ==Hpc Then for path P c To expand, i.e.: p c =tr c +p c And update path P in Tp c ; otherwise, proceed to step 27;

[0018] Step 27: If Htr c ≠Tp c or Ttr c ≠Hp c Then P o =TP, proceed to step three; otherwise, proceed to step two.

[0019] Step 3: Obtain the set of reference argumentation paths P a ;

[0020] Furthermore, the specific process of step three is as follows:

[0021] Step 3.1 Initialize the intersection path set P c ;

[0022] Step 32, if P a Not empty, from P a Extract an observational proof path p from the data. i The set of remaining paths is P′ a =P a -{p i}, Update P a =P′ a Proceed to step three; if P a If empty, proceed to step four;

[0023] Step 33, if P o Not empty, from P o Extract an observational proof path p from the data. j The set of remaining paths is P′ o =P o -{p j}, Update P o =P′ o ; Perform steps three and four; If P o If empty, proceed to step three two;

[0024] Steps 3 and 4: Obtain the observation and verification path p i and reference argument path p j Maximum intersection path p ∩ =p i ∩p j ,

[0025] Further, in step four, the evaluation value of the maximum intersection path is calculated. And return to step three two; evaluation value It can be expressed as follows:

[0026] Where, L ∩ L is the number of nodes in the maximum intersection path. i To observe the number of nodes in the argument path, L j The number of nodes in the reference argument path.

[0027] Furthermore, the number L of nodes in the maximum intersection path ∩ =|p ∩ |;

[0028] The number of nodes L in the observation and demonstration path i =|p i |;

[0029] The number of nodes L in the reference argument path j =|p j |

[0030] Furthermore, the evaluation value E of the critical thinking argumentation effect is calculated and expressed by the following formula:

[0031]

[0032] The present invention also provides a learner argumentation process effectiveness evaluation system, the system including a memory, a processor, and a learner argumentation process effectiveness evaluation program stored in the memory and running on the processor under a critical thinking training mode. When the critical thinking learning mode effectiveness evaluation program is executed by the processor, it implements the steps of the learner argumentation process effectiveness evaluation method under the critical thinking training mode.

[0033] The beneficial effects of this invention: The evaluation method described in this invention, combining knowledge graph modeling and dynamic path matching, achieves a breakthrough in critical thinking assessment. By transforming traditional subjective scoring into quantifiable calculations, it not only objectively assesses the quality of thinking but also accurately diagnoses specific flaws, such as logical gaps or premise errors. Utilizing natural language processing and automated path comparison technology, evaluation efficiency is significantly improved. Furthermore, the evaluation process is presented through an intuitive, visualized knowledge graph, allowing users to clearly understand the logical basis behind the scoring. This intelligent evaluation method ensures both the accuracy of the results and the transparency of the process, providing new technological support for cultivating critical thinking.

[0034] The method described in this invention establishes a two-way representation mechanism between the observed argumentation path and the reference argumentation path, and uses path intersection discovery and path evaluation calculation to achieve refined tracking and quantitative evaluation of the learner's argumentation process.

[0035] This invention transforms textual arguments into entity-relation triples through natural language processing and constructs a visualized knowledge graph, achieving a structured digital mapping of the thought process (existing technologies often remain at the level of keyword extraction or simple scoring). Its innovative dynamic path matching algorithm progressively compares learners' argumentation paths with standard reference paths, employing a critical thinking argumentation effectiveness evaluation method to achieve objective assessment (traditional methods rely on manual qualitative evaluation). Compared to existing technologies, this method innovatively possesses process diagnostic capabilities, meaning it can not only identify the correctness of conclusions but also pinpoint specific flaws in the logical chain. Existing tools (such as the CCTDI scale) can only provide general dimensional scores and cannot achieve refined analysis of the argumentation process. This evaluation method can not only effectively capture the logical coherence and depth of thought in the argumentation process but also provide data support for teaching interventions, thereby promoting the systematic development of critical thinking skills. Attached Figure Description

[0036] Figure 1 This is a simplified flowchart of the learner argumentation process effectiveness evaluation method under the critical thinking training model described in this invention.

[0037] Figure 2 This is a detailed flowchart of the learner argumentation process effectiveness evaluation method under the critical thinking training model described in this invention;

[0038] Figure 3 The effect diagram of the evaluation using the method of the present invention (basic demonstration);

[0039] Figure 4 The results of the evaluation using the method of this invention are shown in the diagram (multi-angle analysis);

[0040] Figure 5 The diagram shows the effect of evaluating the method of this invention (systematic thinking). Detailed Implementation

[0041] Specific Implementation Method 1: Combination Figures 1 to 5 This implementation method describes a method for evaluating the effectiveness of learners' argumentation process under a critical thinking training model. This method is achieved through the following steps:

[0042] Step 1: Use Natural Language Processing (NLP) tools to extract knowledge triples from the learner's argument text, organize the extracted entities and relations into triples (h, r, t), and output the set of triples TR = {(h, r, t)}, where h represents the head entity, r represents the relation, and t represents the tail entity;

[0043] Step 2: Construct a knowledge graph from the extracted triples. Entities within the triples are treated as nodes, and relations as edges, forming a directed graph. Store and visualize the knowledge graph using a graph database. The output is an argument knowledge graph CKG = (v, E), where V is the set of nodes and E is the set of edges.

[0044] Step 3: Generate the existing argument paths and obtain the set of reference argument paths P. a The specific process is as follows:

[0045] Step 31: Initialize the observation and verification path set P o ={}, initialize the temporary path list Tp=TR;

[0046] Step 32: If Tp is not empty, extract a path p from Tp. c p of the path c The first node is Hp c The tail node is Tp c The remainder set is denoted as Tp′=Tp-{p c}, update Tp = Tp′, and proceed to step 33; if Tp is empty, proceed to step 4;

[0047] Step 33: If TR is not empty, take a triplet tr from TR. c triple tr c The first node is Htr c The tail node is Ttr c The remainder set is denoted as TR′=TR-{tr c}, update TR = TR′, and proceed to step 34; if TR is empty, proceed to step 32;

[0048] Step 34: If tr c ==p c Or tr c ∈p c If yes, proceed to step 33; otherwise, proceed to step 35.

[0049] Step 35: If Htr c ==Tp c Then the extended path p c =p c +tr c And update path p in Tp cIf the above steps are executed, proceed to step 36; otherwise, proceed to step 33.

[0050] Step 36: If Ttr c ==Hp c Then the extended path p c =tr c +p c And update path p in Tp c If the above steps are executed, proceed to step 37; otherwise, proceed to step 33.

[0051] Step 37: If Htr c ≠Tp c or Ttr c ≠Hp c , P o =TP, proceed to step 4; otherwise, proceed to step 32.

[0052] Step 4: Process the reference answer argument text according to Steps 1 to 3 to obtain the reference argument path set P. a ; Perform step 5;

[0053] Step 5: For Find the observational proof path p i and reference argument path p j The maximum intersection path includes the following steps:

[0054] Step 51: Initialize the intersection path set P c ={};

[0055] Step 52: If P a Not empty, from P a Extract an observational proof path p from the data. i The remainder set is denoted as P′. a =P a -{p i}, Update P a =P′ a ; Perform step 53; If P a If empty, proceed to step 6;

[0056] Step 53: If P o Not empty, from P o Extract an observational proof path p from the data. j The set of remaining paths is P′ o =P o -{p j}, update p o =P′ o ;Execute step 54;If p o If empty, return to step 52;

[0057] Step 54: Find the observational proof path p i and reference argument path p j Maximum intersection path p ∩ =p i ∩p j .

[0058] Step 55: Calculate the number of nodes L in the maximum intersection path. ∩ =|p ∩ |;

[0059] Step 56: Calculate the number of nodes L in the observation and verification path. i =|p i |;

[0060] Step 57: Calculate the number of nodes L in the reference argument path. j =|p j |;

[0061] Step 58: Calculate the evaluation value of the maximum intersection path. Perform step 52;

[0062] Step 6: Calculate the evaluation value E for the effectiveness of critical thinking argumentation;

[0063] like Figures 3 to 5 As shown, the evaluation method described in this embodiment uses an improved path matching algorithm to achieve a multi-dimensional comparative analysis of the learner's argumentation path and the reference model. Figure 3 , Figure 4 and Figure 5 The quantitative evaluation value E of critical thinking argumentation effectiveness was calculated, thus providing an objective assessment of learners' critical thinking development at different stages and realizing an innovative transformation from traditional subjective experience judgment to scientific quantitative analysis.

[0064] Specific Implementation Method Two: This implementation method provides a learner argumentation process effectiveness evaluation system. The system includes a memory, a processor, and a learner argumentation process effectiveness evaluation program stored in the memory and executable on the processor under a critical thinking training mode. When the processor executes the effectiveness evaluation program of the critical thinking learning mode, it implements the steps of the learner argumentation process effectiveness evaluation method under a critical thinking training mode as described in Specific Implementation Method One.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for evaluating the effectiveness of learners' argumentation process based on a critical thinking training model, characterized by: This method is implemented by the following steps: Step 1: Extract knowledge triples from the argument text to construct a knowledge graph. Store and visualize the knowledge graph using a graph database to obtain the argument knowledge graph. Step 2: Based on the argumentation knowledge graph obtained in Step 1, generate the existing argumentation paths and obtain a set of reference argumentation paths; the specific process is as follows: Step 2.1 Initialize the observation and verification path set P o and temporary path list Tp=TR; Step 22: Check if the temporary path list Tp is not empty. If it is, retrieve a path p from Tp. c Set the path p c The first node is Hp c The tail node is Tp c The set of remaining paths is Tp' = Tp - {p} c }, update Tp = Tp', and proceed to steps two and three; Otherwise, proceed to step two. Steps 2 and 3: Check if TR is not empty. If it is, take a triplet tr from TR. c Set the triple tr c The first node is Htr c The tail node is Ttr c The set of remaining triples is TR' = TR - {tr c }, update TR=TR', and proceed to step two and four; otherwise, return to step two and two. Step 24, if p c ==tr c or tr c ∈p c Proceed to steps two and three; Otherwise, proceed to step two five; Step 25: If Htr c ==Tp c Then for path p c To expand, i.e.: p c =p c +tr c And update path p in Tp c Proceed to step two six; Otherwise, proceed to steps two and three; Step 26: If Ttr c ==Hp c Then for path p c To expand, i.e.: p c =tr c +p c And update path p in Tp c ; otherwise, proceed to step 27; Step 27: If Htr c ≠Tp c or Ttr c ≠Hp c Then P o =TP, proceed to step two eight; Otherwise, proceed to step two. Step 28: Obtain the set of reference argument paths P a ; Step 3: Determine the maximum intersection path between the observational argumentation path and the reference argumentation path; the specific process is as follows: Step 3.1 Initialize the intersection path set P c ; Step 32, if P a Not empty, from P a Extract a reference argument path p from the middle j The set of remaining paths is P' a =P a -{p j }, Update P a =P' a Proceed to step three; if P a If empty, proceed to step four; Step 33, if P o Not empty, from P o Extract an observational proof path p from the data. i The set of remaining paths is P' o =P o -{p i }, Update P o =P' o ; Perform steps three and four; If P o If empty, proceed to step three two; Steps 3 and 4: Obtain the observation and verification path p i and reference argument path p j Maximum intersection path p ∩ =p i ∩p j ; Step 4: Calculate the evaluation value of the maximum intersection path, and calculate the evaluation value of the critical thinking argumentation effect based on the evaluation value.

2. The method for evaluating the effectiveness of learners' argumentation process based on a critical thinking training model according to claim 1, characterized in that: Natural language processing tools are used to extract knowledge triples from learners' argument texts, outputting a set of triples TR = {(h, r, t)}, where h represents the head entity, r represents the relation, and t represents the tail entity. Entities in the triples are used as nodes, and relations are used as edges to construct a directed graph. Based on the directed graph, a graph database is used to store and visualize the knowledge graph, outputting an argument knowledge graph CKG = (V, E), where V is the set of nodes and E is the set of edges.

3. The method for evaluating the effectiveness of learners' argumentation process based on a critical thinking training model according to claim 1, characterized in that: In step four, the evaluation value of the maximum intersection path is calculated. It can be expressed as follows: Where, L ∩ L is the number of nodes in the maximum intersection path. i To observe the number of nodes in the argument path, L j The number of nodes in the reference argument path.

4. The method for evaluating the effectiveness of learners' argumentation process based on a critical thinking training model according to claim 3, characterized in that: The number of nodes in the maximum intersection path L ∩ =|p ∩ |; The number of nodes L in the observation and demonstration path i =|p i |;The number of nodes L in the reference argument path j =|p j | 5. The method for evaluating the effectiveness of learners' argumentation process based on a critical thinking training model according to claim 4, characterized in that: The evaluation value E for the effectiveness of critical thinking argumentation is calculated using the following formula:

6. A learner's argumentation process effectiveness evaluation system, characterized by: The system includes a memory, a processor, and an evaluation program for the effectiveness of learner argumentation process based on a critical thinking training model, which is stored in the memory and runs on the processor. When the evaluation program for the effectiveness of the critical thinking learning model is executed by the processor, it implements the steps of the method for evaluating the effectiveness of learner argumentation process based on a critical thinking training model as described in any one of claims 1 to 5.

Images