A course document understanding method based on multi-agent logical reasoning

Abstract

The present invention discloses a course document understanding method based on multi-agent logical reasoning, comprising the following steps: (1) model selection; (2) data acquisition and preprocessing of course documents; (3) symbolization of course content; (4) generation of reasoning steps; (5) execution of reasoning; and (6) collaborative confrontation of multiple agents. The present invention integrates multiple agents and systematically examines possible defects in the reasoning process of the three modules of course content symbolization, reasoning step generation, and reasoning step execution, such as logical inconsistency, deviation of premise assumptions, or weak argumentation chain, thereby avoiding the rigid dependence of a single model on the preliminary analysis results.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and natural language processing, and specifically is a course document understanding method based on multi-agent logical reasoning. Background Art

[0002] With the continuous advancement of artificial intelligence technology, the role of intelligent tools in course document compilation and teaching applications is becoming increasingly prominent. As an indispensable resource in the teaching process, course documents not only support the development of students' learning abilities but also promote their logical and critical thinking, becoming a crucial component of educational practice. However, using intelligent tools to assist in understanding professional course documents still faces many challenges, particularly in achieving multi-level semantic parsing and rigorous logical reasoning. Specifically, intelligent tools need to be able to break down and identify course document content in detail, such as accurately distinguishing between modules such as teaching objective statements, text analysis paragraphs, and exercise designs, and extracting core elements such as teaching topics, discourse structure, rhetorical features, and the semantic connotations of professional terms. Furthermore, they need to construct a comprehensive logical reasoning framework based on the course document content and teaching objectives, including modeling logical relationships between paragraphs, inferring teaching intent, verifying document coherence, and detecting potential contradictions. Achieving these capabilities is crucial for improving the accuracy and practicality of course document understanding. In recent years, semantic understanding technology has made significant progress in handling course document-related tasks, driven by deep learning models trained on large-scale corpora and their advanced architectural design. This type of technology can effectively capture semantic associations at the lexical, syntactic, and discourse levels, accurately parse the contextual meaning of professional terms, and handle complex concepts and domain-specific expressions commonly found in teaching documents. For example, through a dynamic semantic representation mechanism, the model can adaptively handle polysemous words, metaphorical language, and ambiguous expressions in teaching objectives, thereby demonstrating a contextual understanding capability close to that of humans. This capability lays a solid foundation for deep semantic parsing of course documents. However, relying solely on semantic understanding is not enough to meet the high-level needs of course document understanding, because the analysis of teaching documents requires not only an accurate grasp of the content, but also the use of logical reasoning to reveal its internal structure and teaching intent. Logical reasoning, as a key link in course document understanding, aims to represent the deep logical relationships of teaching content through evidence evaluation, argument construction, and deductive reasoning. Early deep learning-based logical reasoning methods were often built based on convolutional neural networks and recurrent neural networks. For example, CNNs extract local semantic features from text paragraphs in course documents through convolution operations, then integrate these features through pooling layers to form a representation of the local logical relationships within the teaching content. RNNs and their variants (such as long short-term memory (LSTM) networks) are used to capture temporal dependencies within documents, such as the logical progression between paragraphs or the causal relationship between teaching objectives and exercise design. Furthermore, traditional methods incorporate knowledge embedding techniques, using word embedding models such as Word2Vec or GloVe to map text content into a high-dimensional vector space, extract semantic similarities between words, and incorporate graph neural networks (GNNs) to model the structured relationships within teaching documents.Despite this, traditional deep learning methods still face significant limitations in logical reasoning in course documents. Due to their reliance on numerical computation and statistical pattern matching, the ambiguity of natural language expressions and the abstract nature of teaching intent are often difficult to fully capture through vector representations. Furthermore, the computational processes of these methods often lack interpretability, making it difficult to directly reveal the logical basis behind the reasoning conclusions, limiting their application depth in course document understanding. Therefore, developing a method that can both achieve deep semantic parsing and provide clear logical reasoning paths has become a technical challenge that needs to be urgently addressed. Summary of the Invention

[0003] This paper proposes a course document understanding method based on multi-agent logical reasoning, which provides a technical reference for intelligent tools to assist in understanding professional course documents. It can also be applied to scenarios in the field of artificial intelligence that require models to have strong logical reasoning capabilities.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a course document understanding method based on multi-agent logical reasoning, comprising the following steps:

[0005] (1) Model selection: ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct, respectively. and ;

[0006] (2) Data acquisition and preprocessing of course documents: structural analysis of the content in the dataset, preprocessing operations such as text cleaning and syntactic analysis, and further sorting into course document content and questions to form a dataset, represented as , Indicates the size of the dataset for the problem;

[0007] (3) Symbolize the course content: Given Course document content and its problems , using the model Perform semantic segmentation into a set of course document content subsets , Represents the content of the course document The total number of sub-premises after decomposition, and then further and Convert to the corresponding FOL symbolic expression and , the process is expressed as:

[0008]

[0009] in Indicates the total length of the model inference result, The first Token mark, Represents the relevant definitions of FOL symbolic expressions, , They represent the few-shot learning example set and prompt instructions provided in the course content symbolization module, and introduce a mixed expression of the original course document content, symbolic course content, original questions, and symbolic questions. , where concat represents the string concatenation function;

[0010] (4) Generate reasoning step: Determine whether it is possible to Deriving the problem ;

[0011] (5) Perform reasoning execution: Generate a specific execution process based on the reasoning steps in step (4) to obtain the reasoning answer;

[0012] (6) Conduct multi-agent collaborative confrontation; after completing steps (1) to (5) through different models to obtain the initial reasoning process and reasoning results, judge whether the positions of these models are consistent based on the final reasoning results. If the positions are consistent, directly confirm the final reasoning results. Otherwise, let the other party question and stress test the generated content of their three modules to identify potential logical flaws, assumption bias or insufficient evidence.

[0013] Preferably, the step (4) is based on and logical reasoning rules to determine whether the original course document content can be Deriving the problem , specifically: Perform semantic analysis to extract its logical structure, and then predefine a set of first-order logic reasoning rules By prioritizing the applicable rules, the model Iteratively construct reasoning steps , the process is expressed as:

[0014]

[0015] in and They represent the few-shot learning example set and hint instructions provided in the inference plan generation module, respectively.

[0016] Preferably, the step (5) is based on and Generate a specific execution process to obtain the reasoning answer, specifically: let the model know the specific meaning of each truth value attribution, and determine the truth value attribution of the problem statement in question, that is, True, False or Unknown; True means that the problem statement can be inferred based on the following; False means that the problem statement cannot be inferred based on the following; Unknown means that the truth or falsity of the problem statement cannot be inferred based on the following; then, mark the specific logical rules applied from each reasoning step to the next reasoning step in the reasoning step, and verify the applicability of each marked rule. If deviations or errors are found in the rule application, the rule selection is re-evaluated through the dynamic correction mechanism, and if necessary, supplementary reasoning steps are introduced to optimize to further ensure the accuracy of the overall reasoning steps; finally, let the model obtain the execution process according to the optimized reasoning steps and perform complete reasoning to obtain its initial reasoning answer; the process is expressed as:

[0017]

[0018]

[0019] in and They represent the few-shot learning example set and hint instructions provided in the inference step execution module, respectively.

[0020] Preferably, the step (6) utilizes and The two models conduct multi-agent collaborative confrontation, which includes the following steps: Evaluation model for the module on symbolic content of courses The conversion of natural language to symbolic representation, questioning the rationality of its quantifier use and the consistency of propositional logic operators with semantics; generating module evaluation models in the reasoning step Whether the reasoning scheme follows the first-order logic rules, question whether there are instantiation omissions, incorrect premise references or undefined assumptions; execute the module evaluation model at the reasoning step The rigor of the execution steps, questioning whether there are logical jumps or implicit assumptions, and finally The results of the three-stage questioning are organized into a structured questioning report , the process can be expressed as:

[0021]

[0022] in Represents the question prompt instructions provided in the multi-agent collaborative confrontation module.

[0023] Preferably, the step (6) further comprises the following steps: skeptical opinions, model Choose to accept or reject. If you reject, you need to give the reason for rejection. If you accept, you need to optimize and modify according to the doubts. Finally, let the model according to Optimize and verify its own reasoning process to generate the final answer.

[0024] Preferably, the reasoning steps in step (6) strictly follow ;Finally generate the answer , expressed as:

[0025]

[0026] in Represents the verification prompt instructions provided in the multi-agent collaborative confrontation module.

[0027] The present invention has the following beneficial effects: the present invention integrates multiple intelligent agents to systematically examine the possible defects in the reasoning process of the three modules of course content symbolization, reasoning step generation and reasoning step execution, such as logical inconsistency, deviation of premise assumptions or weak argumentation chain, thereby avoiding the rigid dependence of a single model on the preliminary analysis results; and driven by adversarial feedback, collaboratively optimize the reasoning path and verify the logical rigor of its results. Through multi-perspective questioning and verification, this module ensures that the reasoning process can adapt to the diverse teaching scenarios and content structures in the course documents. Through the perspective confrontation and functional complementarity of multiple agents, not only the model's critical reflection ability on its own reasoning is enhanced, but also the risk of reasoning errors is significantly reduced, thereby improving its performance in complex logical reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For those skilled in the art, other relevant drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A flowchart of an intelligent agent collaborative confrontation method based on a symbolic thinking chain is provided for the implementation of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by the first technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is sought, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by the first technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] Example

[0032] The following are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the following embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention.

[0033] See for example Figure 1 As shown, the present invention provides a course document understanding method based on multi-agent logical reasoning, comprising:

[0034] Model selection. The ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct models are selected as the base models, respectively and To ensure the repeatability of the experimental results, set the temperature parameters .

[0035] Data acquisition and preprocessing module: First, the content in the data set is structurally analyzed, and preprocessing operations such as text cleaning and syntax analysis are performed to extract high-quality and effective text content from the complex data set, and further organize it into course document content and question pairs. , Indicates the size of the dataset for the problem. Course Content Symbolization Module: Given Course document content and its problems This module first sets the model role as an expert who is good at symbolizing natural language expressions into first-order logic expressions (FOL). This role setting method can enhance the model's ability to symbolize course content, thereby significantly improving its performance on the module's tasks. Will Perform semantic segmentation into a set of course document content subsets , Represents the content of the course document The total number of sub-premises after decomposition, and then further and Convert to the corresponding FOL symbolic expression and , the process can be expressed as:

[0036]

[0037] in Indicates the total length of the model inference result, The first Token mark, Represents the relevant definitions of FOL symbolic expressions, , They respectively represent the few-shot learning example set and prompt instructions provided in the course content symbolization module.

[0038] In the process of converting natural language into symbolic language, information may be lost to a certain extent. Therefore, the present invention introduces the semantic information of natural language on the basis of symbolic expression to obtain a mixed expression of original course document content, symbolic course content, original questions and symbolic questions. The process can be expressed as:

[0039]

[0040] Where concat represents the string concatenation function.

[0041] Reasoning step generation module: The task of this module is based on and logical reasoning rules to determine whether the original course document content can be Deriving the problem Specifically, this module first sets the model role as an expert who is proficient in first-order logic and reasoning step generation; then, Perform semantic analysis to extract its logical structure (e.g., whether there are universal quantifiers, conditional quantifiers, or conditional statements). Then, predefine a set of first-order logic reasoning rules based on the literature method (Enderton H B. A mathematical introduction to logic [M]. Elsevier, 2001.) By prioritizing the applicable rules, the model Iteratively construct reasoning steps The process can be expressed as:

[0042]

[0043] in and They represent the few-shot learning example set and hint instructions provided in the inference plan generation module, respectively.

[0044] Reasoning step execution module: The task of this module is based on and Generate a specific execution process to get the reasoning answer. Specifically, first let the model Know the specific meaning of each truth attribute and identify the problem statement being discussed The truth value of exist Infer the problem statement ; False means that it is not possible to exist Infer the problem statement ; Unknown means that it is not possible to exist Infer the problem statement Then, mark the reasoning steps The specific logical rules applied from each reasoning step to the next reasoning step in the process are analyzed, and the applicability of each annotation rule is verified. If deviations or errors are found in the application of rules, the rule selection is re-evaluated through the dynamic correction mechanism, and supplementary reasoning steps are introduced when necessary. Optimization is performed to further ensure the accuracy of the overall inference step. Finally, let the model Get the execution process according to the optimized reasoning steps And perform complete reasoning to get its initial reasoning answer The process can be expressed as:

[0045]

[0046] in and They represent the few-shot learning example set and hint instructions provided in the inference step execution module, respectively.

[0047] Multi-agent collaborative confrontation module: First, use and After the two models complete the above three modules and obtain the initial reasoning process and reasoning results, they will judge whether the positions of the two models are consistent based on the final reasoning results. If the positions are consistent, the final reasoning results will be directly confirmed. Otherwise, the other party will be asked to question and stress test the generated content of the three modules to identify potential logical flaws, assumption bias or insufficient evidence. Specifically, using the model Model For example, the model Evaluation model for the module on symbolic content of courses The conversion of natural language to symbolic representation, questioning the rationality of its quantifier use and the consistency of propositional logic operators with semantics; generating module evaluation models in the reasoning step Whether the reasoning scheme follows the first-order logic rules, question whether there are instantiation omissions, incorrect premise references or undefined assumptions; execute the module evaluation model at the reasoning step The rigor of the execution steps is questioned, and whether there are logical jumps or implicit assumptions. The results of the three-stage questioning are organized into a structured questioning report , the process can be expressed as:

[0048]

[0049] in Represents the question prompt instructions provided in the multi-agent collaborative confrontation module.

[0050] according to skeptical opinions, model You can choose to accept or reject. If you reject, you need to give the reason for rejection. If you accept, you need to optimize and modify according to the doubts. according to Optimize and verify its own reasoning process to generate the final answer. Specifically, the model The reasoning content in the symbolization module of the course content needs to ensure that the symbolic expression is strictly equivalent to the natural language; if necessary, the content of the reasoning step generation module should be adjusted and supplemented; and the reasoning steps in the reasoning step execution module should strictly follow the ;Finally generate the answer The process can be expressed as:

[0051]

[0052] in Represents the verification prompt instruction provided in the multi-agent collaborative confrontation module. The effect of the present invention can be further verified from the following experimental results. The present invention uses the FOLIO dataset as the basic data source for complex logical reasoning tasks. The FOLIO dataset is a public dataset designed specifically for evaluating the logical reasoning ability of models. Its characteristic is that it contains a large number of logical reasoning problems based on natural language expressions, covering a variety of reasoning scenarios, such as deductive reasoning, inductive reasoning, and analogical reasoning. In the field of English reading comprehension, the present invention uses the RACE dataset as the experimental benchmark. RACE is a labeled dataset specially constructed for Chinese junior and senior high school English reading comprehension exams, which can effectively support the evaluation of models in semantic understanding and logical reasoning.

[0053] In both the English reading comprehension and general logical reasoning experiments, we selected the most advanced existing methods for comparative testing (reference 1 - Xu J, Fei H, Pan L, et al. "Faithful Logical Reasoning via Symbolic Chain-of-Thought." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024). For the English reading comprehension experiment, we selected the most difficult dev / high subset of the RACE dataset and conducted 150 examples. The test results are shown in the following table:

[0054] Table 1 Test results of the most difficult dev / high subset in the RACE dataset

[0055]

[0056] In the general logic reasoning experiment, all instances in the FOLIO dataset were selected for related experiments. The test results are shown in the following table:

[0057] Table 2 Test results of all instances selected from the FOLIO dataset for related experiments

[0058]

[0059] The experimental results fully demonstrate the effectiveness of the method in this paper in the field of course document understanding. Compared with the method in Document 1, the two benchmark large language models proposed in this study achieved the highest average accuracy in the reading comprehension task of English courses and the general logical reasoning task, especially in the English reading comprehension task. Judging from the experimental results, the course document understanding method based on multi-agent logical reasoning designed by the present invention has certain theoretical significance and practical application value, and the experiment verifies the effectiveness of the method proposed in the present invention. The above embodiments are only for illustrating the technical concept and features of the present invention. Its purpose is to enable people familiar with this technology to understand the content of the present invention and implement it accordingly, and it cannot limit the scope of protection of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A course document understanding method based on multi-agent logical reasoning, characterized by: The following steps are involved: (1) Model selection: ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct, denoted as P GLM and P Llama ; (2) Data acquisition and preprocessing of course documents: structural analysis of the acquired data, preprocessing operations such as text cleaning and syntactic analysis, and then organizing them into course document content and question datasets, represented as N represents the size of the dataset; (3) Symbolize the course content: Given the original course document content C i and its original question Q i , using the model to C i Perform semantic segmentation into a set of course document content subsets J represents the original course document content C i The total number of sub-premises after decomposition, then and Q i Convert to the corresponding first-order logic expression FOL symbolic expression and The process is expressed as: Where |A| represents the total length of the model inference result, a k Indicates the kth token of the model inference result, D FOL Represents the relevant definition of FOL symbolic expression, T FS , T PROMPT They represent the few-shot learning example set and prompt instructions provided in the course content symbolization module, and introduce a mixed expression of the original course document content, symbolic course content, original questions, and symbolic questions. Where concat represents the string concatenation function; (4) Generate reasoning steps: Determine whether it is possible to i Derive question Q i ; Step (4) is based on and a set of logical reasoning rules to determine whether the original course document content C i Derive question Q i , specifically: Perform semantic analysis to extract its logical structure, and then predefine a set of first-order logic reasoning rules R FOL By prioritizing the applicable rules, let the model P GLM Iteratively construct reasoning steps S i , the process is expressed as: Among them, P FS and P PROMPT They represent the few-shot learning example set and hint instructions provided in the inference plan generation module respectively; (5) Perform reasoning execution: Generate a specific execution process based on the reasoning steps in step (4) to obtain the reasoning answer; (6) Conduct multi-agent collaborative confrontation; after completing steps (1) to (5) through different models to obtain the initial reasoning process and reasoning results, judge whether the positions of these models are consistent based on the final reasoning results. If the positions are consistent, directly confirm the final reasoning results. Otherwise, question and stress test the generated content of the other party to identify potential logical flaws, assumption bias or insufficient evidence.

2. The course document understanding method based on multi-agent logical reasoning according to claim 1 is characterized in that: The step (5) is based on S i and R FOL Generate a specific execution process to obtain the reasoning answer, specifically: let the model know the specific meaning of each truth value attribution, and determine the truth value attribution of the problem statement in question, that is, True, False or Unknown; True means that the problem statement can be inferred based on the following; False means that the problem statement cannot be inferred based on the following; Unknown means that the truth or falsity of the problem statement cannot be inferred based on the following; then, mark the specific logical rules applied from each reasoning step to the next reasoning step in the reasoning step, and verify the applicability of each marked rule. If deviations or errors are found in the rule application, the rule selection is re-evaluated through a dynamic correction mechanism; finally, let the model obtain the execution process Ei according to the optimized reasoning steps and perform complete reasoning to obtain its initial reasoning answer Yi.

3. The course document understanding method based on multi-agent logical reasoning according to claim 2 is characterized in that: The step (6) utilizes P Llama and P GLM The two models conduct multi-agent collaborative confrontation, which includes the following steps: Model P Llama In the course content symbolization module evaluation model P GLM The conversion of natural language to symbolic expression, questioning the rationality of its quantifier use and the consistency of propositional logic operators with semantics; generating module evaluation model P in the reasoning step GLM whether the reasoning scheme follows the rules of first-order logic, questioning whether there are omissions of instantiation, incorrect premise citations, or undefined assumptions; At the inference step, the module evaluates the model P. GLM The rigor of the execution steps, questioning whether there are logical jumps or implicit assumptions, and finally the model P Llama The results of the three stages of questioning are organized into a structured questioning report R i , the process can be expressed as: Among them B ATTACK_PROMPT Represents the question prompt instructions provided in the multi-agent collaborative confrontation module.

4. The course document understanding method based on multi-agent logical reasoning according to claim 3 is characterized in that: The step (6) further comprises the following steps: According to R i The questioning opinion of Model P GLM Choose to accept or reject. If you reject, you need to give the reason for rejection. If you accept, you need to optimize and modify according to the doubts. Finally, let the model P GLM According to R i Optimize and verify its own reasoning process to generate the final answer.

5. The course document understanding method based on multi-agent logical reasoning according to claim 4 is characterized in that: The reasoning steps in step (6) strictly follow S i ;Finally generate the answer Y i_final , expressed as: Among them B VER_PROMPT Represents the verification prompt instructions provided in the multi-agent collaborative confrontation module.

Images