Course document understanding method based on multi-agent logical reasoning

Abstract

The invention discloses a course document understanding method based on multi-agent logical reasoning. The method comprises the following steps: (1) selecting a model; (2) performing data acquisition and preprocessing on the course document; (3) performing course content symbolization; (4) generating reasoning; (5) performing reasoning execution; (6) carrying out multi-agent collaborative confrontation; according to the method, a plurality of agents are integrated, and the defects, such as logic inconsistency, precondition hypothesis deviation or argumentation chain weakness, possibly existing in the reasoning process of the course content symbolization module, the reasoning step generation module and the reasoning step execution module are examined in a systematic mode, so that the curing dependence of a single model on a preliminary analysis result is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and natural language processing, and specifically relates to a method for understanding course documents based on multi-agent logical reasoning. Background Art

[0002] With the continuous progress of artificial intelligence technology, the role of intelligent tools in curriculum document compilation and teaching applications has become increasingly prominent. As an indispensable resource in the teaching process, curriculum documents not only support the cultivation of students' learning abilities but also promote the development of their logical and critical thinking, becoming an important part of educational practice. However, when using intelligent tools to assist in understanding professional curriculum documents, many challenges still exist, especially in the realization of multi-level semantic analysis and rigorous logical reasoning capabilities. Specifically, intelligent tools need to have the ability to refine and identify the content of curriculum documents. For example, accurately distinguish modules such as teaching objective statements, text analysis paragraphs, and exercise designs, and extract core elements such as teaching themes, discourse structures, rhetorical features, and the semantic connotations of professional terms. At the same time, it is also necessary to construct a complete logical reasoning framework based on the content of curriculum documents and teaching objectives, including modeling the logical relationships between paragraphs, inferring teaching intentions, verifying document coherence, and detecting potential contradictions. The realization of these capabilities is crucial for improving the accuracy and practicality of curriculum document understanding. In recent years, semantic understanding technology has made remarkable progress in dealing with tasks related to curriculum documents, and its core driving force comes from deep learning models trained on large-scale corpora and their advanced architecture designs. Such technologies can effectively capture semantic associations at the lexical, syntactic, and discourse levels, accurately analyze the contextual meanings of professional terms, and handle complex concepts and domain-specific expressions commonly found in teaching documents. For example, through a dynamic semantic representation mechanism, the model can adaptively handle polysemous words, metaphorical language, and ambiguous expressions in teaching objectives, thus demonstrating a context understanding ability close to that of humans. This ability lays a solid foundation for the deep semantic analysis of curriculum documents. However, relying solely on semantic understanding is not enough to meet the high-order requirements of curriculum document understanding because the analysis of teaching documents not only requires an accurate grasp of the content but also needs to reveal its internal structure and teaching intentions through logical reasoning. As a key link in curriculum document understanding, logical reasoning aims to represent the deep logical relationships of teaching content through evidence evaluation, argument construction, and deductive reasoning. Early deep learning-based logical reasoning methods were often built on convolutional neural networks and recurrent neural networks. For example, CNN extracts local semantic features from the text paragraphs of curriculum documents through convolutional operations, and then integrates these features through pooling layers to form a representation of the local logical relationships of teaching content. While RNN and its variants (such as long short-term memory network LSTM) are used to capture the temporal dependencies in the document, such as the logical progression relationship between paragraphs or the causal connection between teaching objectives and exercise designs. In addition, traditional methods also combine Knowledge Embedding technology to map text content into a high-dimensional vector space through word embedding models such as Word2Vec or GloVe, extract the semantic similarity between words, and combine Graph Neural Networks (GNN) to model the structured relationships of teaching documents.Nevertheless, traditional deep learning methods still face significant limitations in logical reasoning of course documents. Due to their reliance on numerical calculations and statistical pattern matching, the ambiguity of natural language expressions and the abstractness of teaching intentions are often difficult to fully capture through vector representations. In addition, the computational processes of these methods usually lack interpretability and are difficult to directly reveal the logical basis behind the reasoning conclusions, limiting the depth of their application in course document understanding. Therefore, developing a method that can achieve deep semantic parsing and provide a clear logical reasoning path has become an urgent technical problem to be solved. Summary of the Invention

[0003] The present invention proposes a method for understanding course documents based on multi-agent logical reasoning, providing a technical reference for intelligent tool-assisted understanding of professional course documents, and can also be applied to scenarios in the field of artificial intelligence that require the model to have strong logical reasoning capabilities.

[0004] To achieve the above object, the present invention provides the following technical solutions: A method for understanding course documents based on multi-agent logical reasoning, comprising the following steps: (1) Select models. The selected models are ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct, denoted as and respectively; (2) Obtain and preprocess the data of the course document: perform structured parsing on the content in the dataset, execute preprocessing operations such as text cleaning and syntactic analysis, and further organize it into a dataset of course document content and questions, denoted as , where represents the size of the question pair dataset; given the th course document content and its question , use the model to semantically segment into a set of course document content subsets , represents the total number of sub-premises contained in the decomposed course document content and are then further converted into corresponding FOL symbolic expressions and , and the process is expressed as:

[0005] where represents the total length of the model inference result, represents the a token tag, representing the relevant definitions of FOL symbolic expressions, , respectively representing the few-shot learning example set and prompt instructions provided in the course content symbolization module, and introducing a mixed expression of the original course document content, symbolized course content, original questions, and symbolized questions , where concat represents the string concatenation function; (4) Generate inference steps: Determine whether it is possible to derive the question from the original course document content ; (5) Perform inference execution: Generate a specific execution process based on the inference steps in step (4) to obtain the inference answer; (6) Conduct multi-agent collaborative confrontation; After obtaining the initial inference process and inference results by completing steps (1)-(5) through different models, judge whether the positions of these models are consistent according to the final inference results. If the positions are consistent, directly confirm the final inference result. Otherwise, let each other question and conduct stress tests on the generated content of their three modules respectively to identify potential logical defects, assumption biases, or insufficient evidence.

[0006] Preferably, the step (4) is based on and the logical inference rule set to determine whether it is possible to derive the question from the original course document content , specifically: perform semantic analysis on to extract its logical structure. Subsequently, predefined a set of first-order logical inference rules , and by preferentially selecting applicable rules, let the model iteratively construct the inference steps , and its process is expressed as:

[0007] where and respectively represent the few-shot learning example set and prompt instructions provided in the inference plan generation module.

[0008] Preferably, the step (5) is based on and Generate a specific execution process to obtain the inference answer. Specifically: Let the model know the specific meaning of each truth attribution, and determine the truth attribution of the problem statement under discussion, namely True, False, or Unknown; True means that the problem statement can be inferred based on the following; False means that the problem statement cannot be inferred based on the following; Unknown means that the truth or falsehood of the problem statement cannot be inferred based on the following. Subsequently, mark the specific logical rules applied from each inference step to the next in the inference steps, and verify the applicability of each marked rule. If it is found that there are deviations or errors in the rule application, re-evaluate the rule selection through a dynamic correction mechanism, and introduce supplementary inference steps for optimization when necessary to further ensure the accuracy of the overall inference steps. Finally, let the model obtain the execution process based on the optimized inference steps and execute the complete inference to obtain its initial inference answer. The process is expressed as:

[0009]

[0010] where and respectively represent the few-shot learning example set and the prompt instructions provided in the inference step execution module.

[0011] Preferably, step (6) uses and two models for multi-agent collaborative confrontation, which specifically includes the following steps: Model evaluates the natural language to symbolic representation conversion of model in the course content symbolization module, and questions the rationality of its quantifier usage and the consistency of the propositional logic operator with the semantics; in the inference step generation module, evaluates whether the inference scheme of model follows the first-order logic rules, and questions whether there are instantiation omissions, incorrect premise references, or undefined assumptions; in the inference step execution module, evaluates the rigor of the execution steps of model and questions whether there are logical jumps or implicit assumptions. Finally, organize the questioning results of model in the three stages into a structured questioning report , and the process can be expressed as:

[0012] where represents the questioning prompt instructions provided in the multi-agent collaborative confrontation module.

[0013] Preferably, step (6) further includes the following steps: According to the questioning opinions, model Select to accept or reject. If rejected, the reason for rejection must be given. If accepted, optimize and modify according to the doubts, and finally let the model According to Optimize and verify its own reasoning process to generate the final answer.

[0014] Preferably, the reasoning steps in step (6) strictly follow ; Finally generate the answer , expressed as:

[0015] Among them Represents the verification prompt instruction provided in the multi-agent collaborative confrontation module.

[0016] The present invention has the following beneficial effects: The present invention integrates multiple agents and systematically examines the possible defects in the reasoning processes of the three modules of curriculum content symbolization, inference step generation, and inference step execution, such as logical inconsistencies, deviations in premise assumptions, or weak argument chains, thereby avoiding the rigid dependence of a single model on the preliminary analysis results; and driven by adversarial feedback, collaboratively optimizes the reasoning path and verifies the logical rigor of its results. Through multi-perspective questioning and verification, this module ensures that the reasoning process can adapt to the diverse teaching scenarios and content structures in curriculum documents. Through the perspective confrontation and functional complementarity of multiple agents, it not only enhances the model's critical reflection ability on its own reasoning but also significantly reduces the risk of reasoning errors, thereby improving its performance in complex logical reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It provides a schematic flow chart of an intelligent agent collaborative confrontation method based on symbolic thinking chains for the implementation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0020] Embodiment The following are only the preferred embodiments of the present invention, and the scope of protection of the present invention is not limited to the following embodiments. All technical solutions falling within the concept of the present invention belong to the scope of protection of the present invention.

[0021] See, for example Figure 1 As shown, a method for understanding course documents based on multi-agent logical reasoning of the present invention includes: Model selection. Select the ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct models as the base models, denoted as and respectively. To ensure the reproducibility of experimental results, set the temperature parameter .

[0022] Data acquisition and preprocessing module: First, perform structured parsing on the content in the dataset, execute preprocessing operations such as text cleaning and syntactic analysis, extract high-quality and effective body content from the complex dataset, and further organize it into course document content and question pairs , represents the size of the question pair dataset. Course content symbolization module: Given the th course document content and its question , this module first sets the model role as an expert who is good at symbolizing natural language expressions into first-order logic expressions (FOL). This method of setting the role can strengthen the model's ability to perform symbolic conversion on course content, thus significantly improving its performance in the tasks of this module. Then, use the model to perform semantic segmentation into a set of course document content subsets , Indicates the content of the course document The total number of sub-premises contained after decomposition, and then further and Convert to the corresponding FOL symbolic expression and , and the process can be expressed as:

[0023] where Indicates the total length of the model inference result, Indicates the th token of the model inference result, Indicates the relevant definition of the FOL symbolic expression, , respectively represent the few-shot learning example set and the prompt instruction provided in the course content symbolization module.

[0024] In the process of converting natural language to symbolic language, a certain degree of information loss may occur. Therefore, the present invention introduces the semantic information of natural language on the basis of symbolic expression to obtain a mixed expression of the original course document content, symbolized course content, original question and symbolized question . The process can be expressed as:

[0025] where concat represents the string concatenation function. Inference step generation module: The task of this module is based on and the set of logical inference rules to determine whether the question can be derived from the original course document content . Specifically, this module first sets the model role as an expert proficient in first-order logic and inference step generation; then, performs semantic analysis on , extracts its logical structure (such as whether there are universal quantifiers, conditional quantifiers or conditional statements, etc.). Subsequently, based on the method in the literature (Enderton H B. A mathematical introduction to logic[M]. Elsevier, 2001.), a set of first-order logic inference rules is predefined, and by preferentially selecting applicable rules, the model iteratively constructs the inference steps . The process can be expressed as:

[0026] where and respectively represent the few-shot learning example set and the prompt instruction provided in the inference plan generation module.

[0027] Inference step execution module: The task of this module is to generate a specific execution process based on and to obtain the inference answer. Specifically, first let the model know the specific meaning of each truth attribution, and determine the truth attribution of the problem statement , that is, True, False or Unknown; True means that the problem statement can be inferred according to under ; False means that the problem statement cannot be inferred according to under ; Unknown means that the truth or falsehood of the problem statement cannot be inferred according to under ; Subsequently, label the specific logical rules applied from each inference step to the next inference step in the inference step , and verify the applicability of each labeled rule. If it is found that there are deviations or errors in the rule application, re-evaluate the rule selection through the dynamic correction mechanism, and introduce supplementary inference steps to for optimization to further ensure the accuracy of the overall inference steps. Finally, let the model obtain the execution process according to the optimized inference steps and execute the complete inference to obtain its initial inference answer . Its process can be expressed as: where and

[0028] respectively represent the few-shot learning example set and the prompt instruction provided in the inference step execution module. and respectively represent the few-shot learning example set and the prompt instruction provided in the inference step execution module.

[0029] Multi-agent collaborative confrontation module: First, use and two models to complete the processes of the above three modules to obtain the initial inference process and inference result, and then judge whether the positions of the two models are consistent according to the final inference result. If the positions are consistent, directly confirm the final inference result, otherwise let the other party question and conduct stress tests on the generated content of their own three modules respectively to identify potential logical defects, hypothesis deviations or insufficient evidence. Specifically, taking the use of the model to question the multi-module inference result of the model as an example, the model In the evaluation model of the course content symbolization module for the conversion from natural language to symbolic representation, question the rationality of its quantifier usage and the consistency between propositional logic operators and semantics; in the evaluation model of the reasoning step generation module question whether the reasoning scheme follows the first-order logic rules, and question whether there are omissions in instantiation, incorrect premise references, or undefined assumptions; in the evaluation model of the reasoning step execution module question the rigor of the execution steps, and question whether there are logical leaps or implicit assumptions. Finally, organize the questioning results of the model in the three stages into a structured questioning report , and its process can be expressed as:

[0030] where represents the questioning prompt instructions provided in the multi-agent collaborative confrontation module.

[0031] According to the questioning opinions, the model can choose to accept or reject. If it rejects, it needs to give the reason for rejection. If it accepts, it will optimize and modify according to its questioning opinions. Finally, let the model According to optimize and verify its own reasoning process to generate the final answer. Specifically, the reasoning content of the model in the course content symbolization module needs to ensure that the symbolic expression is strictly equivalent to the natural language; adjust and supplement the content of the reasoning step generation module if necessary; ensure that the reasoning steps of the reasoning step execution module strictly follow ; finally generate the answer . Its process can be expressed as:

[0032] where represents the verification prompt instructions provided in the multi-agent collaborative confrontation module. The effect of the present invention can be further verified from the following experimental results. The present invention selects the FOLIO dataset as the basic data source for complex logical reasoning tasks. The FOLIO dataset is a publicly available dataset specifically designed to evaluate the logical reasoning ability of models, and its feature is that it contains a large number of logical reasoning questions expressed in natural language, covering diverse reasoning scenarios such as deductive reasoning, inductive reasoning, and analogical reasoning, etc. In the field of English reading comprehension, the present invention selects the RACE dataset as the experimental benchmark. RACE is an annotated dataset specifically constructed for Chinese junior high school and high school English reading comprehension tests, and can effectively support the evaluation of models in semantic understanding and logical reasoning.

[0033] In the English reading comprehension and general logical reasoning experiments, the most advanced existing methods were selected for the detection effect comparison tests. Specifically, (Literature 1 - Xu J, Fei H, Pan L, et al. "Faithful Logical Reasoning via Symbolic Chain-of-Thought." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024.) In the English reading comprehension experiment, the dev / high subset with the highest difficulty was selected from the RACE dataset, and 150 instances were selected for relevant experiments. The test results are shown in the following table: Table 1 Test results of selecting the dev / high subset with the highest difficulty in the RACE dataset

[0034] In the general logical reasoning experiment, all instances were selected from the FOLIO dataset for relevant experiments. The test results are shown in the following table: Table 2 Test results of selecting all instances from the FOLIO dataset for relevant experiments

[0035] The experimental results fully demonstrate the effectiveness of the method proposed in this paper in the field of course document understanding. Compared with the method in Literature 1, the two benchmark large language models achieved the highest average accuracy rates in both the English course reading comprehension task and the general logical reasoning task, especially showing outstanding performance in the English reading comprehension task. From the experimental results, a course document understanding method based on multi-agent logical reasoning designed in this invention has certain theoretical significance and practical application value, and the experiments verified the effectiveness of the method proposed in this invention. The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit and essence of the present invention should be covered within the protection scope of the present invention.

Claims

1. A course document understanding method based on multi-agent logical reasoning, characterized in that: The following steps are involved: (1) Model selection: ChatGLM-4-9B-Chat and Llama-3.1-8B-Instruct, denoted as and ; (2) Data acquisition and preprocessing of course documents: structural analysis of the content in the data set, preprocessing operations such as text cleaning and syntactic analysis, and further organizing the course document content and questions into a data set, represented as , Indicates the size of the problem dataset; (3) Symbolize the course content: Given Course Document Content and its problems , using the model Perform semantic segmentation into a set of course document content subsets , Represents the course document content The total number of sub-premises after decomposition, and then further and Convert to the corresponding FOL symbolic expression and , the process is expressed as: in Indicates the total length of the model inference result, The first Token tags, Represents the relevant definition of FOL symbolic expression, , They represent the few-shot learning example set and prompt instructions provided in the course content symbolization module, and introduce the mixed expression of the original course document content, symbolized course content, original questions and symbolic questions. , where concat represents the string concatenation function; (4) Generate reasoning step: Determine whether it is possible to Deriving the problem ; (5) Perform reasoning execution: Generate a specific execution process based on the reasoning steps in step (4) to obtain the reasoning answer; (6) Conduct multi-agent collaborative confrontation; after completing steps (1) to (5) through different models to obtain the initial reasoning process and reasoning results, judge whether the positions of these models are consistent based on the final reasoning results. If the positions are consistent, directly confirm the final reasoning results. Otherwise, let the other party question and stress test the generated content of the three modules to identify potential logical flaws, assumption bias or insufficient evidence.

2. The course document understanding method based on multi-agent logical reasoning according to claim 1 is characterized in that: The step (4) is based on and a set of logical reasoning rules to determine whether the original course document content can be Deriving the problem , specifically: Perform semantic analysis to extract its logical structure, and then predefine a set of first-order logic reasoning rules , and by prioritizing the applicable rules, the model Iteratively construct reasoning steps , the process is expressed as: in and They represent the few-shot learning example set and hint instructions provided in the inference plan generation module, respectively.

3. The course document understanding method based on multi-agent logical reasoning according to claim 2 is characterized in that: The step (5) is based on and Generate a specific execution process to get the reasoning answer, specifically: let the model know the specific meaning of each truth value attribution, and determine the truth value attribution of the problem statement under discussion, that is, True, False or Unknown; True means that the problem statement can be inferred based on it; False means that the problem statement cannot be inferred based on it; Unknown means that the truth or falsity of the problem statement cannot be inferred based on it; then, mark the specific logical rules applied from each reasoning step to the next reasoning step in the reasoning step, and verify the applicability of each marked rule. If deviations or errors are found in the rule application, the rule selection is re-evaluated through the dynamic correction mechanism, and if necessary, supplementary reasoning steps are introduced to optimize it to further ensure the accuracy of the overall reasoning steps; finally, let the model obtain the execution process according to the optimized reasoning steps and perform complete reasoning to obtain its initial reasoning answer; the process is expressed as: in and They represent the few-shot learning example set and hint instructions provided in the inference step execution module, respectively.

4. The course document understanding method based on multi-agent logical reasoning according to claim 3 is characterized in that: Step (6) utilizes and The two models conduct multi-agent collaborative confrontation, which includes the following steps: Evaluation model for module on symbolization of course content The conversion from natural language to symbolic representation questioned the rationality of its use of quantifiers and the consistency of propositional logic operators with semantics; the module evaluation model was generated in the reasoning step The reasoning scheme of the model follows the rules of first-order logic, and questions whether there are instantiation omissions, incorrect premise references, or undefined assumptions; the module evaluation model is executed at the reasoning step The rigor of the execution steps, questioning whether there are logical jumps or implicit assumptions, and finally The results of the three-stage questioning are organized into a structured questioning report , the process can be expressed as: in Represents the question prompt instructions provided in the multi-agent collaborative confrontation module.

5. The course document understanding method based on multi-agent logical reasoning according to claim 4 is characterized in that: The step (6) further includes the following steps: according to The questioning opinions, model Choose to accept or reject. If you reject, you need to give the reason for rejection. If you accept, you need to optimize and modify according to their doubts. Finally, let the model according to Optimize and verify its own reasoning process to generate the final answer.

6. The course document understanding method based on multi-agent logical reasoning according to claim 5 is characterized in that: The reasoning steps in step (6) strictly follow ; Finally generate the answer , expressed as: in Represents the verification prompt instructions provided in the multi-agent collaborative confrontation module.

Images