Generative agent-driven teacher thinking cognition diagnosis framework

Abstract

The invention discloses a generative agent cognitive diagnosis framework fusing teacher thinking, and the framework is used for modeling a cognitive judgment process of a teacher in a real teaching scene, systematically constructing task-aware thinking modeling, a behavior track-based observation reflection mechanism, and a causal-driven explanation generation module. Therefore, dynamic modeling and deviation diagnosis of the cognitive state of the student are realized. The method has the advantages that firstly, the method has good cold start capability, and reasonable judgment can be made even if complete historical behavior data is lacked; 2, the method has relatively high interpretability, can output a causal chain of'knowledge state-behavior process-task performance ', and supports accurate teaching intervention; and thirdly, the method has the ability of distinguishing the thinking levels of students, can assist in completing Bloom cognitive classification, and provides effective support for task layering and teaching design.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of education data mining and artificial intelligence, in particular to a framework for dynamically diagnosing the cognitive state of learners by using a generative agent, which realizes fine-grained and interpretable evaluation of the mastery of knowledge points of students by fusing the semantic reasoning ability of the teaching thinking rules of teachers and generative models. BACKGROUND

[0002] Cognitive diagnosis (CD) aims to infer the potential knowledge mastery state and cognitive ability structure of learners by analyzing their historical interaction data, and then support personalized teaching decisions. Existing methods are mostly based on static answer records and have achieved high prediction accuracy in closed item banks or standardized tasks. However, in real teaching scenarios, the behavior of learners shows significant individual differences, strategy evolution and emotional fluctuations, which leads to the decline of the generalization performance of existing models when migrating across students and scenarios, and makes it difficult to track the dynamic evolution process of cognitive state in a fine-grained manner. In addition, existing methods focus on the prediction accuracy of answer results, lack the modeling ability of structured association between "cognitive state-behavior path-task performance", and cannot generate traceable and interpretable cognitive bias attribution, nor can they provide operational decision basis for teaching intervention. Therefore, there is an urgent need for a generative agent cognitive diagnosis framework that fuses the thinking of teachers to realize fine-grained evolution tracking and explainability enhancement of cognitive state, so as to support cognitive attribution and intervention suggestion generation with teaching logic. SUMMARY

[0003] The embodiment of the present application provides a generative agent cognitive diagnosis framework that fuses the thinking of teachers to realize fine-grained evolution tracking and explainability enhancement of cognitive state, so as to support cognitive attribution and intervention suggestion generation with teaching logic.

[0004] 1. A generative agent cognitive diagnosis framework that fuses the thinking of teachers, characterized by the following steps:

[0005] S1, first, learning data of a learner is obtained through a learner terminal and an item bank server, the ability value of the learner and the difficulty value of the item are calculated according to a cognitive diagnosis model, a large language model trained is used to inject the thinking of teachers into the agent, the learning data set of the learner is input, and a judgment set of the learning state of the learner from the perspective of teachers is output;

[0006] S2, secondly, by constructing long-term and short-term learning behavior sequences through the historical learning data of the learners, comparing the judgment set of the learning state with the current actual answer result to obtain a prediction deviation, and inputting the deviation, the current question information and the short-term behavior sequence into a large language model according to the deviation, the current question information and the short-term behavior sequence, the intelligent agent makes a periodic learning summary of the learning state of the student, and stores the learning summary and the corresponding knowledge point label in the storage space, and regularly abstracts and compresses the stored content;

[0007] S3, finally, inputting the long-term learning behavior sequence, the learning summary in the storage space and the corresponding knowledge point label, and the current question into the large language model, outputting the problem solving strategy and the corresponding answer content that the intelligent agent infers that the student may take in the task; inputting the problem solving strategy corresponding to the question, the corresponding answer content and the current question into the large language model, and outputting the prediction result of the intelligent agent simulating the teacher's answer to the student;

[0008] 2. The generative intelligent agent cognitive diagnosis framework fusing teacher thinking according to claim 1, characterized in that: the learner data is obtained, specifically comprising:

[0009] The learner-specific teacher intelligent agent set: , contains n intelligent agents, each of which corresponds to a specific learner and simulates the role of its exclusive teacher;

[0010] The exercise set: , which is composed of m exercises;

[0011] The knowledge concept set: , covering k target knowledge concepts;

[0012] The answer response set: , used to represent the answer results of the learners to the questions, wherein 1 represents correct answer and 0 represents wrong answer;

[0013] The evaluation behavior of the intelligent agent is modeled as a triple , wherein represents a specific teacher intelligent agent, represents the corresponding exercise, represents the answer result of the learner;

[0014] The test record of the learner ;

[0015] 3. The generative agent cognitive diagnosis framework integrating teacher thinking according to claim 2, wherein the ability value of each learner and the difficulty value of each question are calculated by processing the test records of the learners through the cognitive diagnosis model, and the average ability value of all learners and the average difficulty value of all questions are calculated; a large language model is used as a teacher agent, and the learner ability value, the difficulty value of each question, the average ability value of all learners, the average difficulty value of all questions, and the current time large language model context are used as the input of the large language model, and the output is the judgment set of the learning state of the learner:

[0016]

[0017] wherein, is the large language model integrating teacher thinking guidance prompts in the reasoning process, is the learning ability value of the current learner, is the difficulty value of the current question, is the average learning ability value of all learners, is the average difficulty value of all questions, is the current context of the large language model;

[0018] 4. The generative agent cognitive diagnosis framework integrating teacher thinking according to claim 3, wherein the long-term learning behavior sequence is the entire learning record of the learner between the current time:

[0019]

[0020] wherein represents the question content of the i-th learning task, represents the answer result of the learner, 1 represents correct, and 0 represents error; the short-term learning behavior sequence is the behavior of the learner in the recent interaction:

[0021]

[0022] wherein w is a fixed time window; the prediction result of the learner in the judgment set of the learning state is: , the actual answer result is: , and the prediction deviation is: ; the learner learning state judgment set, the short-term learning behavior sequence, the prediction deviation, and the current question are input into the large language model to obtain the personalized cognitive feedback generated by the agent based on the prediction deviation:

[0023]

[0024] Input personalized cognitive feedback, learner learning state judgment set and question information into a large language model to obtain an agent's summary of the current learning state stage:

[0025]

[0026] Store the summary and the corresponding knowledge point label in the storage space, and regularly compress the stored content to form a memory pool:

[0027]

[0028] 5. The generative intelligent agent cognitive diagnosis framework fused with teacher thinking according to claim 4, characterized in that: inputting the student's entire learning record, the current task, the memory pool in the storage space and the current question into a large language model, outputting the problem solving strategy and the answer content that can be taken for the question:

[0029]

[0030] After obtaining the available problem solving strategy and answer content, further evaluate the student's evaluation ability at the cognitive level:

[0031]

[0032] Among them, represents the teacher's prediction of the student's answer, reflecting the degree of ability matching of the student under the current task.

[0033] Beneficial effects:

[0034] ​​The framework proposes a generative agent-driven teacher thinking cognitive diagnosis framework, which integrates the question, the student's historical behavior and the context information in a complex context, generates the preliminary cognitive state of the student's knowledge state, realizes the cognitive state judgment facing the specific task situation, and provides support for subsequent dynamic adjustment and teaching intervention. Combining the student's short-term answering performance and long-term behavior trajectory, the dynamic tracking and strategy adaptation of the cognitive state evolution process are realized. The system introduces a two-level reflection mechanism: the correction type reflection is used to correct the current prediction error, and the summary type reflection is used to periodically optimize the strategy parameters, simulating the cognitive process of the teacher "observation-reflection-adjustment". Finally, the explanation and cognitive chain generation module is modeled for explainability, and through tracking the behavior path and answering sequence of the student, combined with the language model generation mechanism, the causal reasoning chain between "knowledge state-behavior process-task performance" is automatically constructed. The framework has good cold start ability, can make reasonable judgments even in the absence of complete historical behavior data; has strong explainability, supports precise teaching intervention; and has the ability to distinguish the student's thinking level, which can assist in completing Bloom's cognitive classification, and provides an effective framework for teaching task stratification and teaching design in an intelligent education system. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0036] Figure 1 A generative agent-driven teacher thinking cognitive diagnosis framework flowchart

[0037] Figure 2 A data set statistical chart

[0038] Figure 3 A generative agent-driven teacher thinking cognitive diagnosis framework chart

[0039] Figure 4 A performance comparison chart of different data sets, different models and the present framework

[0040] Figure 5 A cold start performance comparison chart of F1 score and ACC on Mech and Prog data sets

[0041] Figure 6 A predicted Bloom cognitive level confusion matrix chart DETAILED DESCRIPTION

[0042] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0043] Referring to Figure 1 is a generative intelligent agent cognitive diagnosis framework flowchart fusing teacher thinking provided by the embodiments of the present application, and the framework comprises the following steps:

[0044] S1, first, learner terminal and question bank server are used to obtain learner learning data, learner ability value and question difficulty value are calculated according to a cognitive diagnosis model, a large language model is used to train the intelligent agent in the teacher thinking, the learner learning data set is input, and the judgment set of the learner learning state from the teacher's perspective is output;

[0045] S2, secondly, long-term and short-term learning behavior sequences are constructed through the historical learning data of the learner, the judgment set of the learning state is compared with the current actual answering result to obtain a prediction deviation, the large language model is input according to the deviation, current question information and short-term behavior sequence, the intelligent agent is enabled to make a stage learning summary of the student learning state, and the learning summary and the corresponding knowledge point label are stored in a storage space, and the stored content is periodically abstracted and compressed;

[0046] S3, finally, the long-term learning behavior sequence, the learning summary and the corresponding knowledge point label in the storage space, and the current question are input into the large language model, and the problem solving strategy and the corresponding answer content that the intelligent agent infers that the student may take in the task are output; the problem solving strategy corresponding to the question, the corresponding answer content and the current question are input into the large language model, and the prediction result of the intelligent agent simulating the teacher's answer to the student is output;

[0047] It is worth noting that the input data of the present example is the mechanical mechanics (Meth), artificial intelligence (AI) and programming (Prog) three sub-datasets in the large MOOC education resource dataset MOOCCubeX maintained by the Knowledge Engineering Laboratory of Tsinghua University and supported by Xuetang Online, and the detailed statistical information of the three sub-datasets is shown in Figure 2, the intelligent agent in the experiment is constructed by DeepSeek-R1-7B and OpenAI's API service using GPT-4o-mini and GPT-4o, the task records of 100 learners are simulated under the GPT-4o configuration, the temperature parameter of GPT is set to 0 to avoid randomness, GPT-4o-mini is used as the large language model, and the data is randomly divided into a training set (70%), a validation set (10%), and a test set (20%).

[0048] Referring to Figure 3 is the complete framework diagram of the framework, the framework processes the test records of learners by using a cognitive diagnosis model tool, the cognitive diagnosis tool used in the present example is item response theory (IRT) to calculate the ability value of each learner and the difficulty value of each question, and to calculate the average ability of all learners and the average difficulty of all questions, and to perform comparative experiments; a large language model is used as a teacher agent, the ability value of the learner, the difficulty value of each question, the average ability value of all learners, the average difficulty value of all questions, and the context of the large language model at the current time are used as the input of the large language model, and the output is a judgment set of the learning state of the learner:

[0049]

[0050] Among them, is a large language model that integrates teacher-oriented prompts in the reasoning process, is the learning ability value of the current learner, is the difficulty value of the current question, is the average learning ability value of all learners, is the average difficulty value of all questions, is the current context of the large language model;

[0051] The long-term learning behavior sequence is the entire learning record of the learner between the current time:

[0052]

[0053] Among them represents the content of the i-th learning task, represents the answer result of the learner, 1 indicates correct, and 0 indicates error; the short-term learning behavior sequence is the behavior of the learner in recent interactions:

[0054]

[0055] Among them, w is a fixed time window; the prediction result of the learner's current task in the judgment set of the learning state is: , and the actual answer result is: , the prediction deviation is: ; the learner learning state judgment set, the short-term learning behavior sequence, the prediction deviation and the current question are input into the large language model to obtain personalized cognitive feedback generated by the agent based on the prediction deviation:

[0056]

[0057] The personalized cognitive feedback, the learner learning state judgment set and the question information are input into the large language model to obtain the agent's stage learning summary of the current learning state:

[0058]

[0059] The learning summary and the corresponding knowledge point label are stored in the storage space, and the storage content is periodically summarized and compressed to form a memory pool:

[0060]

[0061] The student's entire learning record, the current task, the memory pool in the storage space and the current question are input into the large language model, and the solvable problem solving strategy and the answer content are output:

[0062]

[0063] After obtaining the solvable problem solving strategy and the answer content, the student's cognitive level evaluation ability is further evaluated:

[0064]

[0065] Among them, represents the prediction result of the teacher on the student's answer, reflecting the ability matching degree of the student under the current task.

[0066] Referring to Figure 4The proposed framework is compared with various mainstream baseline models, including Deep Knowledge Tracing (DKT), Targeted Hierarchical Knowledge Tracing (THKT), Item Response Theory (IRT), Multidimensional Item Response Theory (MIRT), Deterministic Inputs Noisy "And" Gate (DINA), Neural Cognitive Diagnosis Model (NCDM), and Hierarchical Cognitive Diagnosis (HCD) on three real educational datasets: Mech, AI, and Prog. The evaluation metrics include accuracy (ACC) and F1-score (F1-Score). By evaluating the prediction tasks on multiple datasets, the effectiveness of the framework in modeling learners' cognitive states can be indirectly verified from multiple dimensions. This quantitative evaluation framework provides a solid foundation for further research on intelligent agents in cognitive diagnosis, adaptive teaching, and behavior interpretation. From the comparison results, the framework shows significant advantages in each task, outperforming traditional knowledge tracing (KT) frameworks and cognitive diagnosis (CD) models. Among them, the framework using the GPT-4o-mini as a large language model achieves the best results in four of the six indicators, especially on the Prog dataset, with an accuracy of 83.26% and an F1-Score of 84.93%, significantly higher than other models (P < 0.05). In addition, it is worth noting that the framework using GPT-4o (100) as a large language model only models and analyzes 100 randomly selected students driven by GPT-4o, aiming to verify the effectiveness under the premise of controlling costs. Despite the relatively small amount of data, it still achieves the highest AI-ACC (83.89%) and AI-F1-Score (81.31%) on the AI dataset. This result not only confirms the robustness of the model in small sample scenarios, but also further emphasizes the key role of high-performance large language models in teacher agents. Further observation shows that the ability of large language models driving teacher agents has a key impact on overall performance. Large language models of different sizes and capabilities exhibit different levels of support in reflection generation and cognitive modeling, highlighting the potential of large models in understanding, attribution, and decision-making generation. This shows that building a teaching intelligent agent with human-like cognitive abilities not only requires reasonable system structure design, but also fully leverages the advantages of language models in multi-level cognitive reasoning.

[0067] See Figure 5Experiments are conducted to evaluate the generalization and inference ability of different models in the cold-start environment. To simulate the cold-start environment, the student data in Mech and Prog datasets are processed to retain only the first few interactions of each learner in the domain for model initialization, and the remaining interactions are used for performance evaluation. In this extreme data scarcity condition, models cannot rely on complete student behavior sequences and must complete cognitive state modeling and performance prediction through limited information, which puts higher requirements on the inference and generalization ability of the model. As shown in Figure 5 The performance of the framework on the Mech and Prog datasets is significantly better than that of traditional models, verifying its ability to model student cognitive states and make performance predictions in the zero-interaction condition. Second, it can be observed that the overall performance of the knowledge tracing model DKT is better than that of IRT and NCDM, which may be due to its basic modeling paradigm that emphasizes the time series characteristics of student behavior, thus having stronger generalization ability. In contrast, cognitive diagnosis models such as IRT and NCDM usually rely on sufficient training of each student to show their advantages, so they are less adaptable in cold-start scenarios. The performance of the framework on the two datasets shows that its performance does not decline as sharply as other models, but remains highly consistent, indicating that simulating teacher thinking through large language models can effectively enhance the robustness and inference ability of models in cold-start environments.

[0068] See Figure 6 The framework's ability to simulate teacher thinking at the cognitive level is demonstrated through a Bloom cognitive classification prediction experiment based on the Prog programming education dataset. The dataset consists of answer records of programming learners in a real environment, and each question is labeled with a cognitive level according to the Bloom taxonomy, covering memory, understanding, application, analysis, evaluation, and creation. The answer data of 20 learners were randomly selected as the test set, and an additional task was introduced: the model was asked to predict the Bloom cognitive level label of each question to measure the framework's ability to understand and judge the cognitive requirements of the question. Simulating the classification process of teachers on the cognitive attributes of questions from the model level further demonstrates the teaching explanation potential of the framework. Figure 6The prediction confusion matrix results of this task are shown, from which it can be observed that the model's predictions are relatively accurate at the main cognitive levels. At the four levels of understanding, application, analysis, and evaluation, the model's prediction accuracy rates are 83.5%, 82.1%, 84.6%, and 82.4%, respectively, indicating that the framework can effectively capture the cognitive characteristics corresponding to the questions, and this result can effectively reduce the cost of expert labeling. Using the framework proposed in this paper can achieve results comparable to expert labeling. Secondly, there is a certain degree of confusion between adjacent levels, for example, "understanding" is easily misjudged as "application" (7.6%), and "application" is easily misjudged as "analysis" (7.6%), reflecting the model's challenges in dealing with fuzzy cognitive stages, and indirectly conforming to the structural characteristics of Bloom's hierarchical progression. Finally, there are few misjudgments between distant levels, for example, it is rare for "understanding" to be predicted as "evaluation" or "creation", indicating that the model maintains a good understanding of the overall cognitive level structure.

Claims

1. A generative intelligent agent cognitive diagnostic framework integrating teacher thinking, characterized in that: Includes the following steps: S1. First, learner learning data is obtained through learner terminals and question bank servers. Learner ability value and question difficulty value are calculated according to cognitive diagnostic model. Teacher thinking is injected into the agent using the trained large language model. The learner learning dataset is input and the judgment set of learner learning status from the teacher's perspective is output. S2. Secondly, long-term and short-term learning behavior sequences are constructed through the learner's historical learning data. The judgment set of the learning state is compared with the current actual answer result to obtain the prediction deviation. Based on the deviation, the current question information, and the short-term behavior sequence, the data is input into the large language model so that the agent can make a phased learning summary of the student's learning state. The learning summary and its corresponding knowledge point tags are stored in the storage space, and the stored content is periodically summarized and compressed. S3. Finally, input the long-term learning behavior sequence, the learning summary in the storage space and its corresponding knowledge point labels, and the current question into the large language model, and output the problem-solving strategy that the student may take in the task and the corresponding answer content inferred by the agent; input the problem-solving strategy of the corresponding question, the corresponding answer content and the current question into the large language model, and output the prediction result of the agent simulating the teacher's answer to the student.

2. The generative intelligent agent cognitive diagnostic framework integrating teacher thinking as described in claim 1, characterized in that: Acquiring learner data specifically includes: A collection of teacher-specific intelligent agents for learners: It contains n agents, each agent corresponding to a specific learner, simulating their dedicated teacher role; Exercise set: It consists of m practice questions; Collection of knowledge concepts: It covers k target knowledge concepts; Response set: This is used to represent the learner's answer to the question, where 1 indicates a correct answer and 0 indicates an incorrect answer; The agent's evaluation behavior is modeled as a triple. ,in Represents a specific teacher intelligent agent. This indicates the corresponding practice questions. Indicates the learner's answer result; learner's test records .

3. The generative intelligent agent cognitive diagnostic framework integrating teacher thinking as described in claim 2, characterized in that: By processing learners' test records using a cognitive diagnostic model, the ability value of each learner and the difficulty value of each question are calculated, and the average ability of all learners and the average difficulty of all questions are also calculated. Using a large language model as the teacher agent, the learner's ability score, the difficulty score of each question, the average ability score of all learners, the average difficulty score of all questions, and the current context of the large language model are taken as input to the large language model. The output is a set of judgments on the learner's learning status. in, It is a large language model that incorporates teacher-guided prompts during the reasoning process. This is the learner's current learning ability score. This is the difficulty level of the current question. It is the average learning ability value of all learners. It is the average difficulty value of all questions. It is the current context of the large language model.

4. The generative intelligent agent cognitive diagnostic framework integrating teacher thinking as described in claim 3, characterized in that: Long-term learning behavior sequence is the learner's complete learning record up to the current time: in Describe the content of the i-th learning task. The short-term learning behavior sequence represents the learner's recent interactions: 1 indicates correct, 0 indicates incorrect. Where w is a fixed time window; the prediction result of the learning state judgment set for the learner's current task is: The actual answer was: The prediction bias is: By inputting the learner's learning state judgment set, short-term learning behavior sequence, prediction bias, and current question into the large language model, personalized cognitive feedback generated by the agent based on the prediction bias is obtained: By inputting personalized cognitive feedback, learner learning status judgment set, and question information into the large language model, we can obtain the agent's phased learning summary of the current learning status: The learning summaries and corresponding knowledge point tags are stored in the storage space, and the stored content is periodically summarized and compressed to form a memory pool:

5. The generative intelligent agent cognitive diagnostic framework integrating teacher thinking as described in claim 4, characterized in that: Input all student learning records, the current task, the memory pool in the storage space, and the current question into the large language model, and output the possible problem-solving strategies for the question. and answer content : After obtaining the available problem-solving strategies and answer key, further assess the student's cognitive assessment abilities: in, This indicates the teacher's prediction of the student's answer, reflecting the degree to which the student's ability matches the current task.

Images