Simulation method and system for students with different cognitive levels based on large language model

Abstract

The application discloses a simulation method and system for students with different cognitive levels based on a large language model. The method constructs a cognitive prototype based on the past learning records of students, and clearly expresses the mastery of students on different knowledge concepts through a knowledge graph. This cognitive prototype can be used to accurately predict the performance of students in new tasks, including whether they can solve problems and specific errors they may make. Meanwhile, the application introduces a self-evaluation-self-optimization cycle process based on beam search to iteratively generate student answers consistent with predicted behavior, thereby achieving more realistic, natural and cognitively consistent student simulation without additional training. Related results show that the student simulation framework proposed by the application, which is driven by a cognitive prototype in stages, is superior to existing mainstream methods in multiple dimensions and can more realistically reproduce the behavior characteristics of students with different cognitive levels.

Description

Technical Field

[0001] This invention belongs to the field of generative language models, specifically relating to a simulation method and system for students with different cognitive levels based on a large language model. Background Technology

[0002] In the wave of AI-driven educational transformation, Large Language Models (LLMs) have become a key technology, widely applied in scenarios such as personalized tutoring, curriculum design, and adaptive assessment. One important application is "student simulation"—by having models act as students, researchers can evaluate teaching strategies, test the effectiveness of intelligent tutoring systems, and drive the development of educational AI tools at low cost.

[0003] However, to achieve realistic and effective student simulations, the model must be able to reflect the differences in students' cognitive abilities. An ideal simulation should not only showcase the near-perfect problem-solving process of high-achieving students but also naturally reproduce the common errors and cognitive biases of lower-achieving students. In reality, however, current large language models do not perform ideally in this regard. Specifically, current large language models generally tend to generate correct answers with a high level of cognitive understanding, failing to realistically reproduce the common errors and behavioral characteristics of lower-achieving students, resulting in simulation results lacking cognitive diversity and authenticity.

[0004] The crux of the problem lies in the fact that most existing large language models are trained as "assistants in providing accurate answers." They naturally tend to generate accurate, standardized, and even overly advanced responses. This generation method ignores the error patterns that naturally occur during students' learning process, leading to distorted simulation results, especially evident in simulations of lower-level students. For example, models often overestimate the abilities of elementary school students and fail to reproduce their expected error-prone behaviors, rendering the simulation results unreliable.

[0005] Although some studies have attempted to fine-tune models on erroneous data to enable them to learn and reproduce common errors, these methods have two serious drawbacks: first, they easily introduce erroneous knowledge into the model, reducing its overall performance; and second, they ignore the individual differences in student errors and cannot generate personalized errors based on the cognitive state of different students. Summary of the Invention

[0006] The purpose of this invention is to solve the above-mentioned problems in the prior art and to provide a simulation method and system for students with different cognitive levels based on a large language model.

[0007] The specific technical solution adopted in this invention is as follows:

[0008] In a first aspect, the present invention provides a simulation method for students with different cognitive levels based on a large language model, comprising:

[0009] S1. Obtain the historical answer record sequence of the target student, where each answer record includes the question stem text, the student's solution process text, and the student's solution analysis;

[0010] S2. Information is extracted from the historical answer record sequence using a large language model, and a cognitive prototype of the target student is constructed. The cognitive prototype is a knowledge graph, whose nodes are the knowledge points covered by the question stem text in all historical answer records. The edges between nodes are constructed according to the relationship between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the student's answer analysis, which is used to represent the target student's global cognitive state of the knowledge point.

[0011] S3. For the target question text, use the big language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype. Then input the retrieval results into the big language model to predict the possible erroneous behavior descriptions of the target student when answering the current question.

[0012] S4. Based on the target question text and the description of the error behavior, use a bundle search-based self-evaluation-self-optimization loop strategy to predict the target student's answer process text through a large language model.

[0013] As a preferred embodiment of the first aspect above, the question stem text and student answer process text in the target student's historical answer record sequence are extracted from an online learning platform, and the student answer analysis is generated manually based on expert knowledge.

[0014] As a preferred embodiment of the first aspect above, the specific method for constructing the cognitive prototype of the target student is as follows:

[0015] S21. For each historical answer record in the historical answer record sequence, the knowledge points covered and their detailed descriptions are extracted from the question text using a large language model. Then, the relationships between the extracted knowledge points are extracted to establish a local subgraph with knowledge points as nodes and relationships as edges. At the same time, diagnostic analysis is performed on the student's answer process text and student answer analysis to obtain the target student's mastery level score and detailed description of mastery level for each knowledge point, and this is recorded as the target student's local cognitive state for the corresponding knowledge point.

[0016] S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges, and comprehensively analyze the local cognitive states recorded in the merged nodes through a large language model to generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, thus forming the target student's cognitive prototype.

[0017] As a preferred embodiment of the first aspect mentioned above, for any two knowledge points A and B, the relationship between the knowledge points is divided into four types: A is a prerequisite for B, A is used for B, A is a special case of B, and A is a part of B.

[0018] As a preferred embodiment of the first aspect, when integrating the local subgraphs, all knowledge points involved in the local subgraphs are extracted, each pair of knowledge points is traversed in pairs, and the similarity of the knowledge points is calculated based on the detailed description of their respective knowledge points. If the similarity of the knowledge points exceeds the threshold, the pair of knowledge points is merged. At the same time, if there are two edges and their corresponding two nodes are merged, then these two edges are also merged.

[0019] As a preferred embodiment of the first aspect mentioned above, the global cognitive state is a text describing the target student's comprehensive mastery of knowledge points, generated by a large language model.

[0020] As a preferred embodiment of the first aspect above, the specific method for predicting the target student's solution process text using the beam search-based self-evaluation-self-optimization loop strategy is as follows:

[0021] S41. Input the target question text and the description of the error behavior into the large language model to generate the initial solution process text;

[0022] S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate each of them, and the solution process text that best matches the description of the error behavior is selected to complete the self-optimization. Then, the selected solution process text is input into the large language model for optimization. The self-evaluation and self-optimization loop is iteratively executed. After the iteration terminates, the final solution process text is obtained as the prediction result.

[0023] Secondly, this invention provides a simulation system for students with different cognitive levels based on a large language model, comprising:

[0024] The answer record acquisition module is used to acquire the historical answer record sequence of the target student, where each answer record includes the question stem text, the student's solution process text, and the student's solution analysis;

[0025] The cognitive prototype construction module is used to extract information from the historical answer record sequence using a large language model and construct the cognitive prototype of the target student. The cognitive prototype is a knowledge graph, whose nodes are the knowledge points covered by the question stem text in all historical answer records. The edges between nodes are constructed according to the relationship between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the student's answer analysis, which is used to represent the target student's global cognitive state of the knowledge point.

[0026] The error behavior description module is used to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the cognitive prototype of the target student for the target question text using a large language model. The retrieval results are then input into the large language model to predict the error behavior description that the target student may make when answering the current question.

[0027] The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the error behavior, using a bundle search-based self-evaluation-self-optimization loop strategy and a large language model.

[0028] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the simulation method for students with different cognitive levels based on a large language model as described in any of the first aspects above.

[0029] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0030] The memory is used to store computer programs;

[0031] The processor is configured to, when executing the computer program, implement the simulation method for students with different cognitive levels based on a large language model as described in any of the first aspects above.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] This invention constructs a cognitive prototype based on students' past learning records, explicitly expressing students' mastery of different knowledge concepts through a knowledge graph. This cognitive prototype can be used to accurately predict students' performance in new tasks, including whether they can solve problems and the specific mistakes they might make. Simultaneously, this invention introduces a self-evaluation-self-optimization loop process based on bundle search to iteratively generate student solutions consistent with predicted behavior, thereby achieving a more realistic, natural, and cognitively consistent student simulation without additional training. Experimental results fully demonstrate that the phased, cognitive prototype-driven student simulation framework proposed in this invention outperforms existing mainstream methods in terms of "prediction accuracy," "behavioral rationality," and "response naturalness," and can more realistically reproduce the behavioral characteristics of students at different cognitive levels. Attached Figure Description

[0034] Figure 1 A schematic diagram illustrating the steps of a simulation method for students with different cognitive levels based on a large language model;

[0035] Figure 2 This is an exemplary construction process for a cognitive prototype;

[0036] Figure 3 A diagram showing the modular composition of a simulation system for students with different cognitive levels based on a large language model;

[0037] Figure 4 This is a schematic diagram of the structure of a computer electronic device. Detailed Implementation

[0038] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0039] like Figure 1 As shown, in a preferred embodiment of the present invention, a simulation method for students with different cognitive levels based on a large language model is provided, which includes:

[0040] S1. Obtain the sequence of historical answer records of the target student, where each answer record includes the question text, the student's solution process text, and the student's solution analysis.

[0041] It should be noted that the question stem text and student solution process text in the above-mentioned target students' historical answer record sequence can be extracted from the online learning platform. If the student solution analysis is available on the learning platform, it can be extracted as well; otherwise, it can be generated manually based on expert knowledge.

[0042] It should be noted that the historical answer record sequence constructed in step S1 of this invention differs from existing student answer record datasets. Most existing student answer record datasets typically represent each question using an index and only include a determination of whether the student's answer is correct, lacking crucial information such as the question stem text and student solution analysis necessary for student simulation. Therefore, in this embodiment of the invention, a large-scale student answer record sequence dataset is acquired and constructed based on an online programming platform for student simulation research based on a large language model. This dataset contains a series of Python programming answer record sequences from different students. Each student's answer record sequence contains a series of answer records for different programming questions. Each answer record includes the question stem text t, the student's solution process text s, and the student's solution analysis b. In this embodiment of the invention, the specific student answer record sequence sample construction process is as follows:

[0043] S11. Obtain the initial student answer record sequence from the online programming platform. Each initial answer record contains the question stem text t and the student's solution process text s.

[0044] S12. A human evaluates each student's answer in the initial answer record and compiles a student answer analysis. In this embodiment, 10 domain experts are convened to check and evaluate each student's answer in the initial answer record and compile a student answer analysis b, which includes a determination of whether the student answered the question correctly; if the student did not answer the question correctly, it also includes a detailed explanation of the student's mistake.

[0045] S2. Information is extracted from the historical answer record sequence using a large language model, and a cognitive prototype of the target student is constructed. The cognitive prototype is a knowledge graph, whose nodes are the knowledge points covered by the question stem text in all historical answer records. The edges between nodes are constructed according to the relationship between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the student's answer analysis, which is used to represent the target student's global cognitive state of the knowledge point.

[0046] It should be noted that step S2 of this invention involves extracting a cognitive prototype in natural language form from the student's historical answer record sequence. Previous student cognitive diagnosis methods typically only use question indices to represent the answer sequence, without considering the question text or other information. Furthermore, they rely on implicit, parameterized knowledge from deep networks to represent the student's cognitive state, resulting in a lack of interpretability in the diagnosis. Therefore, this invention utilizes a large language model to extract the student's cognitive prototype from the text-rich historical answer record sequence. This cognitive prototype, combined with a knowledge graph, expresses the student's mastery of each knowledge point in natural language form, thus reflecting the student's cognitive state. Specifically, the cognitive prototype of the target student can be constructed using the following specific method:

[0047] S21. For each historical answer record in the historical answer record sequence, the knowledge points covered and their detailed descriptions are extracted from the question text using a large language model. Then, the relationships between the extracted knowledge points are extracted to establish a local subgraph with knowledge points as nodes and relationships as edges. At the same time, diagnostic analysis is performed on the student's answer process text and student answer analysis to obtain the target student's mastery level score and detailed description of mastery level for each knowledge point, and this is recorded as the target student's local cognitive state for the corresponding knowledge point.

[0048] S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges (the local cognitive state recorded in the nodes and the corresponding relationships of the edges are also merged), and comprehensively analyze the local cognitive state recorded in the merged nodes through a large language model to generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, thus forming the target student's cognitive prototype.

[0049] In embodiments of the present invention, the construction process of the cognitive prototype of the target student can be considered as two steps: A) Local cognitive state extraction, corresponding to S21, used to extract the student's local cognitive state from each historical answer record; B) Global cognitive state analysis, corresponding to S22, used to analyze all local cognitive states to obtain the global cognitive state after all historical answer records have been processed. Figure 2 As shown in the figure, the construction process of the cognitive prototype is illustrated. The specific implementation process of these two stages will be described in detail below with reference to this example.

[0050] A) Extraction of local cognitive states

[0051] A1) Traverse all historical answer records in the historical answer record sequence and extract relevant knowledge points from the current historical answer record.

[0052] It should be noted that in step A1) of this invention, given the current historical answer record P i =(t i ,s i ,b i ), where t i For the question stem text, s i For students' solution process text, b i To provide students with answers and analysis, a large language model was used to extract several knowledge points from the question text. i,1 ,v i,2 [,…] and a detailed description of the knowledge point. These knowledge points will serve as nodes in the local subgraph corresponding to the current historical answer record, and will also serve as partial nodes in the knowledge graph of the student's cognitive prototype during subsequent subgraph integration.

[0053] A2) Extracts relationships from the knowledge points extracted in A1) to obtain the connections between the knowledge points.

[0054] It should be noted that in the steps of this invention, for any two knowledge points A and B, four types of knowledge point associations are predefined to standardize the extraction of knowledge point associations. These are: A is a prerequisite for B, A is used for B, A is a special case of B, and A is a part of B. The large language model is used to analyze the possible associations from the knowledge points extracted in step S21, and each association is classified into one of the above four types to obtain the associations between knowledge points [e]. i,1 ,e i,2 The connections between these knowledge points will be constructed as edges in the local subgraph corresponding to the current historical answer record, and will also serve as partial edges in the knowledge graph of the student's cognitive prototype during subsequent subgraph integration.

[0055] A3) Analyze the knowledge points extracted from A1) to determine the students' local cognitive state of the knowledge points and add them to the local cognitive state database of the knowledge points.

[0056] It should be noted that students' performance on the same knowledge point may differ across different answer records. Therefore, each knowledge point uses a local cognitive state database to record the student's local cognitive state regarding the same knowledge point across different historical answer records. Step A3) of this invention aims to analyze the student's local mastery of the knowledge point from their current historical answer record performance, thus constructing the student's local cognitive state. For each knowledge point v extracted in A1),... i,* Using a large language model, we can diagnose students' local cognitive states regarding a particular knowledge point from their solution process text and solution analysis. i,*The diagnostic method is as follows: First, assess the student's partial mastery of the knowledge point, classifying it into "Good" or "Bad." "Good" indicates that the student has a good grasp of the knowledge point, while "Bad" indicates that the student's partial mastery is poor. Then, provide a detailed explanation of this assessment. Following this, assess the student's partial cognitive state (c) for each knowledge point. i,* This information will be added to the local cognitive state database for the corresponding knowledge point as a record. In this way, the student's local cognitive state is extracted from their current historical answer records.

[0057] Therefore, for each current historical answer record during the traversal process, we can use knowledge points as nodes and extract relationships to determine edges, thereby constructing a local subgraph corresponding to the current historical answer record. Each node in the local subgraph records the target student's mastery level score and detailed description of mastery level for each knowledge point through the local cognitive state database, which represents the target student's local cognitive state for the corresponding knowledge point.

[0058] B) Global Cognitive State Analysis

[0059] B1) Remove duplicate nodes and edges from the knowledge graph constructed in the above steps to obtain a knowledge graph without redundancy.

[0060] It should be noted that different historical answer records may overlap when extracting knowledge points and their relationships; for example, a certain knowledge point may appear in multiple historical answer records. Therefore, the steps of this invention aim to merge highly similar knowledge points and their relationships, removing redundancy in the knowledge graph. Specifically, firstly, all knowledge points involved in the local subgraphs are extracted, and all knowledge points are paired up. Each pair of knowledge points contains a detailed description of the knowledge point extracted in step A1). For each pair of knowledge points, the similarity of their detailed descriptions can be calculated using a large language model. If the similarity is higher than a set threshold δ1, the pair of knowledge points is considered to be the same knowledge point and needs to be merged. For all redundant knowledge points, their local cognitive state databases are merged. Similarly, if both nodes of two edges are considered to be the same knowledge point, these two edges are also merged, and the final relationship type is the one with the highest frequency of occurrence. In this embodiment, δ1 is set to 0.85.

[0061] B2) By comprehensively analyzing the content of the local cognitive state database of each knowledge point in the merged global knowledge graph, the student's global cognitive state for each knowledge point is obtained, thereby constructing a complete student cognitive prototype.

[0062] It should be noted that the local cognitive state obtained in step A3) of this invention will vary depending on different historical answer records. Therefore, it is necessary to comprehensively analyze all the student's local cognitive states to obtain a unified global cognitive state. Therefore, step B1) should be performed after all the student's historical answer records have been processed through steps A1)-A3). Figure 2 As shown, a comprehensive analysis of the frequency of "Good" and "Bad" and related descriptions in the local cognitive state database of each knowledge point is conducted using a large language model. This analysis yields the student's overall mastery of the knowledge point, which is then used as the global cognitive state to construct a complete student cognitive prototype. It is important to note that the global cognitive state is a text describing the target student's overall mastery of the knowledge point, generated by the large language model, and is expressed in natural language, such as "very good mastery" and "very poor mastery."

[0063] S3. For the target question text, use the big language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype. Then, input the retrieval results into the big language model to predict the possible erroneous behavior descriptions of the target student when answering the current question.

[0064] It should be noted that the student simulation strategy and the student cognitive diagnosis strategy have completely opposite processes. Student cognitive diagnosis requires analyzing the student's behavior when solving the problem based on the student's existing solution process text, generating a solution analysis; however, student simulation first needs to predict what behaviors the student will exhibit on the problem, including whether they will make mistakes and where those mistakes will occur, and then simulate the student's solution process text based on the predicted behaviors. Therefore, step S3 of this invention is to predict the student's problem-solving behavior based on the student cognitive prototype obtained in S2, including the following steps:

[0065] S31. For each question to be predicted, use the student's cognitive prototype to retrieve the most relevant knowledge points to the question stem and the student's overall cognitive state of these knowledge points, as the basis for subsequent predictions.

[0066] It should be noted that previous methods typically relied on directly retrieving the historical answer record most similar to the question stem of the current simulated answer record as the basis for subsequent predictions. However, this method based on question stem similarity is highly susceptible to the influence of the question stem wording, retrieving historical answer records with similar question stems but completely different core knowledge points. This fails to reflect the student's mastery of the questions in the simulated answer record, thus misleading subsequent behavioral predictions. Therefore, this invention focuses on knowledge points and utilizes the student's cognitive prototypes to retrieve the student's overall cognitive state.

[0067] Specifically, for the current question to be predicted, this invention can utilize its question stem text t j By using a large language model to search the knowledge graph of students' cognitive prototypes, a set of knowledge points [v1, v2, ... v] most relevant to the question is obtained. p ], and extract the corresponding global cognitive states of students [C1, C2, ... C] from the student cognitive prototype. p The p-value is a hyperparameter and can be adjusted according to actual conditions. In this embodiment, the p-value is 5.

[0068] S32. Using students' overall cognitive state regarding the most relevant knowledge points as a reference, a large language model is used to predict potential errors students may make when answering the current question. This includes whether it will make mistakes and what specific mistakes it might make.

[0069] S4. Based on the target question text and the description of the error behavior, use a bundle search-based self-evaluation-self-optimization loop strategy to predict the target student's answer process text through a large language model.

[0070] It should be noted that, because current large language models are typically trained to produce correct and reliable outputs, they tend to output correct content. Therefore, when simulating a student's problem-solving process text based on descriptions of student behavior, especially when these descriptions contain potential errors, the initial student solution text generated by the large language model may not accurately reflect these errors. Therefore, this invention proposes a process that allows the large language model to continuously self-evaluate and self-optimize, and introduces a beam search mechanism to increase the sampling space of the model's output, thereby increasing the probability of successful simulation. Specifically, the method for predicting the target student's solution text using the aforementioned beam search-based self-evaluation and self-optimization loop strategy can be implemented as follows:

[0071] S41. Input the target question text and the description of the error behavior into the large language model to generate the initial solution process text;

[0072] S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate each of them, and the solution process text that best matches the description of the error behavior is selected to complete the self-optimization. Then, the selected solution process text is input into the large language model for optimization. The self-evaluation and self-optimization loop is iteratively executed. After the iteration terminates, the final solution process text is obtained as the prediction result.

[0073] Therefore, the aforementioned self-evaluation and self-optimization require continuous iteration. Assuming this process involves L iterations, the specific number of iterations L can be adjusted according to actual needs; in this embodiment, L is taken as 3. In this embodiment of the invention, each iteration includes the following steps:

[0074] C1) Given the question stem text t of the current question to be predicted. j and predicted student behavior Generate initial student answer process text using a large language model

[0075] C2) Based on the solution process text generated in the previous iteration (selected during the first iteration) We use a large language model for optimization and sample several optimized solution process texts.

[0076] It should be noted that the purpose of sampling multiple optimized solution process texts in step C2) of this invention is to increase the sampling space, thereby increasing the probability that the sampling results accurately match the predicted student behavior. Specifically, let the current iteration number be l (1≤l≤L), and the solution process text generated in the previous iteration be... This invention samples B optimized solution process texts based on a large model. In this embodiment, B is set to 2.

[0077] C3) Use a large language model to evaluate and score several solution process texts in this iteration, and select the solution process text with the highest score as the final solution process text for this iteration.

[0078] It should be noted that step C3) of this invention aims to utilize the self-evaluation ability of the large language model to select the solution process text that best matches the predicted student behavior description, thereby reducing the sampling space in subsequent iterations. Specifically, this step utilizes the large language model to analyze the solution process text sampled in C2). Each solution text is evaluated individually, primarily assessing whether it strictly matches the predicted student behavior and accurately reflects the predicted student behavior description. Based on this evaluation, each solution text is scored between 0 and 1. Finally, the solution text with the highest score is selected as the final solution text for this iteration.

[0079] Furthermore, in the embodiments of the present invention, two iteration termination conditions are set in the above-mentioned iterative process: the first is reaching the set upper limit of the number of iterations L; the second is that the self-evaluation score corresponding to the final solution process text of a certain iteration exceeds the set threshold δ2. The iteration terminates when either of the two conditions is met. In this embodiment, δ2 is set to 0.9. The solution text generated at the time of iteration termination is the final simulated student solution text.

[0080] In summary, the core of this invention is to construct a cognitive prototype based on students' past learning records, and to explicitly express students' mastery of different knowledge concepts through a knowledge graph. This cognitive prototype can be used to accurately predict students' performance in new tasks, including whether they can solve problems and the specific mistakes they might make. Simultaneously, by introducing a self-evaluation-self-optimization loop process based on bundle search, iteratively generating student solutions consistent with predicted behavior, a more realistic, natural, and cognitively consistent student simulation is achieved without the need for additional training.

[0081] It should be noted that the method steps shown in S1 to S4 above can essentially be implemented in the form of a computer program or functional module.

[0082] Therefore, based on the same inventive concept, the present invention also provides a simulation system for students with different cognitive levels based on a large language model, corresponding to the simulation method for students with different cognitive levels based on a large language model provided in the above embodiments, which includes:

[0083] The answer record acquisition module is used to acquire the historical answer record sequence of the target student, where each answer record includes the question stem text, the student's solution process text, and the student's solution analysis;

[0084] The cognitive prototype construction module is used to extract information from the historical answer record sequence using a large language model and construct the cognitive prototype of the target student. The cognitive prototype is a knowledge graph, whose nodes are the knowledge points covered by the question stem text in all historical answer records. The edges between nodes are constructed according to the relationship between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the student's answer analysis, which is used to represent the target student's global cognitive state of the knowledge point.

[0085] The error behavior description module is used to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the cognitive prototype of the target student for the target question text using a large language model. The retrieval results are then input into the large language model to predict the error behavior description that the target student may make when answering the current question.

[0086] The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the error behavior, using a bundle search-based self-evaluation-self-optimization loop strategy and a large language model.

[0087] Furthermore, based on the same inventive concept, such as Figure 4 As shown, the present invention also provides a computer electronic device corresponding to the simulation method for students with different cognitive levels based on a large language model provided in the above embodiments, which includes a memory and a processor;

[0088] The memory is used to store computer programs;

[0089] The processor is configured to implement, when executing the computer program, the simulation method for students with different cognitive levels based on a large language model as described above.

[0090] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0091] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to a simulation method for students with different cognitive levels based on a large language model. The storage medium stores a computer program, which, when executed by a processor, can realize the simulation method for students with different cognitive levels based on a large language model as described above.

[0092] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the simulation method for students with different cognitive levels based on a large language model as described above.

[0093] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S4.

[0094] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0095] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0096] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0097] The present invention will further demonstrate the detailed implementation process and technical effects of the simulation method for students with different cognitive levels based on a large language model, as described in steps S1 to S4 above, on a specific dataset, so as to facilitate understanding of the essence of the present invention.

[0098] Example

[0099] The steps in this embodiment are the same as the simulation method for students with different cognitive levels based on the large language model shown in steps S1 to S4 above, and will not be repeated here. The main focus is on showing the specific dataset, some specific parameter settings and implementation results of this embodiment.

[0100] To verify the effectiveness of this embodiment, the present invention obtains initial answer records (including question stem text and student solution process text) based on an online programming platform, and convenes 10 domain experts to check and evaluate the student solutions in each initial answer record, and writes student solution analysis b, which includes a determination of whether the student answered the question correctly; if the student did not answer the question correctly, it also includes a detailed explanation of the student's mistake. Thus, a novel large-scale student answer record sequence dataset named Student_100 is constructed for student simulation research based on a large language model. This dataset contains Python programming answer record sequences of 100 students, with each student's answer record sequence containing 50 answer records. The earliest 40 answer records are divided into a historical answer record sequence, and the latest 10 answer records are divided into a sequence of answer records to be simulated. Each answer record contains question stem text t, student solution process text s, and student solution analysis b. The sequence of answer records to be simulated, consisting of the latest 10 answer records, is used to measure the performance of each method in student simulation tasks.

[0101] This invention selects four mainstream large-scale language models for student simulation experiments, including LLaMA-3.3-70B-Instruct, Claude-3.5-Sonnet, GPT-3.5, and GPT-4o, and conducts a systematic comparison with existing student simulation methods. Considering that the student simulation framework proposed in this invention consists of two key stages—problem-solving behavior prediction and solution process simulation—we introduce comparative methods for each stage and adopt a phased evaluation strategy to comprehensively measure the performance of each method in student simulation tasks.

[0102] Phase 1: Behavior Prediction

[0103] This stage aims to predict students' performance when faced with new tasks, that is, to determine whether they can solve the problems correctly and to provide behavioral descriptions that correspond to their abilities. We designed the following five comparison methods:

[0104] Random: Randomly selects a task from a student's past records as a reference;

[0105] Similarity: Select the historical task most similar to the current task text for prediction;

[0106] Level: Estimate students' ability level based on their accuracy in answering history questions;

[0107] Level+Random: Combining student ability estimation with random retrieval strategies;

[0108] Level+Similarity: Combining student ability estimation with similar task retrieval strategies;

[0109] Prototype Mapping (the method of this invention): Construct cognitive prototypes based on students' knowledge graphs and map them to the concepts involved in the current task to make accurate predictions.

[0110] In this phase, we use the following two metrics for evaluation:

[0111] Acc: Measures whether the model accurately determines whether a student can correctly complete the current task;

[0112] Con1: Used to evaluate the consistency between model-generated student behavior descriptions and real student behavior. Specifically, we compare the model-generated behavior descriptions with real student behavior (such as whether the task was completed correctly, what mistakes were made, etc.) and use an independent large language model scorer (o1-mini) to evaluate their semantic consistency. The score ranges from 1 to 5, with higher scores indicating that the simulated behavior is closer to the real situation.

[0113] It is important to emphasize that the Acc and Con1 metrics are only used to evaluate the behavior prediction method in the first stage and are unrelated to the subsequent solution simulation method. In other words, different second-stage methods will not affect the scores of these metrics.

[0114] Phase Two: Solution Simulation

[0115] This stage, based on the student behavior predictions generated in the first stage, further generates simulated solutions that match the expected behavior. We compared the following three solution simulation methods:

[0116] IO: Generates the final solution directly based on the behavioral description prompts;

[0117] CoT (Chain-of-Thought): Guides the model to perform chain-like reasoning to enhance the rationality of the solution process;

[0118] Self-Refinement (the method of this invention): Based on behavior prediction, a bundle search and self-evaluation mechanism is introduced. Through multiple rounds of iterative generation, the initial solution is gradually optimized so that the final output is highly consistent with the predicted student behavior.

[0119] Since the second-stage simulation is based on the behavior prediction results of the first stage, each solution simulation method can be combined with any of the six behavior prediction methods, ultimately resulting in 6 (behavior prediction) × 3 (solution simulation) = 18 different combination configurations. This combination setting allows for a more comprehensive evaluation of the synergistic effects between methods at different stages, as well as their respective impact on the final simulation quality.

[0120] The evaluation indicators for this stage are:

[0121] Con2: The similarity and consistency between the student solutions generated by the model and the code solutions submitted by real students is evaluated. The generated simulated solutions and real solutions are input into an LLM scorer (also using o1-mini) for semantic evaluation. The scoring criteria comprehensively consider grammatical style, usage of knowledge points, possible error types, and overall problem-solving approach, with scores ranging from 1 to 5. This indicator reflects whether the solution process faithfully reflects the student's cognitive state and is a key metric for measuring the "naturalness" and "realism" of the simulation.

[0122] The experiment in this embodiment was conducted on the Student_100 dataset mentioned above, and the results are shown in Tables 1 and 2.

[0123] Table 1. The effect of this invention on problem-solving behavior prediction

[0124]

[0125] Table 2. The effect of this invention on the simulation of the solution process

[0126]

[0127]

[0128] Among all model and method combinations, the method of this invention corresponds to the "Prototype Mapping + Self-Refinement" combination, which shows significant advantages in both stages.

[0129] In the behavior prediction phase, the Prototype Mapping method of this invention achieved the highest Acc (0.94) and Con1 (3.77) scores on the GPT-4o model, significantly higher than other methods (Acc mostly between 0.45 and 0.61, and Con1 mostly between 2.2 and 2.6). This indicates that this method can more accurately characterize students' true cognitive state and predict their behavior.

[0130] During the simulation phase, the Self-Refinement method of this invention achieved the highest score (3.65) on Con2, showing a significant improvement compared to other methods such as IO or CoT. The advantages of Self-Refinement are particularly pronounced when using more powerful language models (such as GPT-4o), thanks to its stronger self-evaluation and adjustment capabilities in each generation round.

[0131] In summary, the experimental results fully demonstrate that the phased, cognitive prototype-driven student simulation framework proposed in this invention outperforms existing mainstream methods in terms of "prediction accuracy," "behavioral rationality," and "naturalness of answers," and can more realistically reproduce the behavioral characteristics of students at different cognitive levels.

[0132] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A simulation method for students with different cognitive levels based on a large language model, characterized in that, include: S1. Obtain the historical answer record sequence of the target student, where each answer record includes the question stem text, the student's solution process text, and the student's solution analysis; S2. Information is extracted from the historical answer record sequence using a large language model, and a cognitive prototype of the target student is constructed. The cognitive prototype is a knowledge graph, where each node represents a knowledge point covered by the question stem text in all historical answer records. The edges between nodes are constructed based on the relationships between corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the analysis of the student's answer, representing the target student's global cognitive state of the knowledge point. The specific method for constructing the cognitive prototype of the target student is as follows: S21. For each historical answer record in the historical answer record sequence, the knowledge points covered and their detailed descriptions are extracted from the question text using a large language model. Then, the relationships between the extracted knowledge points are extracted to establish a local subgraph with knowledge points as nodes and relationships as edges. At the same time, diagnostic analysis is performed on the student's answer process text and student answer analysis to obtain the target student's mastery level score and detailed description of mastery level for each knowledge point, and this is recorded as the target student's local cognitive state for the corresponding knowledge point. S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges, and comprehensively analyze the local cognitive states recorded in the merged nodes through a large language model to generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, thus forming the target student's cognitive prototype. S3. For the target question text, use the big language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype. Then input the retrieval results into the big language model to predict the possible erroneous behavior descriptions of the target student when answering the current question. S4. Based on the target question text and the description of the error behavior, use a bundle search-based self-evaluation-self-optimization loop strategy to predict the target student's answer process text through a large language model.

2. The simulation method for students with different cognitive levels based on a large language model as described in claim 1, characterized in that, The question stems and student answer process texts in the target student's historical answer record sequence are extracted from the online learning platform, and the student answer analysis is generated manually based on expert knowledge.

3. The simulation method for students with different cognitive levels based on a large language model as described in claim 1, characterized in that, For any two knowledge points A and B, the relationship between the knowledge points can be divided into four types: A is a prerequisite for B, A is used for B, A is a special case of B, and A is a part of B.

4. The simulation method for students with different cognitive levels based on a large language model as described in claim 1, characterized in that, When integrating the local subgraphs, all knowledge points involved in the local subgraphs are extracted. Each pair of knowledge points is traversed in pairs, and the similarity of the knowledge points is calculated based on the detailed description of their respective knowledge points. If the similarity of the knowledge points exceeds the threshold, the pair of knowledge points is merged. At the same time, if there are two edges and their corresponding two nodes are merged, then these two edges are also merged.

5. The simulation method for students with different cognitive levels based on a large language model as described in claim 1, characterized in that, The global cognitive state is a text describing the target student's overall mastery of knowledge points, generated by a large language model.

6. The simulation method for students with different cognitive levels based on a large language model as described in claim 1, characterized in that, The specific method for predicting the target student's solution process text using the aforementioned bundle search-based self-evaluation-self-optimization loop strategy is as follows: S41. Input the target question text and the description of the error behavior into the large language model to generate the initial solution process text; S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate each of them, and the solution process text that best matches the description of the error behavior is selected to complete the self-optimization. Then, the selected solution process text is input into the large language model for optimization. The self-evaluation and self-optimization loop is iteratively executed. After the iteration terminates, the final solution process text is obtained as the prediction result.

7. A simulation system for students with different cognitive levels based on a large language model, characterized in that, include: The answer record acquisition module is used to acquire the historical answer record sequence of the target student, where each answer record includes the question stem text, the student's solution process text, and the student's solution analysis; A cognitive prototype construction module is used to extract information from the historical answer record sequence using a large language model and construct a cognitive prototype of the target student. The cognitive prototype is a knowledge graph, where nodes represent the knowledge points covered by the question stem text in all historical answer records. Edges between nodes are constructed based on the relationships between corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point, obtained from the student's answer process text and answer analysis, representing the target student's global cognitive state regarding the knowledge point. The specific method for constructing the target student's cognitive prototype is as follows: S21. For each historical answer record in the historical answer record sequence, the knowledge points covered and their detailed descriptions are extracted from the question text using a large language model. Then, the relationships between the extracted knowledge points are extracted to establish a local subgraph with knowledge points as nodes and relationships as edges. At the same time, diagnostic analysis is performed on the student's answer process text and student answer analysis to obtain the target student's mastery level score and detailed description of mastery level for each knowledge point, and this is recorded as the target student's local cognitive state for the corresponding knowledge point. S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges, and comprehensively analyze the local cognitive states recorded in the merged nodes through a large language model to generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, thus forming the target student's cognitive prototype. The error behavior description module is used to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the cognitive prototype of the target student for the target question text using a large language model. The retrieval results are then input into the large language model to predict the error behavior description that the target student may make when answering the current question. The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the error behavior, using a bundle search-based self-evaluation-self-optimization loop strategy and a large language model.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the simulation method for students with different cognitive levels based on a large language model as described in any one of claims 1 to 6.

9. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the simulation method for students with different cognitive levels based on a large language model as described in any one of claims 1 to 6.

Images