Simulation method and system for students with different cognitive levels based on large language model

Abstract

The invention discloses a simulation method and system for students with different cognitive levels based on a large language model. According to the method, the cognitive prototype is constructed based on the past learning records of the students, and the mastering degrees of the students on different knowledge concepts are clearly expressed through the knowledge graph. The cognitive prototype can be used for accurately predicting the performance of the student in a new task, including whether the problem can be solved or not and specific errors possibly caused. Meanwhile, a self-evaluation-self-optimization loop process based on beam search is introduced, and a student answer consistent with a prediction behavior is iteratively generated, so that student simulation which is more real, natural and consistent in cognition is realized on the premise that additional training is not needed. Related results show that the staged and cognitive prototype-driven student simulation framework provided by the invention is superior to the existing mainstream method in multiple dimensions, and behavior characteristics of students with different cognitive levels can be reproduced more truly.

Description

Technical Field

[0001] The present invention belongs to the field of generative language models, and in particular relates to a simulation method and system for students of different cognitive levels based on a large language model. Background Art

[0002] As artificial intelligence accelerates the transformation of education, large language models (LLMs) have become a key technology, widely used in scenarios such as personalized tutoring, curriculum design, and adaptive assessment. A key application area is student simulation. By having models act as students, researchers can cost-effectively evaluate teaching strategies, test the effectiveness of intelligent tutoring systems, and promote the development of educational AI tools.

[0003] However, to achieve authentic and effective student simulation, the model must be able to reflect the differences in students' cognitive abilities. An ideal simulation should not only demonstrate the near-perfect problem-solving process of high-performing students, but also naturally reproduce the common errors and cognitive biases of lower-performing students. However, in reality, current large language models fall short in this regard. Specifically, current large language models generally tend to generate correct answers that are cognitively advanced, making it difficult to faithfully reproduce the common errors and behavioral characteristics of lower-performing students, resulting in a lack of cognitive diversity and authenticity in the simulation results.

[0004] The root of the problem lies in the fact that existing large language models are mostly trained to be "assistants that provide accurate answers." They naturally tend to generate accurate, standard, and even overly advanced answers. This generation approach ignores the natural error patterns that occur during student learning, leading to distorted simulation results, which is particularly evident in simulations of low-level students. For example, the models often overestimate the abilities of elementary school students and fail to replicate their expected error-prone behaviors, rendering the simulation results unreliable.

[0005] Although some studies have attempted to fine-tune the model on erroneous data to encourage it to learn and reproduce common errors, this type of approach has two serious drawbacks: first, it easily instills erroneous knowledge into the model, reducing its overall performance; second, it ignores the individual differences in students' errors and cannot generate personalized errors based on the cognitive states of different students. Summary of the Invention

[0006] The purpose of the present invention is to solve the above problems in the prior art and to provide a simulation method and system for students of different cognitive levels based on a large language model.

[0007] The specific technical solutions adopted in the present invention are as follows:

[0008] In a first aspect, the present invention provides a simulation method for students of different cognitive levels based on a large language model, comprising:

[0009] S1. Obtain a sequence of historical answer records of the target student, where each answer record includes the question text, the student's answer process text, and the student's answer analysis;

[0010] S2. Utilize a large language model to extract information from the history question answer record sequence and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph whose nodes are the knowledge points covered by the question stem text in all history question answer records. Edges between nodes are constructed based on the relationships between corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the text of the student's answer process and the analysis of the student's answers, representing the target student's global cognitive state of the knowledge point.

[0011] S3. For the target question text, use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype. Then, input the retrieval results into the large language model to predict the description of the target student's possible incorrect behavior when answering the current question;

[0012] S4. Based on the target question text and the description of the incorrect behavior, a self-evaluation-self-optimization cycle strategy based on beam search is used to predict the target student's answer process text through a large language model.

[0013] As a preferred embodiment of the first aspect, in the historical answer record sequence of the target student, the question stem text and the student answer process text are extracted from the online learning platform, and the student answer analysis is manually generated based on expert knowledge.

[0014] As a preferred embodiment of the first aspect, the specific method for constructing the cognitive prototype of the target student is as follows:

[0015] S21. For each history question answer record in the history question answer record sequence, extract the covered knowledge points and detailed descriptions of the knowledge points from the question stem text using a large language model, then extract the relationships between the extracted knowledge points, thereby establishing a local subgraph with the knowledge points as nodes and the relationships as edges. Simultaneously, perform diagnostic analysis on the student's answer process text and the student answer analysis to obtain the target student's mastery score and detailed description of the mastery level for each knowledge point, and record these scores as the target student's local cognitive state of the corresponding knowledge point.

[0016] S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges, and comprehensively analyze the local cognitive states recorded in the merged nodes through a large language model, generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, and finally form the target student's cognitive prototype.

[0017] As a preferred embodiment of the first aspect above, for any two knowledge points A and B, the relationship between the knowledge points is divided into four types, namely: A is the predecessor of B, A is used for B, A is a special case of B, and A is a part of B.

[0018] As a preferred embodiment of the above-mentioned first aspect, when the local subgraphs are integrated, the knowledge points involved in all the local subgraphs are extracted, each pair of knowledge points is traversed two by two, and the similarity of the knowledge points is calculated based on the detailed description of their corresponding knowledge points. If the similarity of the knowledge points exceeds a threshold, the pair of knowledge points are merged. At the same time, if there are two edges and their corresponding two nodes are merged, the two edges are also merged.

[0019] As a preferred embodiment of the first aspect, the global cognitive state is a text describing the target student's comprehensive mastery of knowledge points generated by a large language model.

[0020] As a preferred embodiment of the first aspect, the specific method for predicting the target student's answer process text using the self-evaluation-self-optimization cycle strategy based on beam search is as follows:

[0021] S41, inputting the target question stem text and the description of the incorrect behavior into the large language model to generate an initial solution process text;

[0022] S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate them respectively, and the solution process text that best matches the description of the erroneous behavior is selected to complete the self-optimization. The selected solution process text is then input into the large language model for optimization, and the self-evaluation and self-optimization cycle is iteratively executed. After the iteration is terminated, the final solution process text is obtained as the prediction result.

[0023] In a second aspect, the present invention provides a simulation system for students of different cognitive levels based on a large language model, comprising:

[0024] The answer record acquisition module is used to obtain the historical answer record sequence of the target student, where each answer record includes the question text, the student's answer process text, and the student's answer analysis;

[0025] A cognitive prototype construction module is used to extract information from the historical question answer record sequence using a large language model and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph whose nodes are the knowledge points covered by the question stem text in all historical question answer records. The edges between the nodes are constructed based on the relationships between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the text of the student's answer process and the analysis of the student's answer, which is used to represent the target student's global cognitive state of the knowledge point;

[0026] The error behavior description module is used to use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype based on the target question text. The retrieval results are then input into the large language model to predict the description of the target student's possible error behaviors when answering the current question;

[0027] The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the wrong behavior, using a self-evaluation-self-optimization cycle strategy based on beam search, and through a large language model.

[0028] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the simulation method for students of different cognitive levels based on a large language model as described in any one of the first aspects above is implemented.

[0029] In a fourth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0030] The memory is used to store computer programs;

[0031] The processor is configured to implement the method for simulating students of different cognitive levels based on a large language model as described in any one of the first aspects above when executing the computer program.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The method of the present invention constructs a cognitive prototype based on students' past learning records, and clearly expresses the students' mastery of different knowledge concepts through knowledge graphs. This cognitive prototype can be used to accurately predict students' performance in new tasks, including whether they can solve the problem and the specific mistakes they may make. At the same time, the present invention introduces a self-evaluation-self-optimization cycle based on beam search to iteratively generate student answers that are consistent with the predicted behavior, thereby achieving a more realistic, natural, and cognitively consistent student simulation without the need for additional training. The experimental results fully demonstrate that the staged, cognitive prototype-driven student simulation framework proposed in the present invention is superior to existing mainstream methods in terms of "prediction accuracy", "behavioral rationality" and "naturalness of answers", and can more realistically reproduce the behavioral characteristics of students at different cognitive levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the steps of the simulation method for students with different cognitive levels based on the large language model;

[0035] Figure 2 An exemplary construction process for cognitive prototypes;

[0036] Figure 3 A diagram showing the module composition of a simulation system for students with different cognitive levels based on a large language model;

[0037] Figure 4 It is a structural diagram of computer electronic equipment. DETAILED DESCRIPTION

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0039] like Figure 1 As shown, in a preferred embodiment of the present invention, a simulation method for students of different cognitive levels based on a large language model is provided, which includes:

[0040] S1. Obtain a sequence of historical answer records of the target student, where each answer record includes the question stem text, the student's answer process text, and the student's answer analysis.

[0041] It should be noted that in the historical answer record sequence of the above-mentioned target students, the question text and the student answer process text can be extracted from the online learning platform, and the student answer analysis can be extracted together if it is available on the learning platform. If not, it can be manually generated based on expert knowledge.

[0042] It should be noted that the historical answer record sequence constructed in step S1 of the present invention is different from the existing student answer record data set. Most existing student answer record data sets usually use the index of the question to represent each question, and only contain the judgment of whether the student's answer is correct, lacking key information such as the question stem text, student answer analysis, etc. necessary for student simulation. To this end, in an embodiment of the present invention, a large-scale student answer record sequence data set is acquired and constructed based on an online programming platform for student simulation research based on a large language model. The data set contains a series of Python programming answer record sequences of different students, and each student's answer record sequence contains a series of answer records for different programming questions, and each answer record contains the question stem text t, the student's answer process text s, and the student's answer analysis b. In an embodiment of the present invention, the specific student answer record sequence sample construction process is as follows:

[0043] S11. Obtain the initial student answer record sequence from the online programming platform. Each initial answer record contains the question stem text t and the student answer process text s.

[0044] S12. Manually evaluate the student's answer in each initial answer record and compile a student answer analysis. In this embodiment, 10 domain experts were recruited to review and evaluate each student's answer in the initial answer record and compile a student answer analysis (b), which includes a determination of whether the student answered the question correctly; if the student did not answer the question correctly, it also includes a detailed explanation of the student's error.

[0045] S2. Use a large language model to extract information from the historical question answering record sequence and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph, whose nodes are the knowledge points covered by the question stem text in all historical question answering records, and the edges between the nodes are constructed according to the relationship between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the student's answer process text and the student's answer analysis, which is used to represent the target student's global cognitive state of the knowledge point.

[0046] It should be noted that step S2 of the present invention is to extract a cognitive prototype in the form of natural language from the student's historical answer record sequence. Previous student cognitive diagnosis work usually only uses the question index to represent the answer sequence, without involving information such as the question text, and uses the implicit and parameterized knowledge in the deep network to represent the student's cognitive state, which also leads to the lack of interpretability of its diagnosis of the student's cognitive state. Therefore, the present invention uses a large language model to extract the student's cognitive prototype from the text-rich student's historical answer record sequence. The cognitive prototype is combined with the knowledge graph to express the student's mastery of each knowledge point in the form of natural language, thereby reflecting the student's cognitive state. Specifically, the cognitive prototype of the above-mentioned target student can be constructed according to the following specific method:

[0047] S21. For each history question answer record in the history question answer record sequence, extract the covered knowledge points and detailed descriptions of the knowledge points from the question stem text using a large language model, then extract the relationships between the extracted knowledge points, thereby establishing a local subgraph with the knowledge points as nodes and the relationships as edges. Simultaneously, perform diagnostic analysis on the student's answer process text and the student answer analysis to obtain the target student's mastery score and detailed description of the mastery level for each knowledge point, and record these scores as the target student's local cognitive state of the corresponding knowledge point.

[0048] S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges (the information such as the local cognitive state recorded in the nodes and the corresponding relationships of the edges are also merged), and conduct a comprehensive analysis of the local cognitive state recorded in the merged nodes through a large language model, generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, and finally form the target student's cognitive prototype.

[0049] In the embodiment of the present invention, the process of constructing the cognitive prototype of the target student can be regarded as two steps: A) local cognitive state extraction, corresponding to S21, for extracting the student's local cognitive state from each historical answer record; B) global cognitive state analysis, corresponding to S22, for analyzing all local cognitive states to obtain the global cognitive state after all historical answer records are processed. Figure 2 As shown in FIG, the construction process of the cognitive prototype is exemplified. The specific implementation process of these two links is described in detail below based on this example.

[0050] A) Local cognitive state extraction

[0051] A1) Traverse all historical answer records in the historical answer record sequence and extract relevant knowledge points from the current historical answer record.

[0052] It should be noted that, in step A1) of the present invention, given the current historical answer record P i =(t i ,s i ,b i ), where t i is the text of the question, s i Answer the process text for students, b i To analyze the answers for students, we use a large language model to extract several knowledge points from the question text. i,1 ,v i,2 ,…] and a detailed description of the knowledge point. These knowledge points will serve as nodes in the local subgraph corresponding to the current historical answer record, and will also serve as part of the knowledge graph in the student's cognitive prototype during subsequent subgraph integration.

[0053] A2) performs relationship extraction on the knowledge points extracted in A1) to obtain the associations between the knowledge points.

[0054] It should be noted that in the steps of the present invention, for any two knowledge points A and B, four types of relationships between knowledge points are predefined to standardize the extraction of knowledge point relationships, namely: A is a precondition of B, A is used in B, A is a special case of B, and A is a part of B. The large language model is used to analyze the possible relationships from the knowledge points extracted in step S21, and each relationship is classified into one of the four types mentioned above to obtain the relationships between knowledge points [e i,1 ,e i,2 ,…]. The associations between these knowledge points will be constructed as edges in the local subgraph corresponding to the current historical answer record, and will also serve as part of the edges of the knowledge graph in the student's cognitive prototype in the subsequent subgraph integration.

[0055] A3) analyzes the student's local cognitive state of the knowledge point extracted in A1) and adds it to the local cognitive state library of the knowledge point.

[0056] It should be noted that for the same knowledge point, students may perform differently in different answer records. Therefore, a local cognitive state library is used to record the student's local cognitive state for the same knowledge point in different history answer records. Step A3) of the present invention aims to analyze the student's local mastery of the knowledge point from the student's current history answer record and form the student's local cognitive state. For each knowledge point v extracted in A1), i,* , using a large language model to diagnose the student's local cognitive state of the knowledge point from the student's answer process text and answer analysis i,*The diagnostic method is as follows: First, the student's local mastery of the knowledge point is judged and classified into "Good" or "Bad". "Good" means that the student has a good grasp of the knowledge point, while "Bad" means that the student's local mastery is poor. Then, a detailed explanation of the judgment is given. Subsequently, the student's local cognitive status c for each knowledge point is i,* It will be added to the local cognitive state library of the corresponding knowledge point as a record. In this way, the student's local cognitive state is extracted from the student's current historical answer record.

[0057] Therefore, for each current historical answer record in the traversal process, the knowledge point can be used as the node and the edge can be determined by the extracted relationship, so as to construct a local subgraph corresponding to the current historical answer record. Each node in the local subgraph records the target student's mastery score and detailed description of each knowledge point through the local cognitive state library, which represents the target student's local cognitive state of the corresponding knowledge point.

[0058] B) Global cognitive state analysis

[0059] B1) De-duplicate the nodes and edges of the knowledge graph constructed in the above steps to obtain a non-redundant knowledge graph.

[0060] It should be noted that when extracting knowledge points and associations between knowledge points, different historical answer records may overlap. For example, a certain knowledge point appears in multiple historical answer records. Therefore, the steps of the present invention are intended to merge highly similar knowledge points and associations between knowledge points to remove redundancy in the knowledge graph. Specifically, first extract the knowledge points involved in all local subgraphs, and group all knowledge points into pairs, with each pair of knowledge points containing a detailed description of the knowledge points extracted in step A1). For each pair of knowledge points, the similarity of the detailed descriptions of the knowledge points between them can be calculated by a large language model. If the similarity is higher than the set threshold δ1, it is considered that the pair of knowledge points is the same knowledge point and needs to be merged. For all redundant knowledge points, merge their local cognitive state libraries. Similarly, if the two nodes of two edges are both considered to be the same knowledge point, the two edges are also merged, and the final relationship type is the type with the highest number of occurrences. In this embodiment, δ1 is 0.85.

[0061] B2) Comprehensively analyze the contents of the local cognitive state library of each knowledge point in the merged global knowledge graph to obtain the student's global cognitive state of each knowledge point, thereby constructing a complete student cognitive prototype.

[0062] It should be noted that the local cognitive state obtained in step A3) of the present invention will vary depending on different historical answer records, so it is necessary to conduct a comprehensive analysis of all local cognitive states of the student to obtain a unified global cognitive state. Therefore, this step B1) should be performed after all the historical answer records of the student have been processed through steps A1)-A3). Figure 2 As shown, the large language model comprehensively analyzes the frequency and description of "Good" and "Bad" in the local cognitive state library for each knowledge point. This derives the student's overall mastery of that knowledge point as the global cognitive state, thereby constructing a complete student cognitive prototype. It should be noted that the global cognitive state is a textual description of the target student's comprehensive mastery of the knowledge point, generated by the large language model, expressed in natural language, such as "excellent mastery" and "poor mastery."

[0063] S3. For the target question text, use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype, and then input the retrieval results into the large language model to predict the description of the target student's possible incorrect behavior when answering the current question.

[0064] It should be noted that the strategies of student simulation and student cognitive diagnosis have completely opposite processes. When conducting student cognitive diagnosis, it is necessary to analyze the student's behavior when solving the problem based on the student's existing solution process text to generate a solution analysis; however, student simulation first needs to predict what behaviors the student will have on the problem, including whether they will make mistakes and where they will make mistakes, and then simulate the student's solution process text based on the predicted behavior. Therefore, step S3 of the present invention is to predict the student's problem-solving behavior based on the student cognitive prototype obtained in S2, including the following steps:

[0065] S31. For each question to be predicted, use the student’s cognitive prototype to retrieve several knowledge points that are most relevant to the question text and the student’s global cognitive status of these knowledge points as the basis for subsequent predictions.

[0066] It should be noted that previous methods usually directly retrieve a historical answer record that is most similar to the question text of the current simulated answer record as the basis for subsequent predictions. However, this method based on question text similarity is very easily affected by the expression of the question text, and retrieves historical answer records with similar question texts but completely different core test knowledge points. This fails to reflect the student's mastery of the questions in the simulated answer record, thereby misleading subsequent behavior predictions. Therefore, the present invention takes knowledge points as the core and uses students' cognitive prototypes to retrieve students' global cognitive state.

[0067] Specifically, for the current question to be predicted, the present invention can use its stem text t j , through the large language model, search on the knowledge graph of the student's cognitive prototype and obtain a set of knowledge points [v1, v2, ... v p ], and extract the students’ global cognitive states [C1, C2, …C p The p value is a hyperparameter and can be adjusted according to actual conditions. In this embodiment, the p value is 5.

[0068] S32. Using the students’ global cognitive state of the most relevant extracted knowledge points as a reference, the large language model is used to predict the possible incorrect behavior descriptions of students when answering the current predicted question. Including whether it will make mistakes and what specific mistakes it will make.

[0069] S4. Based on the target question text and the description of the incorrect behavior, a self-evaluation-self-optimization cycle strategy based on beam search is used to predict the target student's answer process text through a large language model.

[0070] It should be noted that, since current large language models are usually trained according to correct and reliable output targets, they tend to output correct content. Therefore, when simulating the student's problem-solving process text based on the student's behavior description, especially when the student's behavior description contains mistakes that the student may make, the initial student's solution process text generated by the large language model may not accurately reflect the errors described in the predicted student behavior. Therefore, the present invention proposes a process of continuous self-evaluation and self-optimization of the large language model, and introduces a beam search mechanism to increase the sampling space of the model output, thereby increasing the probability of successful simulation. Specifically, using the above-mentioned self-evaluation-self-optimization cycle strategy based on beam search, the specific method of predicting the target student's solution process text can be implemented according to the following process:

[0071] S41, inputting the target question stem text and the description of the incorrect behavior into the large language model to generate an initial solution process text;

[0072] S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate them respectively, and the solution process text that best matches the description of the erroneous behavior is selected to complete the self-optimization. The selected solution process text is then input into the large language model for optimization, and the self-evaluation and self-optimization cycle is iteratively executed. After the iteration is terminated, the final solution process text is obtained as the prediction result.

[0073] It can be seen that the above self-evaluation and self-optimization need to be continuously iterated. Assume that the process includes L iterations. The specific number of iterations L can be adjusted according to actual conditions. In this embodiment, L is 3. In the embodiment of the present invention, each iteration includes the following steps:

[0074] C1) Given the current question to be predicted, the stem text t j and predicted student behavior Generate initial student answer process text using a large language model

[0075] C2) Based on the solution process text generated in the previous iteration (selected in the first iteration ), optimize using a large language model, and sample several optimized solution process texts at the same time.

[0076] It should be noted that the purpose of sampling multiple optimized solution process texts in step C2) of the present invention is to increase the sampling space, thereby increasing the probability that the sampling results accurately match the predicted student behavior. Specifically, let the current iteration number be l (1≤l≤L), and the solution process text generated in the previous iteration be The present invention samples B optimized solution process texts based on the large model: In this embodiment, B is 2.

[0077] C3) Use the large language model to evaluate and score the several solution process texts optimized in this iteration, and select the solution process text with the highest score as the final solution process text for this iteration.

[0078] It should be noted that step C3) of the present invention is intended to use the self-evaluation ability of the large language model to select a solution process text that best matches the predicted student behavior description, thereby reducing the sampling space of the subsequent iterative process. Specifically, this step uses the large language model to select the solution process text sampled in C2). Each solution process text is evaluated one by one, mainly to evaluate whether the solution process text strictly matches the predicted student behavior and whether it accurately reflects the predicted student behavior description. Based on this evaluation, each solution process text is scored between 0 and 1. Finally, the solution process text with the highest score is selected as the final solution process text of this round of iteration.

[0079] In addition, in the above-mentioned iterative process, the embodiment of the present invention sets two conditions for the termination of the iteration: the first is that the upper limit L of the number of iterations is reached; the second is that the self-evaluation score corresponding to the final solution process text of a certain round of iteration exceeds the set threshold δ2. When either of the two conditions is met, the iteration terminates. In this embodiment, δ2 is set to 0.9. The solution text generated at the termination of the iteration is the final simulated student solution text.

[0080] In summary, the core of the proposed method is to construct a cognitive prototype based on students' past learning records, using a knowledge graph to clearly express the student's mastery of different knowledge concepts. This cognitive prototype can be used to accurately predict students' performance in new tasks, including whether they can solve the problem and the specific errors they may make. Furthermore, by introducing a self-evaluation and self-optimization cycle based on beam search, student answers consistent with the predicted behavior are iteratively generated, thus achieving a more realistic, natural, and cognitively consistent student simulation without the need for additional training.

[0081] It should be noted that the method steps shown in S1 to S4 above can essentially be implemented in the form of computer programs or functional modules.

[0082] Therefore, based on the same inventive concept, the present invention further provides a simulation system for students with different cognitive levels based on a large language model, corresponding to the simulation method for students with different cognitive levels based on a large language model provided in the above embodiment, comprising:

[0083] The answer record acquisition module is used to obtain the historical answer record sequence of the target student, where each answer record includes the question text, the student's answer process text, and the student's answer analysis;

[0084] A cognitive prototype construction module is used to extract information from the historical question answer record sequence using a large language model and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph whose nodes are the knowledge points covered by the question stem text in all historical question answer records. The edges between the nodes are constructed based on the relationships between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the text of the student's answer process and the analysis of the student's answer, which is used to represent the target student's global cognitive state of the knowledge point;

[0085] The error behavior description module is used to use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype based on the target question text. The retrieval results are then input into the large language model to predict the description of the target student's possible error behaviors when answering the current question;

[0086] The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the wrong behavior, using a self-evaluation-self-optimization cycle strategy based on beam search, and through a large language model.

[0087] In addition, based on the same inventive concept, Figure 4 As shown, the present invention also provides a computer electronic device corresponding to the simulation method of students with different cognitive levels based on a large language model provided in the above embodiment, which includes a memory and a processor;

[0088] The memory is used to store computer programs;

[0089] The processor is configured to implement the aforementioned method for simulating students of different cognitive levels based on a large language model when executing the computer program;

[0090] Furthermore, the logic instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention.

[0091] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to a simulation method for students with different cognitive levels based on a large language model, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it can implement the simulation method for students with different cognitive levels based on a large language model as described above.

[0092] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the simulation method for students of different cognitive levels based on a large language model as described above.

[0093] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by the processor to perform the above steps S1 to S4.

[0094] It is understood that the storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage medium may be any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0095] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0096] It should also be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0097] The present invention will further demonstrate the detailed implementation process and technical effects of the simulation method for students of different cognitive levels based on the large language model shown in the above steps S1 to S4 on a specific data set through a specific embodiment, so as to facilitate understanding of the essence of the present invention.

[0098] Example

[0099] The steps of this embodiment are the same as the simulation method for students of different cognitive levels based on a large language model as shown in the aforementioned steps S1 to S4, and will not be repeated here. The specific data set, some specific parameter settings and implementation results of this embodiment will be mainly demonstrated.

[0100] To verify the effectiveness of this embodiment, the present invention obtains initial answer records (including the question stem text and the student's answer process text) based on an online programming platform. Ten domain experts are then recruited to review and evaluate the student answers in each initial answer record and write a student answer analysis b, which includes a determination of whether the student answered the question correctly; if the student did not answer the question correctly, it also includes a detailed explanation of the student's error. Thus, a new large-scale student answer record sequence dataset named Student_100 is constructed for student simulation research based on large language models. This dataset contains Python programming answer record sequences of 100 students. Each student's answer record sequence contains a total of 50 answer records. The earlier 40 answer records are divided into the historical answer record sequence in chronological order, and the later 10 answer records are divided into the to-be-simulated answer record sequence. Each answer record contains the question stem text t, the student's answer process text s, and the student answer analysis b. The to-be-simulated answer record sequence consisting of the later 10 answer records is used to measure the performance of each method in the student simulation task.

[0101] We conducted student simulation experiments using four mainstream large-scale language models: LLaMA-3.3-70B-Instruct, Claude-3.5-Sonnet, GPT-3.5, and GPT-4o. We systematically compared these models with existing student simulation methods. Given that our proposed student simulation framework consists of two key phases: problem-solving behavior prediction and problem-solving process simulation, we introduced comparative methods for each phase and employed a phased evaluation strategy to comprehensively measure the performance of each method on the student simulation task.

[0102] Phase 1: Behavior Prediction

[0103] This stage aims to predict students' performance when faced with new tasks, that is, to determine whether they can answer correctly and provide a description of their behavior consistent with their ability. We designed the following five comparison methods:

[0104] Random: Randomly select a task from the student's past records as a reference;

[0105] Similarity: Select the historical task that is most similar to the current task text for prediction;

[0106] Level: Estimates the student's ability level based on their accuracy in answering history questions;

[0107] Level+Random: combines student ability estimation with random retrieval strategy;

[0108] Level+Similarity: combines student ability estimation with similar task retrieval strategy;

[0109] Prototype Mapping (method of the present invention): Construct cognitive prototypes based on students' knowledge graphs and map them to concepts involved in the current task to make accurate predictions.

[0110] In this phase, we use the following two metrics for evaluation:

[0111] Acc: measures whether the model accurately judges whether the student can complete the current task correctly;

[0112] Con1: This function evaluates the consistency between the model-generated descriptions of student behaviors and actual student behaviors. Specifically, we compare the model-generated descriptions with the actual student behaviors (e.g., whether the tasks were completed correctly, what errors occurred, etc.) and use a separate large language model scorer (o1-mini) to assess their semantic consistency on a scale of 1 to 5, with higher scores indicating closer alignment between the simulated behaviors and the real-world situations.

[0113] It is important to emphasize that the Acc and Con1 metrics are only used to evaluate the first-stage behavior prediction method and are unrelated to the subsequent solution simulation method. In other words, different second-stage methods will not affect the scores of these metrics.

[0114] Phase 2: Solution Simulation

[0115] This phase further generates simulated answers that meet the expected student behavior based on the predicted student behavior generated in the first phase. We compared the following three answer simulation methods:

[0116] IO: Generate the final answer directly based on the behavior description prompt;

[0117] CoT (Chain-of-Thought): guides the model to perform chain reasoning to enhance the rationality of the solution process;

[0118] Self-Refinement (method of the present invention): Based on behavior prediction, beam search and self-evaluation mechanisms are introduced to gradually optimize the initial solution through multiple rounds of iterative generation process, so that the final output is highly consistent with the predicted student behavior.

[0119] Because the second-stage simulation is based on the behavior prediction results of the first stage, each solution simulation method can be combined with any of the six behavior prediction methods, ultimately resulting in 6 (behavior prediction) × 3 (solution simulation) = 18 different combination configurations. This combination setup allows for a more comprehensive evaluation of the synergistic effects between the methods in different stages and their respective impact on the final simulation quality.

[0120] The evaluation indicators at this stage are:

[0121] Con2: Evaluate the similarity and consistency between the model-generated student solutions and the code solutions submitted by real students. The generated simulated solutions and real solutions were fed into the LLM scorer (also using o1-mini) for semantic evaluation. The scoring criteria comprehensively consider grammatical style, knowledge point usage, possible error types, and overall problem-solving approach, with a score range of 1 to 5. This metric reflects whether the solution process is faithful to the student's cognitive state and is a key indicator of the "naturalness" and "authenticity" of the simulation.

[0122] The experiment of this embodiment was carried out on the above-mentioned Student_100 dataset, and the results are shown in Tables 1 and 2.

[0123] Table 1. Effect of the present invention on problem-solving behavior prediction

[0124]

[0125] Table 2. Effect of the present invention on the solution process simulation

[0126]

[0127]

[0128] Among all the combinations of models and methods, the method of the present invention corresponds to the "Prototype Mapping+Self-Refinement" combination, which shows significant advantages in both stages.

[0129] In the behavior prediction phase, our Prototype Mapping method achieved the highest Acc (0.94) and Con1 (3.77) scores on the GPT-4o model, significantly higher than other methods (Acc ranged from 0.45 to 0.61, and Con1 ranged from 2.2 to 2.6). This demonstrates that our method can more accurately capture students' true cognitive states and predict their behaviors.

[0130] In the solution simulation phase, our Self-Refinement method achieved the highest score (3.65) on Con2, significantly improving over other methods such as IO or CoT. The advantages of Self-Refinement are particularly pronounced when using more powerful language models (such as GPT-4o), thanks to its enhanced self-assessment and adjustment capabilities during each generation round.

[0131] In summary, the experimental results fully demonstrate that the staged, cognitive prototype-driven student simulation framework proposed in this invention is superior to existing mainstream methods in terms of "prediction accuracy", "behavioral rationality" and "naturalness of answers", and can more realistically reproduce the behavioral characteristics of students at different cognitive levels.

[0132] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A simulation method for students of different cognitive levels based on a large language model, characterized in that: include: S1. Obtain a sequence of historical answer records of the target student, where each answer record includes the question text, the student's answer process text, and the student's answer analysis; S2. Utilize a large language model to extract information from the history question answer record sequence and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph whose nodes are the knowledge points covered by the question stem text in all history question answer records. Edges between nodes are constructed based on the relationships between corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the text of the student's answer process and the analysis of the student's answers, representing the target student's global cognitive state of the knowledge point. S3. For the target question text, use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype. Then, input the retrieval results into the large language model to predict the description of the target student's possible incorrect behavior when answering the current question; S4. Based on the target question text and the description of the incorrect behavior, a self-evaluation-self-optimization cycle strategy based on beam search is used to predict the target student's answer process text through a large language model.

2. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 1, characterized in that: In the target student's historical answer record sequence, the question stem text and the student's answer process text are extracted from the online learning platform, and the student answer analysis is manually generated based on expert knowledge.

3. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 1, characterized in that: The specific method of constructing the cognitive prototype of the target students is as follows: S21. For each history question answer record in the history question answer record sequence, extract the covered knowledge points and detailed descriptions of the knowledge points from the question stem text using a large language model, then extract the relationships between the extracted knowledge points, thereby establishing a local subgraph with the knowledge points as nodes and the relationships as edges. Simultaneously, perform diagnostic analysis on the student's answer process text and the student answer analysis to obtain the target student's mastery score and detailed description of the mastery level for each knowledge point, and record these scores as the target student's local cognitive state of the corresponding knowledge point. S22. Integrate the local subgraphs of all historical answer records in the historical answer record sequence, merge duplicate nodes and edges, and comprehensively analyze the local cognitive states recorded in the merged nodes through a large language model, generate the target student's global cognitive state for the corresponding knowledge point and record it in the merged nodes, and finally form the target student's cognitive prototype.

4. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 3, characterized in that: For any two knowledge points A and B, there are four types of relationships between the knowledge points: A is a predecessor of B, A is used for B, A is a special case of B, and A is a part of B.

5. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 3, characterized in that: When the local subgraphs are integrated, the knowledge points involved in all the local subgraphs are extracted, each pair of knowledge points is traversed two by two, and the similarity of the knowledge points is calculated according to the detailed description of their respective corresponding knowledge points. If the similarity of the knowledge points exceeds a threshold, the pair of knowledge points is merged. At the same time, if there are two edges and their corresponding two nodes are merged, the two edges are also merged.

6. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 3, characterized in that: The global cognitive state is a text describing the target student's comprehensive mastery of knowledge points generated by the large language model.

7. The method for simulating students of different cognitive levels based on a large language model as claimed in claim 3, characterized in that: The specific method for predicting the target student's solution process text using the self-evaluation-self-optimization cycle strategy based on beam search is as follows: S41, inputting the target question stem text and the description of the incorrect behavior into the large language model to generate an initial solution process text; S42. The initial solution process text is re-input into the large language model for optimization, and multiple optimized solution process texts are sampled. Then, the large language model is used to self-evaluate them respectively, and the solution process text that best matches the description of the erroneous behavior is selected to complete the self-optimization. The selected solution process text is then input into the large language model for optimization, and the self-evaluation and self-optimization cycle is iteratively executed. After the iteration is terminated, the final solution process text is obtained as the prediction result.

8. A simulation system for students of different cognitive levels based on a large language model, characterized by: include: The answer record acquisition module is used to obtain the historical answer record sequence of the target student, where each answer record includes the question text, the student's answer process text, and the student's answer analysis; A cognitive prototype construction module is used to extract information from the historical question answer record sequence using a large language model and construct a cognitive prototype of the target student; the cognitive prototype is a knowledge graph whose nodes are the knowledge points covered by the question stem text in all historical question answer records. The edges between the nodes are constructed based on the relationships between the corresponding knowledge points. Each node records a detailed description of the target student's mastery of the corresponding knowledge point based on the text of the student's answer process and the analysis of the student's answer, which is used to represent the target student's global cognitive state of the knowledge point; The error behavior description module is used to use the large language model to retrieve the most relevant knowledge points and the global cognitive state recorded in the knowledge points from the target student's cognitive prototype based on the target question text. The retrieval results are then input into the large language model to predict the description of the target student's possible error behaviors when answering the current question; The solution process prediction module is used to predict the target student's solution process text based on the target question text and the description of the wrong behavior, using a self-evaluation-self-optimization cycle strategy based on beam search, and through a large language model.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the simulation method for students of different cognitive levels based on a large language model according to any one of claims 1 to 7 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the method for simulating students of different cognitive levels based on a large language model as claimed in any one of claims 1 to 7 when executing the computer program.

Images