Full-attribute automatic generation knowledge tracking method and system based on large language model

Abstract

The application discloses a full-attribute automatic generation knowledge tracking method and system based on a large language model. The core knowledge points and weights of each question are automatically extracted by the LLM, and are embedded and fused with the question number to accurately express the question semantics. A time sequence multi-head attention mechanism is adopted, the real and counterfactual answer sequences are combined, the student knowledge state is dynamically modeled and mapped into a multi-knowledge point mastery probability. The LLM is used to generate multi-dimensional scoring features for the question-answer text, which is further fused with the knowledge state features to strengthen the cognitive process representation. A monotonicity constraint is specially set to ensure that the knowledge state changes reasonably with the answer result. The application can effectively improve the accuracy of student answer correctness prediction and the model interpretability, and is suitable for intelligent education and individualized teaching scenarios.

Description

Technical Field

[0001] This invention relates to the fields of educational big data mining, neural networks and student behavior modeling, and specifically to a method and system for automatic generation of knowledge tracking based on a large language model. Background Technology

[0002] With the rapid development of information technology, Knowledge Tracing (KT) has become one of the key technologies for the intelligent development of education. By analyzing learners' behavioral data in learning activities such as answering questions, KT dynamically monitors and predicts students' mastery of various knowledge points, providing strong support for the realization of personalized teaching and adaptive learning systems. It also helps teachers to identify students' weaknesses in a timely manner and provide precise intervention.

[0003] Currently, mainstream Knowledge Tracking (KT) models include Deep Knowledge Tracking (DKT), Dynamic Key-Value Memory Network (DKVMN), Context-Aware Attention Knowledge Tracking (AKT), Context-Aware Dual-Information Modeling Knowledge Tracking (DIMKT), Self-Attention Knowledge Tracking (SAKT), Learning Process Knowledge Tracking (LPKT), Fine-Grained Knowledge Tracking (FDKT), and Dynamic Hybrid Knowledge Tracking (DMKT). These methods primarily utilize architectures such as RNNs, LSTMs, and Transformers, focusing on modeling students' temporal response behavior across knowledge points or test questions. Specifically, DKT automatically mines the relationship between students' historical responses and knowledge mastery through deep neural networks; DKVMN utilizes key-value memory mechanisms to strengthen the modeling of relationships between multiple knowledge points; AKT incorporates a contextual attention mechanism to enhance historical dependency understanding; DIMKT further combines dual-information modeling to improve representation richness; SAKT uses a self-attention mechanism to characterize the sensitivity of answer sequences; LPKT focuses on learning process behavior analysis; FDKT emphasizes fine-grained modeling of knowledge points; and DMKT achieves dynamic hybrid modeling of knowledge states, improving model flexibility and adaptability.

[0004] Despite the continuous emergence of innovative methods in the KT field, existing models generally have the following shortcomings: (1) They have limited ability to integrate multi-source information such as test questions, knowledge points, and answer performance, making it difficult to fully extract and utilize the deep connections between the content of the questions, knowledge structure, and students' answer performance; (2) They rely heavily on manually labeled question-knowledge point mapping or use only a single feature input, lacking automated knowledge attribution and feature extraction mechanisms, and are unable to fully explore new implicit knowledge information in unlabeled questions; (3) They lack modeling of process evaluation features such as the quality of students' answer process and cognitive depth, resulting in an insufficient cognitive characterization of real learning behavior; (4) The knowledge state update and prediction mechanism needs to be improved in terms of modeling individual differences, temporal memory decay, and dynamic changes in knowledge mastery, especially in terms of interpretability and controllability.

[0005] Therefore, how to combine advanced Large Language Models (LLMs) to automatically extract and fuse multi-source features such as question knowledge concepts and student scores, dynamically and precisely model the transition process between students' true and counterfactual knowledge states, and improve the interpretability and predictive performance of the model through effective constraints has become a key problem that urgently needs to be solved in the field of knowledge tracing. This invention addresses the above needs by proposing a dynamic representation and prediction method for student knowledge based on large-scale model knowledge attribution and multi-dimensional scoring feature fusion. This method achieves adaptive integration of multi-source high-dimensional information and causal interpretable knowledge tracing, effectively improving the accuracy of predicting the correctness of student answers and the practical application value of the model. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technologies. This study utilizes a large language model to automatically extract core knowledge concepts and weights related to the questions, and forms semantic representations of the questions through embedding and fusion. A multi-head attention mechanism is introduced to integrate real and counterfactual answer histories, dynamically generating the mastery probability of multiple knowledge points. A large language model is used to perform multi-dimensional scoring on the question-answer text, obtaining scoring features and fusing them with knowledge state features to enhance the characterization of students' cognitive processes. Furthermore, a counterfactual monotonicity constraint is set to ensure that the knowledge state changes reasonably with the answers, improving the model's interpretability and predictive ability.

[0007] The objective of this invention is achieved through the following technical measures: a knowledge tracking method based on multi-view dynamic evolution, comprising the following steps:

[0008] Step 1: Construct a knowledge and rating feature extraction module to obtain the semantic representation of each question text, and combine the answer content of each question with the question text to obtain multi-dimensional rating features about the answer;

[0009] Step 2: Construct a question semantic embedding and fusion module, and obtain the question representation vector by combining the question semantic representation and question number embedding;

[0010] Step 3: Construct a module for sequential knowledge state modeling and monotonicity constraints, and output the current time step. The main branch knowledge state and the counterfactual branch knowledge state are introduced with monotonicity constraints to map the main branch knowledge state to the knowledge point mastery probability space.

[0011] Step 4: Construct a scoring feature modeling auxiliary module to encode multi-dimensional scoring features, combine the main branch knowledge state with the scoring feature encoding to obtain comprehensive fusion features, apply a multi-head self-attention mechanism, and dynamically adjust the weights according to time to obtain scoring time-series features.

[0012] Step 5: Concatenate the knowledge state of the main branch with the scoring time sequence features to obtain a comprehensive student representation feature. Input the comprehensive student representation feature into a fully connected prediction network. After multi-layer nonlinear transformation, the final output is the probability of the student's answer score on the current question.

[0013] Furthermore, in step 1, regarding the problem text... Using Large Language Modeling (LLM) combined with analysis of prompt words Automatically generate knowledge concepts and their weights The semantic representation of the question is obtained by embedding knowledge concepts into a pre-trained language model and then weighting the results accordingly. ;in , This represents the total number of questions.

[0014] Furthermore, in step 1, for each question-and-answer data point, the question text is... and the answer texts submitted by the students Assemble the question - provide a response text ,in For the splicing operation, among which , The total number of responses; combine the question-response text with another set of analytical prompts. The data is fed into a large language model (LLM), which outputs a comprehensive score that forms the feature vector of the answer across different evaluation dimensions. , ,in As evaluation dimensions.

[0015] Furthermore, in step 2, the current time is first... The semantic representation of the question Perform dimensionality reduction to reduce the dimension to the level of the embedded question number. With the same dimension, we obtain the dimensionality-reduced semantic representation of the question. Next, the semantic representation of the problem after dimensionality reduction is... embedding question numbers Perform LayerNorm operations respectively to obtain and Then on and Perform adaptive fusion operation on the gating network to obtain the problem representation vector. , , Among them, the fusion weight vector , , and These are the weight matrix and bias vector of the gated network, respectively, both of which are learnable parameters. This represents the vector concatenation operation. This represents the Sigmoid operation.

[0016] Furthermore, the specific implementation method for obtaining the knowledge state of the main branch and the counterfactual branch in step 3 is as follows:

[0017] (3-1) Constructing main branch features and counterfactual features

[0018] Students in time step 1 to The process of answering questions is represented as a sequence of questions. Sequence of Response Results each The value can be 0 or 1, representing the student's response to the question. The answer is either correct or incorrect, and a counter-response sequence is introduced based on the original sequence. Specifically, this involves reversing the result of the most recent step in the sequence, i.e. ;

[0019] For the knowledge state of the main branch, the problem representation vector obtained at the current time t The corresponding answer result is embedded ,in Embed the result into a matrix. The first row contains the "incorrect" embeddings, and the second row contains the "correct" embeddings. Then, the question embeddings are fused with the corresponding answer embeddings to obtain a joint representation. , ;

[0020] For counterfactual branch knowledge states, the answer result is only displayed at the input layer. Replace with counterfactual labels Generate corresponding counterfactual features , ;

[0021] (3-2) A multi-head attention mechanism is used to dynamically model the semantics and historical behavior of each time step;

[0022] For the sequence number At each time step, first obtain Moment-based semantic fusion embedding as query vector and every point in history Response feature encoding as a key vector and numerical vectors , Under the multi-head attention mechanism, Will be projected to Each subspace has a dimension of [number] heads. , making ;

[0023] For each attention head The input embedding is projected into each subspace through a learnable linear mapping, i.e. , , , here , The parameters for each head are the query, key, and value. Within each attention head, the inner product is used to measure the relevance between the current context and historical nodes, and content similarity is calculated. , ;

[0024] At the same time, a time-decreasing weight is introduced for each historical node; specifically, the current interval and the historical interval are defined. And set a learnable decay rate parameter. Thus, the time-series decay factor is obtained. , ;

[0025] Next, the content similarity is multiplied by the temporal decay factor to obtain a weighted score, which is then normalized using softmax to obtain the final attention distribution. ,in No. The height in Time is assigned to historical nodes The normalized attention weights are then assigned; subsequently, each attention head performs a weighted sum of all historical value vectors according to the above assignment to obtain the context state representation of that head. , Finally, combining all The context outputs of each head are concatenated and then output linearly projected. This yields the final main branch knowledge state representation. , ,in This represents the knowledge state at the current moment; similarly, it represents the knowledge state of the counterfactual branch. The structure is the same, representing each point in history. Response feature encoding Replace with counterfactual response feature encoding as a key vector and numerical vectors That's all.

[0026] Furthermore, the monotonicity constraints specifically include:

[0027] Let the global knowledge point set be , To represent the total number of knowledge concepts, a multi-knowledge point distribution model is achieved through linear transformation of a fully connected layer and Sigmoid normalization. , ,in Normal knowledge state The transformed set of knowledge points has a probability set and ,Right now ,and State of counterfactual knowledge The transformed set of knowledge point mastery probability; the output at each time step. In, each dimension Explained as students in Always keep the knowledge points The probability of mastery, satisfying Continuous assignment, Map weight matrices to knowledge points. For bias, Use the Sigmoid activation function;

[0028] By comparison and Construct monotonicity constraint terms for the output of each knowledge point. ,in This represents the total number of knowledge points involved in the current problem. This is the set of knowledge point numbers involved in the current problem. The value is 0 or 1, representing whether the student answered the question correctly or incorrectly, respectively; a correct answer is considered correct. At that time, the probability of mastering the knowledge points involved in the current problem. Higher than the probability of knowing counterfactual facts When a question is answered incorrectly, the probability of mastering the current knowledge point is... Lower than the probability of knowing counterfactual facts .

[0029] Furthermore, in step 4, for the current moment... , the main branch knowledge state With rating feature vector Merging involves first encoding the scoring feature vectors using a multilayer perceptron and mapping them to the latent space to obtain a unified high-level representation. , Then, the parts are spliced ​​together to obtain... , ;

[0030] Dimension reduction projection of the concatenated vector , ,in, The parameters for the fusion layer are given; a multi-head self-attention mechanism is introduced, and the final scoring time-series features are obtained as follows: .

[0031] Furthermore, in step 5, the overall loss function is: ,in The weights of the monotonicity loss term are given by the binary cross-entropy loss: ,in Given the total number of samples, the monotonicity constraint loss is... Used to improve the theoretical consistency of the model; monotonicity constraint loss. ,in This represents the total number of knowledge points involved in the current problem. This is a set of numbered knowledge points related to the current problem.

[0032] The present invention also provides a knowledge tracking system for automatic generation of all attributes based on a large language model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a knowledge tracking method for automatic generation of all attributes based on a large language model as described above.

[0033] The present invention also provides a computer storage medium storing a computer-readable program, characterized in that: when the computer-readable program is invoked, it executes a knowledge tracking method for automatic generation of all attributes based on a large language model as described in the above technical solution.

[0034] This invention presents a knowledge tracking method based on a large language model that automatically generates knowledge points from all attributes. It can automatically attribute the knowledge points in the questions and generate weighted embeddings, effectively integrate multi-source features such as questions, answers, and scores, and dynamically model students' knowledge states using a multi-head attention mechanism. It also compares and constrains the evolution of real and counterfactual knowledge, significantly improving prediction accuracy and model interpretability. This provides a scientific tool for intelligent academic profiling, personalized teaching recommendations, and educational decision-making. Attached Figure Description

[0035] Figure 1 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, a method for automatically generating knowledge tracing based on a large language model includes the following steps:

[0038] Step 1, Construct the Knowledge & Scoring Feature Extraction Module (KSFEM): For each question... Using Large Language Model (LLM) combined with specific analysis prompts Automatically generated A knowledge concept and their weights The semantic representation of the question is obtained by embedding knowledge concepts into a pre-trained language model and then weighting the results accordingly. Similarly, The student's answer to each question is then combined with the question text to obtain the question-answer text. Next, the question-and-answer text will be displayed. With specific analysis prompts The data is input into a large model to obtain multidimensional scoring features of the answers. ;Specifically:

[0039] (1-1) Knowledge Feature Extraction

[0040] Knowledge and rating feature extraction, such as Figure 1 As shown in 'a', for knowledge concept extraction, firstly, for each question, the module extracts its question text. (in , (This refers to the total number of questions) and specially designed question analysis prompts. After concatenation, the data is input into a large pre-trained language model (LLM). Analysis of prompts effectively guides the model to focus on the sub-task of knowledge point mining, enabling the large model to automatically analyze the semantic content of the question text and abstract concepts, thereby accurately identifying the top five core knowledge concepts most relevant to the question and their corresponding weights relative to the question. , This refers to the attribution reasoning process in large language models. It is a collection of knowledge concepts generated by the large model based on the content of the question text, that is ,in The maximum number of knowledge concepts, where . This represents the concept of each knowledge point. The set of weights corresponding to the knowledge concepts generated for the large model, i.e. , In this context, the sum of the weights of all knowledge concepts given by the large model for a given problem is 1, that is... Meanwhile, since the knowledge concepts provided by the large model are phrases, the model uses ModernBERT to process each knowledge concept and provides corresponding word embeddings, i.e. ,in It is the set of word embeddings generated by the BERT model for each knowledge concept, i.e. ,in Embedding words corresponding to knowledge concepts, , is the dimension of the word embedding vector.

[0041] Considering the importance weights of each knowledge concept, the model performs a weighted sum of their embedding vectors to obtain the final knowledge semantic representation vector for the problem. ,in This vector can comprehensively reflect the distribution of the core knowledge points covered by the question and its internal structure, providing a fine-grained semantic reference for distinguishing different types of questions.

[0042] (1-2) Scoring Feature Extraction

[0043] For each question-and-answer data point, the model will process the question text... and the answer texts submitted by the students (in , (To form a unique question-and-answer combined text, based on the total number of answers) ,in This is a concatenation operation. The model combines this text with another set of analytical prompts specifically designed for evaluating the quality of responses. All data is input into the LLM model. The model will conduct a detailed analysis and multi-dimensional scoring of each response's performance in solving the original problem based on six key dimensions, such as accuracy, completeness, comprehensibility, and relevance. The large model will output a comprehensive score, forming a feature vector of the response's score across different evaluation dimensions. . ,in As evaluation dimensions.

[0044] Step 2, construct the Question Semantic Embedding & Fusion Module (QSEFM): First, the current time... The semantic representation of the question Perform dimensionality reduction to reduce the dimension to the level of the embedded question number. With the same dimension, we obtain the dimensionality-reduced semantic representation of the question. Next, the semantic representation of the problem after dimensionality reduction is... embedding question numbers Perform LayerNorm operations respectively to obtain and This reduces the interference of norm differences on fusion and improves training stability and convergence speed; then... and Perform adaptive fusion operation on the gating network to obtain the problem representation vector. ;Specifically:

[0045] The question semantic embedding obtained by embedding and fusing using BERT We extract interpretable information from the question text and knowledge structure, enabling us to generalize to unseen questions; we assign an independent embedding vector to each question based on its question number. ,in Using this as the feature dimension allows us to learn instance-level latent features (such as difficulty) from historical data, characterizing individual differences and statistical patterns. To fuse the question semantic embedding and the question number embedding for more stable and higher overall performance, we use a gating mechanism for fusion, such as... Figure 1 Part b in the text. Due to the dimension of the semantic embedding of the problem. With feature dimension Unlike others, we perform a dimensionality reduction operation on the semantic embedding of the question, that is... ,in However, due to the differences in scale and statistical distribution between the two embeddings, and the potential for internal covariate shifts, we use LayerNorm to address this. This brings them to a consistent scale, reduces the interference of norm differences on fusion, and improves training stability and convergence speed. , , ,in This represents the semantic embedding of the question at time t. The semantic feature vectors after LayerNormalization ensure the stability of the feature distribution. This indicates the embedding of the question number at time t. The results of Layer Normalization characterize the structured information in the sequence of questions.

[0046] Meanwhile, since different samples have varying degrees of dependence on semantic generalization and numbered memory, and different upstream noise levels, we use gated fusion to handle this. This allows for adaptive allocation of weights for the two embeddings, favoring semantics during cold starts, strengthening numbered memory when data is plentiful, and suppressing unreliable sources to obtain a more robust representation. The final fused question representation vector is as follows: , Among them, the fusion weight vector , , and These are the weight matrix and bias vector of the gating network, respectively, both of which are learnable parameters. This represents the vector concatenation operation. This represents the Sigmoid operation, which merges weights. The dynamic gating coefficients are consistent with the dimension of the feature vector, which can adaptively determine whether semantic features or structural features should be emphasized at the current moment.

[0047] Step 3, Construct the Sequential Knowledge State Modeling & Monotonicity Constraint Module (SKSMMC): First, construct the main branch knowledge state, setting the current time... Problem representation vector Its corresponding answer result is embedded By fusing the data, a joint representation can be obtained. Meanwhile, to improve the model's sensitivity to fluctuations in students' cognition, a counterfactual branch is constructed, which flips the most recent answer and generates the corresponding counterfactual feature. This is to achieve comparative modeling of knowledge states under real and virtual paths. In the main branch knowledge state modeling process, a multi-head attention mechanism is employed, based on the current... Semantic representation of the problem after time fusion For query vector Every point in history Characteristics of answering questions as a key vector and numerical vectors It performs weighted aggregation of historical behaviors and ultimately outputs the current moment. knowledge status The same modeling process is applied to counterfactual sequences to obtain the corresponding counterfactual knowledge states. Simultaneously, monotonicity constraints are introduced to define the knowledge state. Mapping to the probability space of knowledge point mastery Regarding the knowledge points covered in the current question, when a student answers correctly, what is the probability that they have mastered those knowledge points? The probability of mastering knowledge points constructed from counterfactual states should be higher than that of knowledge points constructed from counterfactual states. If the answer is incorrect, the value should be lower than the counterfactual state. Simultaneously, a contrastive loss function should be used. By incorporating deviations from the monotonicity assumption into the penalty term, the model's interpretability and robustness in inference boundary areas and sparse data regions are effectively improved. Specifically:

[0048] Students' response sequences contain rich temporal dependencies and knowledge transfer information. To fully explore the dynamic characteristics of this information, we utilize a multi-head attention mechanism to establish and update students' knowledge states. This mechanism integrates temporal decay patterns, knowledge relevance discrimination, and multi-scale information aggregation, enabling it to efficiently identify and reinforce historical response fragments that influence current predictions, thus improving the granularity and adaptability of knowledge state modeling. Figure 1 Part c in the text.

[0049] (3-1) Construction of features and anti-features

[0050] In modeling students' knowledge states, we found that modeling a single response path often fails to fully reflect the sensitive changes in students' underlying cognitive processes. Therefore, we innovatively designed a sequence comparison mechanism to uncover the response characteristics of students' knowledge states to sudden changes in responses, thereby improving the model's ability to analyze causal relationships and determine inference boundaries.

[0051] Specifically, students in time step 1 to The process of answering questions is represented as a sequence of questions. Sequence of Response Results Each The value can be 0 or 1, representing the student's response to the question. The answer may be correct or incorrect. Unlike traditional methods, we introduce a counter-response sequence based on the original sequence. ,Right now Only the most recent answer in the sequence is flipped, while the previous trajectory is preserved. By comparing the knowledge states under real and "virtual" paths, the diversity and representativeness of the input data are improved, while also laying a theoretical foundation for subsequent comparison and constraint mechanisms.

[0052] The problem representation vector obtained at the current time t We configure proprietary embeddings for both types of response results. , ,in Embed the result into a matrix. The first row contains the "incorrect" embeddings, and the second row contains the "correct" embeddings. Then, the question embeddings and the corresponding answer result embeddings are fused together, specifically by summing, to obtain a joint representation. , This achieves a tight integration of question semantics and actual response signals. This design explicitly reflects the interaction characteristics between the question and response, providing synchronized and aligned contextual information for subsequent relationship modeling. For counterfactual branches, we only need to input the response result at the input layer. Replace with counterfactual labels Generate corresponding counterfactual features , In fact, the introduction of counterfactual input also provides theoretical support for the interpretability of model reasoning ability, enabling it not only to "track" students' current mastery, but also to infer "how students' knowledge would change if they answered the previous question correctly or incorrectly," thus revealing the micro-dynamic nature of cognition more accurately.

[0053] (3-2) Knowledge State Modeling

[0054] First, in order to focus on the present moment To investigate the potential connections between sequence number and historical behavior, a multi-head attention mechanism is employed to dynamically model the semantics and historical behavior at each time step. At each time step, the semantic fusion embedding of the problem at that moment is first obtained. as query vector and every point in history Response feature encoding as a key vector and numerical vectors , This mapping lays the representational foundation for downstream attention allocation. Under the multi-head attention mechanism, Will be projected to Each subspace has a dimension of [number] heads. , making .

[0055] To understand sequence associations separately from different subspaces, for each attention head The input embedding is projected into each subspace through a learnable linear mapping, i.e. , , , here , These are the query, key, and value parameters for each head. Within each attention head, we use the inner product to measure the relevance between the current context and historical nodes, and calculate content similarity. , This normalized dot product can effectively prevent excessively large scores in large dimensions and ensure the stability of the score distribution.

[0056] Meanwhile, to better simulate the "recency effect," we introduce a time-decreasing weight for each historical node. Specifically, we define the current interval and the historical interval. And set a learnable decay rate parameter. (One copy per head), thus obtaining the time-series decay factor. , By introducing this mechanism, historical nodes that are closer to the current moment can be assigned higher attention, reflecting the forgetting characteristics of dynamic memory. At the same time, each attention point can autonomously adjust its forgetting curve.

[0057] Next, the model multiplies the content similarity score by the temporal decay factor to obtain a weighted score, and then normalizes all historical nodes using softmax to obtain the final attention distribution. ,in No. The height in Time is assigned to historical nodes The normalized attention weights are then applied. Next, each attention head, based on the above allocation, performs a weighted sum of all historical value vectors to obtain the context state representation of that head. , Finally, combining all The context outputs of each head are concatenated and then output linearly projected. The final knowledge state representation is obtained. , .in This represents the current state of knowledge. Similarly, for counterfactual state of knowledge... The structure is the same, representing each point in history. Response feature encoding Replace with counterfactual response feature encoding as a key vector and numerical vectors That's it. Through the above structure, the model can comprehensively consider the relevance of the current context and historical behavior in terms of content, as well as the decreasing trend in time sequence, thereby adaptively allocating attention and achieving dynamic tracking and perception of the state of historical knowledge.

[0058] (3-3) Monotonicity Constraint

[0059] Cognitive psychology research clearly demonstrates that after a student answers a question correctly, the probability of mastering related knowledge points should increase, while an incorrect answer should appropriately decrease it. To explicitly incorporate this educational measurement theory, the model introduces contrastive monotonicity loss to reinforce the theoretical expectation that knowledge state transfer aligns with the transition. Furthermore, since students' actual knowledge state is often influenced by multiple knowledge points simultaneously, and manifests as different granularities of ability expression for different test questions, the model needs to precisely map the sequence features at each time step to the mastery probability of each knowledge point. The global knowledge point set is as follows: , This represents the total number of knowledge concepts, corresponding to the knowledge point tags in the question bank. Multi-knowledge point distribution modeling is achieved through linear transformation of a fully connected layer and Sigmoid normalization. , ,in Normal knowledge state The transformed set of knowledge points has a probability set and ,Right now ,and State of counterfactual knowledge The transformed set of probabilities of mastering knowledge points. The output at each time step. In, each dimension This can be interpreted as students in Always keep the knowledge points The probability of mastery, satisfying The continuous assignment of values ​​can not only reflect the differences in capabilities, but is also applicable to subsequent tasks such as downstream problem prediction and knowledge intervention. Map weight matrices to knowledge points. For bias, This is the Sigmoid activation function.

[0060] By comparison and For the output of each knowledge point, we construct a monotonicity constraint term. ,in This represents the total number of knowledge points involved in the current problem. This is the set of knowledge point numbers involved in the current problem. The value can be 0 or 1, representing the student's response to the question. The answer is either correct or incorrect. Correct answer ( When ), the probability of mastering the knowledge points involved in the current problem. It should be higher than the probability of knowing counterfactual facts. When a question is answered incorrectly, the probability of mastering the current knowledge point is... It should be lower than the probability of knowing counterfactual facts. Any deviation from theoretical expectations will be penalized by this loss term. This loss term design significantly improves the model's sensitivity to changes in knowledge acquisition, allows the output to better interpret the monotonicity assumption in pedagogy, and has stronger generalization ability in sparse data regions. Furthermore, this constraint helps the model improve the reliability of its output under boundary samples, greatly enhancing the model's credibility and explanatory power in practical applications.

[0061] Step 4, construct the Scoring Feature Modeling Auxiliary Module (SFMAM): obtain the current time step from the KSFEM module. Multidimensional scoring characteristics of responses By encoding rating features using a multilayer perceptron (MLP) and mapping them to a unified latent space, a higher-order rating representation can be obtained. Then, the knowledge state of the main branch will be... With rating feature encoding Feature concatenation is performed to form a parallel fused feature representation. The fusion features after splicing. This indicates that a dimension-reduced linear projection is used to obtain comprehensive fused features. A multi-head self-attention mechanism is applied to the rating feature sequence, focusing on key behavioral nodes and dynamically adjusting weights according to time to model the temporal changes in student rating behavior. This results in the output of rating temporal features. . Specifically:

[0062] In knowledge tracing tasks, relying solely on knowledge point mastery modeling is insufficient to fully reveal students' actual cognitive processes. Therefore, we utilize a large model to conduct detailed analysis and multi-dimensional scoring of each answer's performance in solving the original problem, obtaining scoring characteristics for each answer. These characteristics reflect problem-solving strategies, cognitive depth, and behavioral inertia. Scoring characteristics, as important process signals, are crucial for characterizing students' knowledge transfer and performance changes. Furthermore, since scoring characteristics reflect students' behavioral motivations and process changes, while the main branch knowledge embedding directly characterizes the knowledge state, their signal sources and meanings are complementary. To enable the model to simultaneously understand "what knowledge students have mastered" and "students' actual score performance in the problem-solving process," we employ feature concatenation operations for the current moment... Output the main branch With rating feature encoding However, since the two features have different dimensions, we first encode the scoring features using a multilayer perceptron (MLP) and map them to the latent space to obtain a unified high-level representation. , Next, we pieced them together to obtain... , This splicing mechanism not only preserves the parallel information of knowledge state and scoring performance, but also allows the model to automatically learn the effective collaborative relationship between various inputs in the fusion layer, laying the foundation for subsequent multi-scale interaction.

[0063] To further enhance the expressiveness of the fused features and fully explore the potential correlations between the two types of information, we perform dimensionality reduction projection on the concatenated vector. , ,in, These are the parameters for the fusion layer. Thus, the fused features encompass a comprehensive representation of the student's current knowledge state and scoring behavior, supporting more refined temporal decision-making. The fused scoring feature sequence also needs to have its dynamic impact over time modeled. We introduce a multi-head self-attention mechanism, enabling the model to autonomously focus on important behavioral segments in the scoring sequence relevant to the current node and automatically adjust weight allocation based on temporal proximity. The final obtained temporal scoring features are... It is used together with the output of the main knowledge branch for downstream problem prediction tasks.

[0064] Step 5, make predictions: predict the knowledge state of the main branch. With rating time series characteristics By performing feature concatenation, a comprehensive student representation feature is obtained. Integrating features The input is a fully connected prediction network, which mitigates the risk of overfitting in high-dimensional space through multi-layer nonlinear transformations, and finally outputs the probability of the student's answer to the current question. .

[0065] To achieve a probabilistic estimation of students' future answer results, we further concatenate the main branch knowledge state output with the scoring time-series features to obtain the final fused features. , ,in This indicates that the features are concatenated along the last dimension. Subsequently, we input the fused features sequentially into the following fully connected prediction network, which outputs the student's predicted score for correctness on each question. ,in It is a fully connected layer. To alleviate overfitting, This represents the predicted score for each question. The loss function is a weighted sum of the binary cross-entropy loss and the monotonicity constraint loss; the overall loss function is... ,in The weights of the monotonicity loss term are given by the binary cross-entropy loss: ,in The total number of samples, and the monotonicity-constrained loss term. Used to improve the consistency of the model with theory.

[0066] Table 1 below compares our method with baseline methods. We selected classic or state-of-the-art methods as baselines. The top of the table shows the datasets, and the rightmost shows the existing models. The data in the table are AUC and ACC metrics, with the best results highlighted in bold. The table shows that our method, LLM-DFAKT, consistently outperforms other baseline models across all datasets, demonstrating its significant advantage in improving response prediction accuracy.

[0067] Table 1. Comparison results between the method of the present invention and existing methods.

[0068] This invention also provides a knowledge tracking system for automatic generation of all attributes based on a large language model, comprising:

[0069] One or more processors;

[0070] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the full-attribute automatic generation knowledge tracking method based on a large language model as described above.

[0071] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically generating knowledge tracking based on a large language model, characterized in that, Includes the following steps: Step 1: Construct a knowledge and rating feature extraction module to obtain the semantic representation of each question text, and combine the answer content of each question with the question text to obtain multi-dimensional rating features about the answer; Step 2: Construct a question semantic embedding and fusion module, and obtain the question representation vector by combining the question semantic representation and question number embedding; Step 3: Construct a module for sequential knowledge state modeling and monotonicity constraints, and output the current time step. The main branch knowledge state and the counterfactual branch knowledge state are mapped to the knowledge point mastery probability space by introducing monotonicity constraints. The specific implementation method for obtaining the knowledge state of the main branch and the counterfactual branch in step 3 is as follows: (3-1) Constructing main branch features and counterfactual features Students in time step 1 to The process of answering questions is represented as a sequence of questions. Sequence of Response Results each The value can be 0 or 1, representing the student's response to the question. The answer is either correct or incorrect, and a counter-response sequence is introduced based on the original sequence. Specifically, this involves reversing the result of the most recent step in the sequence, i.e. ; For the knowledge state of the main branch, the problem representation vector obtained at the current time t The corresponding answer result is embedded ,in Embed the result into a matrix. The first line contains the "incorrect" embedding, and the second line contains the "correct" embedding. Subsequently, the question embedding and the corresponding answer result embedding are fused to obtain a joint representation. , ; For counterfactual branch knowledge states, the answer result is only displayed at the input layer. Replace with counterfactual labels Generate corresponding counterfactual features , ; (3-2) A multi-head attention mechanism is used to dynamically model the semantics and historical behavior of each time step; For the sequence number At each time step, first obtain Moment-based semantic fusion embedding as query vector and every point in history Response feature encoding as a key vector and numerical vectors , Under the multi-head attention mechanism, Will be projected to Each subspace has a dimension of [number] heads. , making ; For each attention head The input embedding is projected into each subspace through a learnable linear mapping, i.e. , , , here , The parameters for each head are the query, key, and value. Within each attention head, the inner product is used to measure the relevance between the current context and historical nodes, and content similarity is calculated. , ; At the same time, a time-decreasing weight is introduced for each historical node; specifically, the current interval and the historical interval are defined. And set a learnable decay rate parameter. Thus, the time-series decay factor is obtained. , ; Next, the content similarity is multiplied by the temporal decay factor to obtain a weighted score, which is then normalized using softmax to obtain the final attention distribution. ,in No. The height in Time, assigned to historical nodes The normalized attention weights are then assigned; subsequently, each attention head performs a weighted sum of all historical value vectors according to the above assignment to obtain the context state representation of that head. , Finally, combining all The context outputs of each head are concatenated and then output linearly projected. This yields the final main branch knowledge state representation. , ,in This represents the knowledge state at the current moment; similarly, it represents the knowledge state of the counterfactual branch. The structure is the same, representing each point in history. Response feature encoding Replace with counterfactual response feature encoding as a key vector and numerical vectors That's all; Monotonicity constraints specifically include: Let the global knowledge point set be , To represent the total number of knowledge concepts, a multi-knowledge point distribution model is achieved through linear transformation of a fully connected layer and Sigmoid normalization. , ,in Normal knowledge state The transformed set of knowledge points has a probability set and ,Right now ,and State of counterfactual knowledge The transformed set of knowledge point mastery probability; the output at each time step. In, each dimension Explained as students in Always keep the knowledge points The probability of mastery, satisfying Continuous assignment, Map weight matrices to knowledge points. For bias, Use the Sigmoid activation function; By comparison and Construct monotonicity constraint terms for the output of each knowledge point. ,in This represents the total number of knowledge points involved in the current problem. This is the set of knowledge point numbers involved in the current problem. The value is 0 or 1, representing whether the student answered the question correctly or incorrectly, respectively; a correct answer is considered correct. At that time, the probability of mastering the knowledge points involved in the current problem. Higher than the probability of knowing counterfactual facts When a question is answered incorrectly, the probability of mastering the current knowledge point is... Lower than the probability of knowing counterfactual facts ; Step 4: Construct a scoring feature modeling auxiliary module to encode multi-dimensional scoring features, combine the main branch knowledge state with the scoring feature encoding to obtain comprehensive fusion features, and apply a multi-head self-attention mechanism to obtain scoring time-series features. Step 5: Concatenate the knowledge state of the main branch with the scoring time sequence features to obtain a comprehensive student representation feature. Input the comprehensive student representation feature into a fully connected prediction network. After multi-layer nonlinear transformation, the final output is the probability of the student's answer score on the current question.

2. The method for automatic generation and tracking of knowledge based on a large language model as described in claim 1, characterized in that: In step 1, regarding the problem text Using Large Language Modeling (LLM) combined with analysis of prompt words Automatically generate knowledge concepts and their weights The semantic representation of the question is obtained by embedding knowledge concepts into a pre-trained language model and then weighting the results accordingly. ;in , This represents the total number of questions.

3. The method for automatic generation and tracking of knowledge based on a large language model as described in claim 1, characterized in that: In step 1, for each question and answer data point, the question text is... and the answer texts submitted by the students Assemble the question and answer the combined text ,in For the splicing operation, among which , The total number of responses; combine the question-response text with another set of analytical prompts. The data is fed into a large language model (LLM), which outputs a comprehensive score that forms the feature vector of the answer across different evaluation dimensions. , ,in As evaluation dimensions.

4. The method for automatic generation and tracking of knowledge based on a large language model as described in claim 1, characterized in that: In step 2, first set the current time The semantic representation of the question Perform dimensionality reduction to reduce the dimension to the level of the embedded question number. With the same dimension, we obtain the dimensionality-reduced semantic representation of the question. Next, the semantic representation of the problem after dimensionality reduction is... embedding question numbers Perform LayerNorm operations respectively to obtain and Then on and Perform adaptive fusion operation on the gating network to obtain the problem representation vector. , , Among them, the fusion weight vector , , and These are the weight matrix and bias vector of the gated network, respectively, both of which are learnable parameters. This represents the vector concatenation operation. This represents the Sigmoid operation.

5. The method for automatic generation and tracking of knowledge based on a large language model as described in claim 1, characterized in that: In step 4, for the current moment , the main branch knowledge state With rating feature vector Merging involves first encoding the scoring feature vectors using a multilayer perceptron and mapping them to the latent space to obtain a unified high-level representation. , Then, the parts are spliced ​​together to obtain... , ; Dimension reduction projection of the concatenated vector , ,in, The parameters for the fusion layer are given; a multi-head self-attention mechanism is introduced, and the final scoring time-series features are obtained as follows: .

6. The method for automatic generation and tracking of knowledge based on a large language model as described in claim 1, characterized in that: In step 5, the overall loss function is: ,in The weights for the monotonicity loss term are given by the binary cross-entropy loss: ,in Given the total number of samples, the monotonicity constraint loss is... Used to improve the theoretical consistency of the model ,in This represents the total number of knowledge points involved in the current problem. This is a set of numbered knowledge points related to the current problem.

7. A knowledge tracking system for automatic generation of all attributes based on a large language model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a method for automatically generating knowledge tracking based on a large language model, as described in any one of claims 1 to 6.

8. A computer storage medium storing a computer-readable program, characterized in that: When the computer-readable program is invoked, it executes the full-attribute automatic generation knowledge tracking method based on a large language model as described in any one of claims 1 to 6.

Images