Answer correctness prediction method based on NLKT knowledge tracking model

Abstract

This invention discloses a method for predicting the correctness of answers based on the NLKT knowledge tracing model, comprising: acquiring exercise text data and knowledge component features; constructing a knowledge tracing model based on natural language understanding, the natural language understanding-based knowledge tracing model including an exercise text understanding module, an exercise difficulty coefficient processing module, and a deep knowledge tracing module, the exercise text understanding module being constructed based on a pre-trained BERT model; inputting the exercise text data into the natural language understanding-based knowledge tracing model, extracting features from the exercise text data through the exercise text understanding module to obtain a comprehensive text vector, calculating and generating a comprehensive difficulty coefficient of the question based on the knowledge component features through the exercise difficulty coefficient processing model, generating a comprehensive feature vector based on the comprehensive text vector and the comprehensive difficulty coefficient of the question, and performing deep knowledge tracing processing on the comprehensive feature vector through the deep knowledge tracing module to obtain the final predicted answer correctness result.

Description

Technical Field

[0001] This invention belongs to the field of knowledge tracing technology, and in particular relates to a method for predicting the correctness of answers based on the NLKT knowledge tracing model. Background Technology

[0002] Online education, as a crucial application of modern information technology in education, has achieved remarkable results. Through platforms such as Intelligent Tutoring Systems (ITS) and Massive Open Online Courses (MOOCs), it leverages the internet and digital technologies to overcome the time and space limitations of traditional education, providing students with flexible and diverse learning methods and abundant learning resources. With the help of artificial intelligence, online education can offer personalized learning recommendations and intelligent assessments, thereby providing students with an efficient and supportive learning experience. Researching the development and application of online education is crucial for building high-quality online learning environments, improving the efficiency of educational resource sharing and utilization, and promoting the openness and inclusiveness of education, ultimately contributing to the creation of a networked, digitalized, personalized, and lifelong education system.

[0003] Knowledge tracing is a key technology in AI-assisted education. Students read exercise texts and apply knowledge to solve problems, extracting the corresponding knowledge sequences and interaction sequences of exercises. This can typically be represented as an ordered pair, where each pair represents a question the student answered at a given time and their answer to that question. Generally, 0 and 1 represent incorrect and correct answers, respectively. By modeling the student's mastery of exercises, skills, or knowledge components, the future knowledge state of the student can be predicted. In other words, it uses the student's historical performance to predict the probability of the student answering questions correctly at the next point in time. Simply put, it predicts the probability of the student answering questions correctly given a sequence of historical learning performance and the number of questions they can answer at a given time. This prediction result can be used for personalized exercise recommendations and personalized learning path design.

[0004] Traditional knowledge tracing models are mainly based on structured interaction data between students and exercises, using one-hot encoding of exercises and knowledge concepts as features. However, they often overlook the rich information contained in the exercise text, resulting in insufficient accuracy in predicting the correctness of students' answers. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method for predicting the correctness of answers based on the NLKT knowledge tracing model, thereby resolving the issues present in the existing technologies.

[0006] To achieve the above objectives, this invention provides a method for predicting the correctness of answers based on the NLKT knowledge tracing model, comprising: Obtain exercise text data and knowledge component characteristics; A knowledge tracking model based on natural language understanding is constructed. The knowledge tracking model based on natural language understanding includes a problem text understanding module, a problem difficulty coefficient processing module, and a deep knowledge tracking module. The problem text understanding module is constructed based on a pre-trained BERT model. The exercise text data is input into the knowledge tracking model based on natural language understanding. The exercise text understanding module extracts features from the exercise text data to obtain a comprehensive text vector. The exercise difficulty coefficient processing model calculates and generates a comprehensive difficulty coefficient for the question based on the knowledge component features. A comprehensive feature vector is generated based on the comprehensive text vector and the comprehensive difficulty coefficient for the question. The deep knowledge tracking module performs deep knowledge tracking processing on the comprehensive feature vector to obtain the final predicted answer correctness result.

[0007] Optionally, the process of feature extraction from exercise text data includes: The exercise text data is preprocessed and segmented, and includes paragraph information, question information, and option information. The paragraph information and question information are divided into paragraph sequences and question sequences, respectively. The paragraph sequences are then divided using a sliding window to generate several window segments, each of which contains a fixed number of tokens and has overlapping parts. For different window segments, the window segment, question sequence, and option information are concatenated to obtain several input sequences; The BERT model processes several input sequences to obtain several output vectors. The output vector is weighted using an attention mechanism, and a weighted sum is calculated based on the assigned weights to generate a comprehensive text vector.

[0008] Optionally, the process of generating the overall difficulty coefficient of the problem includes: The overall difficulty level of the problem Generated based on a basic difficulty score and a weighted score that incorporates multiple knowledge components: in, Basic difficulty rating The difficulty-weighted scoring is applied to multiple knowledge components, where KC represents the corresponding knowledge component feature. Indicates performance adjustment items, The basic difficulty score is obtained based on the accuracy of the questions. The multi-knowledge-component difficulty weighted score is obtained by weighting and summing different knowledge-component features, including literal comprehension features. Characteristics of reasoning and understanding and characteristics of critical understanding , in These are the weights of literal comprehension features, inferential comprehension features, and critical comprehension features, respectively.

[0009] The calculation method for performance adjustment items is as follows: in, and These represent the number of times a student succeeds and fails on a specific KC problem, respectively. It is a constant used when the denominator is not zero.

[0010] Optionally, the weights of the literal understanding features, reasoning understanding features, and critical understanding features are obtained through linear regression analysis.

[0011] Optionally, the process of generating the comprehensive feature vector includes: Obtain user identifiers and exercise identifiers, convert the user identifiers and exercise identifiers into numerical indices through an embedding layer, and map the numerical indices into identifier feature vectors; The overall difficulty coefficient of the problem is normalized by using a fully connected layer to generate a high-dimensional difficulty vector. The comprehensive text vector, the identifier feature vector, and the high-dimensional difficulty vector are concatenated to generate an input sequence. The input sequence is then concatenated to generate a comprehensive feature vector.

[0012] Optionally, the deep knowledge tracking model includes a CNN layer, a multi-head attention layer, a Bi-GRU layer, and a fully connected layer connected in sequence. The process involves using a CNN layer to extract features from the comprehensive feature vector, generating a fused feature vector. A multi-head attention layer then performs self-attention calculation and linear transformation on the fused feature vector to obtain attention output features. A Bi-GRU layer processes the attention output features at different time steps, and a fully connected layer converts the output of the Bi-GRU layer into the final predicted probability of the correct answer to the question.

[0013] Optionally, in the CNN layer, the comprehensive feature vector is convolved, and the dimensionality of the convolved comprehensive feature vector is reduced by max pooling. The dimensionality-reduced comprehensive feature vector is then fused and concatenated to generate a fused vector.

[0014] Optionally, the knowledge tracking model based on natural language understanding is optimized using a binary cross-entropy loss function: The binary cross-entropy loss function is: in, It is the number of training samples. It is the first The students in time step The actual answer result (1 indicates correct, 0 indicates incorrect). It represents the probability of the model predicting the correctness of the answer.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: The integrated application of advanced text embedding technology: For the first time, the pre-trained BERT model is deeply integrated into the deep knowledge tracking model to process the passage text, question text, and choice text of reading comprehension questions, and convert them into text embedding matrices. This integrated application not only improves the expressive power of text features, but also captures the deep semantic information in the text more accurately, providing strong support for understanding complex reading comprehension tasks.

[0016] A custom dynamic difficulty coefficient algorithm not only considers the inherent attributes of the questions, such as features related to knowledge components (KCs), including textual comprehension (kc_literal), inference (kc_inferential), and critical thinking (kc_critical), but also dynamically adapts to different types of reading comprehension questions. It determines the specific contribution of each knowledge component to the difficulty level through a data-driven approach. The advantage of this method lies in its reliance on actual student performance, dynamically adjusting the difficulty assessment based on student answer data. Therefore, it more accurately reflects the true difficulty level of the questions, making it more precise and personalized.

[0017] A bidirectional GRU deep knowledge tracking model combining multi-head attention mechanism: By integrating convolutional neural network (CNN), multi-head attention mechanism and bidirectional gated recurrent unit (Bi-GRU), it effectively captures local and global features and temporal dependencies of data, and provides an effective strategy, especially in dealing with the complex features and dependencies of student answer data. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a diagram of the NLKT model architecture according to an embodiment of the present invention; Figure 2 This is a flowchart of the pre-trained BERT exercise text processing according to an embodiment of the present invention; Figure 3 This is a diagram illustrating the multidimensional dynamic problem difficulty algorithm architecture of an embodiment of the present invention. Figure 4 This is a diagram of the GRU-CKT module architecture according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a bidirectional GRU gated loop unit according to an embodiment of the present invention; Figure 6 This is a diagram of the internal architecture of the GRU gated loop unit according to an embodiment of the present invention; Figure 7 This is a comparison chart of clustering capabilities in an embodiment of the present invention; Figure 8 This is a comparison chart of the embedding vector context information capture capabilities of embodiments of the present invention. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0021] This invention proposes a Natural Language Understanding-based Knowledge Tracing Model (NLKT) to evaluate students' answer accuracy. This method aims to improve knowledge tracing performance by employing advanced Natural Language Understanding (NLU) technology to capture different semantic relationships within the context of the exercise text. Taking reading comprehension tasks as an example, it enhances the prediction accuracy of student answer correctness by deeply analyzing students' reading comprehension interaction data. The NLKT model combines a pre-trained BERT model, a custom difficulty coefficient algorithm, and a composite deep learning structure (including CNN, multi-head attention mechanism, and Bi-GRU) to improve the accuracy of answer prediction and support the design of personalized learning paths.

[0022] Model Architecture To understand and predict student performance on tasks heavily reliant on text-based exercises, such as reading comprehension, this invention proposes a Natural Language Understanding-based Knowledge Tracking Model (NLKT). This model integrates pre-trained BERT models, Convolutional Neural Networks (CNNs), Gated Recurrent Units (GRUs), and Multi-head Attention mechanisms, among other deep learning techniques. The NLKT model primarily consists of three parts: a text-based exercise understanding module (Sem-TextEG module), a question difficulty coefficient processing module (Multi-DDiff module), and a deep knowledge tracking module employing a bidirectional GRU with multi-head attention (GRU-CKT module). The model's goal is to predict the correctness of students' answers based on the exercise text material, questions, options, and question difficulty in the interaction sequence. The model architecture is as follows: Figure 1 As shown: Exercise Text Comprehension Module Utilizing natural language understanding (NLP) technology to delve into the rich semantics of exercise content is crucial for understanding and analyzing student learning behavior. The Exercise Text Understanding Module (Sem-TextEG) in the NLKT model leverages a pre-trained BERT model combined with sliding window technology to effectively process long exercise texts, ensuring text integrity and contextual coherence. This method addresses the sequence length limitation encountered when processing long texts and guarantees a comprehensive understanding of the deeper meaning of the exercise texts.

[0023] Specifically, the goal of the exercise text understanding module is to generate deep semantic feature embeddings of the exercise text data, i.e., a comprehensive feature vector of the text data, such as... Figure 2 As shown, this module uses a pre-trained BERT model to vectorize text. For long paragraphs of text, such as those used in reading comprehension, a sliding window method is employed to overcome the maximum length limitation of the BERT model when processing long sequences, while simultaneously maintaining the contextual information of the text. By segmenting long texts into several overlapping small window segments and processing each segment independently, the vector representations of each segment can be effectively aggregated without sacrificing the semantic integrity of the text, thereby achieving a deeper understanding and analysis of the overall text.

[0024] First, this module preprocesses and segments the exercise text data. The exercise text data includes paragraph information, question information, and option information. The paragraph and question information are segmented into paragraph sequence P and question sequence Q, both of which are token sequences. Then, a sliding window technique is used to segment the paragraph sequence P, generating multiple overlapping window segments. Each window contains a fixed number of tokens and is designed with a certain degree of overlap to maintain text coherence.

[0025] Next, the input sequence is constructed. For each window segment divided by the paragraph sequence P, it is compared with the question sequence Q and each option information. The sequences are concatenated to form multiple independent input sequences. Represents the first segment of P after being segmented by the sliding window. i A window segment, input sequence It is And the question sequence Q and the option sequence The concatenated sequence is represented as follows: Next, this module processes each input sequence using the BERT model, obtaining the text semantic feature vector representation corresponding to the [CLS] marker in each input sequence as the output vector for that input sequence. That is, for... The output vector obtained through BERT processing is represented as : To synthesize information from the entire exercise text data, this module aggregates the output vector representations of all window fragments. This is done by dynamically assigning an attention mechanism to the output vector of each window. Assign weights i and j represent different window segment labels, and N represents the total number of corresponding window segments. This is used to measure the relative importance of each window segment. After calculating the weighted average, a comprehensive text vector is obtained. : This approach not only effectively preserves the contextual information of the text but also avoids information loss that may occur due to text length limitations. By aggregating the vector representations of each window, a comprehensive understanding of the entire text content can be achieved, providing an effective text vectorization strategy for improving knowledge tracking performance in reading comprehension tasks.

[0026] Exercise Difficulty Coefficient Processing Module In intelligent education systems, accurately assessing the difficulty of exercises is crucial for achieving personalized learning paths and improving learning outcomes. To address this need, the NLKT model introduces a module for processing exercise difficulty coefficients—the Multidimensional Dynamic Difficulty Algorithm (Multi-Ddiff)—providing an innovative solution for knowledge tracking and difficulty adjustment in complex tasks such as reading comprehension.

[0027] The Multi-Ddiff algorithm integrates the knowledge structure of the exercises, such as knowledge components (KCs) features, including textual comprehension features, reasoning comprehension features, and critical comprehension features. These features are fields inherent in the dataset. They indicate whether understanding the literal meaning of the question, reasoning, or critical comprehension is required to answer a question correctly. In the dataset, if a question involves all three parts, these three fields are set to 1; otherwise, they are empty. Furthermore, a dynamic adaptation mechanism is employed. By analyzing student answer data, the contribution of each knowledge component feature to the difficulty of the exercises is dynamically adjusted in a data-driven manner. Figure 3 As shown. Its advantage lies in its ability to more realistically and accurately map the difficulty level of exercises, i.e., the difficulty coefficient of the exercises, supporting more precise and personalized learning path planning. The specific method is as follows: Overall difficulty level of the problem Based on basic difficulty rating Weighted scoring based on the difficulty of multiple knowledge components Joint decision: in, This represents the baseline difficulty score calculated based on the overall performance of all students on a specific question. It is estimated using the accuracy rate of the questions, and the calculation formula is as follows: Indicates the number of correct answers to a question. This indicates the total number of times the question has been answered.

[0028] Multi-knowledge component difficulty weighted scoring Considering the impact of different knowledge components (KCs) on the difficulty of the problem, a specific weight is defined for each knowledge component KC, and the calculation formula is as follows: in These are the weights of literal comprehension features, inferential comprehension features, and critical comprehension features, respectively.

[0029] Performance Adjustment Items The difficulty of questions is dynamically adjusted by considering users' historical performance in literal comprehension, inferential comprehension, and critical comprehension. If a user frequently answers incorrectly in a certain knowledge component (KC), then that question is considered more difficult for that user. This adjustment is calculated as follows: in, and These represent the number of times a student succeeds and fails on questions involving a specific knowledge component, KC. It is a small constant (e.g.) ), used to ensure that the denominator is not zero.

[0030] In this process, weight This was obtained using a linear regression analysis algorithm. In the linear regression analysis algorithm: 1. Input: Dataset D = {(Qi, KCsi, Difficulty;)}, where i = 1, ..., n, Q = {Q1, Q2, ..., Q} is the problem set, KCsi = (kc_literal;, kc_inferential;, kc_critical;) is a binary vector representing the existence of knowledge components in problem Qi, and Difficultyyi ∈ R is the difficulty level of problem Qi.

[0031] 2: Output: Vector W∈Rm, where m is the number of knowledge components and represents the weight of each knowledge component.

[0032] 3: Initialize the linear regression model M.

[0033] 4: Construct the feature matrix X∈{0,1}n×m and the target vector y∈Rn.

[0034] 5: for i=1 to n do 6: X[i]←KCsi 7: y[i]←Difficultyi 8: end for 9: Train model M on dataset (X, y).

[0035] 10: M ← Fit (X, y), minimize ||XW-y||2.

[0036] 11: Extract the weight vector W from model M.

[0037] 12: W ← coefficient (M) 13: Return the weight vector W Linear regression is a predictive analysis method used to find the relationship between one or more independent variables and a dependent variable. Its basic form can be expressed as: in, It is the dependent variable. It is the independent variable. It is the intercept term. These are the coefficients (weights) of the independent variables, representing the degree of influence of each independent variable on the dependent variable. This is the error term.

[0038] In the Multi-Ddiff algorithm for dynamic problem difficulty, the dependent variable is the difficulty of the problem, and the independent variables are the presence or absence of three knowledge components. This analysis can reveal the actual contribution of each knowledge component to the problem difficulty, thus enabling the problem difficulty score to not only reflect the student's overall performance but also to be personalized to each student's specific situation.

[0039] Input modeling The input modeling process aims to integrate exercise text data, user information, and question difficulty scores into a comprehensive feature vector, which can then be effectively used in the model. Specifically, the model encodes and integrates the input data through the following steps: (1) Text embedding. In the exercise text understanding module, the BERT model is used to process the exercise text data. The chapter and paragraph information, question information and option information are converted into a vector with 768 feature dimensions. The text data is divided into multiple paragraphs, and the information of each paragraph is vectorized separately. The sliding window strategy is used to process the segmented paragraph sequence, and the segments are concatenated and weighted to calculate a comprehensive text vector.

[0040] (2) Embedding of User ID and Problem ID. The User ID and Problem ID are converted into numerical indices through an embedding layer, and the numerical indices are mapped into dense identifier feature vectors, including User ID embedding vectors and Problem ID embedding vectors.

[0041] (3) Embedding of the comprehensive difficulty coefficient of the problem. In order to make the comprehensive difficulty coefficient of the problem compatible with other feature vectors and facilitate subsequent feature fusion, the comprehensive difficulty coefficient of the problem obtained by the multidimensional dynamic difficulty algorithm module is normalized by a fully connected layer and mapped to a higher-dimensional space to generate a high-dimensional difficulty vector. This is represented as: in, This indicates that the problem's overall difficulty coefficient d is normalized and then mapped to a higher-dimensional space.

[0042] (4) Model Input Concatenation. Before inputting the above-mentioned integrated text vector, identifier feature vector, and high-dimensional difficulty vector into the neural network, this invention concatenates the above feature vectors into a unified feature vector. This process ensures that the dimensions of all features are correctly aligned to form a comprehensive feature vector representation. Sample Integrated Feature Vector Represented as: The symbol “;” represents the vector concatenation operation, making… , The dimension extracted by the BERT model is The text feature vector, It is a dimension of The user ID embedding vector, It is a dimension of Problem ID embedding vector, It is a scalar value representing the difficulty of the problem. It is a high-dimensional difficulty vector representation of the problem difficulty after it has been transformed by a fully connected layer.

[0043] For the entire comprehensive feature sequence Each of them For each sample at time step t, the above concatenation process is performed to obtain the comprehensive feature vector s: ; in, This represents the sample comprehensive feature vector at time step T, where the superscript T indicates the time step number.

[0044] Deep Knowledge Tracking Module The deep knowledge tracing module, also known as the GRU-CKT module, is the core network architecture of the NLKT model. It is designed to process and analyze input data through a series of neural network layers to predict students' answers to reading comprehension tasks.

[0045] The GRU-CKT module uses a convolutional neural network (CNN) to extract local features, a multi-head attention mechanism to capture dependencies from different perspectives, and a bidirectional gated recurrent unit (Bi-GRU) to understand the contextual relationships of sequential data. The NLKT model aims to predict students' future responses based on their past interactions, employing this composite architecture (such as...). Figure 4 (As shown) To improve the model's prediction performance, the comprehensive feature vector at each time step is sequentially input into the CNN-multi-head attention mechanism-Bi-GRU structure corresponding to different time steps.

[0046] (1) CNN layer In a convolutional layer, multiple filters (convolutional kernels) synthesize each feature vector in the sample. Perform convolution operations to extract rich local features. Each filter Its operation can be expressed as: in, It is a filter The weight, It is a bias term. This represents a convolution operation. It allows each filter to slide across the entire fused vector and capture specific local patterns. This represents the activation function.

[0047] In the pooling layer, max pooling is applied to each local feature to extract the most salient features and reduce the feature dimensionality: Features obtained from pooling The data is then fused and processed to form a long convolutional feature vector. : in, and These are the weights and biases of the fully connected layer, respectively.

[0048] (2) Multi-Head Attention layer In the Multi-Head Attention layer, the convolutional feature vectors are... The system is divided into multiple attention heads. Each attention head undergoes an independent linear transformation of its query vector (Q), key vector (K), and value vector (V). Then, self-attention is calculated for each attention head to obtain its output. Finally, the outputs of all attention heads are concatenated together and a linear transformation is performed on the output.

[0049] For each attention head h, first... Transformed into three different representations: query vector Q, key vector K, and value vector V, with the index h representing the corresponding attention head: in, , , This represents the query vector, key vector, and value vector corresponding to the attention head h. , , These represent the parameter matrices for the query vector, key vector, and value vector, respectively. , , These represent the trainable parameter matrices for the query vector, key vector, and value vector, respectively. This indicates the output of the i-th header.

[0050] For each head, compute the dot product attention between the query and the key, and then apply softmax to obtain the attention weights. : in, It is the dimension of the key vector; this scaling factor helps with training stability.

[0051] The outputs of all the heads are concatenated and then subjected to a linear transformation to obtain the output of the attention layer. : Where W represents the linear transformation coefficients.

[0052] (3) Bi-GRU layer Bi-GRU (Bi-GRU) allows the model to consider both past and future information simultaneously, providing a more comprehensive understanding of the data sequence. The hidden state at each time step is updated jointly using the current input and the previous state. Bi-GRU does not change the internal structure of GRU; it employs two GRU iterations in opposite directions, merging the two sets of GRU results as the final output, as shown below. Figure 5 As shown, this processing method effectively enhances the model's ability to capture and process sequential data, making it suitable for handling time series problems in educational data analysis. By combining forward and backward information flows, Bi-GRU can more comprehensively capture the temporal dependencies in the learning sequence, thereby improving the accuracy of knowledge tracing.

[0053] The GRU gated recurrent unit uses update and reset gates to store and filter information. Because the model does not fade new inputs each time, but instead retains relevant information and passes it to the next time step of the network, it eliminates the gradient vanishing problem. Its unit architecture is as follows: Figure 6 As shown.

[0054] The GRU unit uses the input obtained in the previous step... The hidden state of the previous time step Update its hidden state at each time step t. : The update gate in the model determines how much information from the past should be passed to the future state, and how much information between the previous and current time steps must be retained. The update gate is calculated... The expression is: in, It is a weight matrix. This represents the sigmoid function. Indicates bias.

[0055] The reset gate determines how much past information needs to be forgotten; calculating the reset gate... The expression is: in, It is a weight matrix. This represents the sigmoid function. Indicates bias.

[0056] Updating and resetting gates will affect the final output.

[0057] First, there's the current hidden state, which involves introducing new memory content and using a reset gate to store relevant information from the past: The final hidden state is calculated using the update gate, which determines the information gathered from the current hidden state and the previous step. (4) Output prediction layer Finally, the model uses a fully connected layer to transform the output of the Bi-GRU layer into the final predicted output, which is the probability that the student answers the question correctly at a specific time step: in, The model predicts the probability that a student will answer the next question correctly at time step t. and These are the weights and biases of the output layer. That is, the sigmoid activation function, which ensures that the output value is between 0 and 1.

[0058] Model optimization objective The choice of optimization objective directly affects the effectiveness of model training. To ensure that the proposed Natural Language Understanding-based Knowledge Tracking (NLKT) model can accurately predict students' future performance, the optimization objective is defined as minimizing the Binary Cross-Entropy Loss, allowing the model to finely adjust its weights to improve prediction results. The specific loss function is defined as follows: in, It is the number of training samples. It is the first The students in time step The actual answer result (1 indicates correct, 0 indicates incorrect). This is the probability of the model predicting the correctness of the answer. By minimizing this loss, the model's predicted values ​​gradually approach the true label values, thereby improving the overall prediction performance.

[0059] Model optimization strategy To optimize the above model, the AdamW optimizer was employed. This optimizer improves upon the traditional Adam optimizer by introducing weight decay, which helps enhance the model's generalization ability. The model's weight decay coefficient was set to 0.01. Furthermore, to improve training stability and efficiency, this invention introduces learning rate preheating. Error! Reference source not found. and gradual decay Error! Reference source not found. The strategy involves gradually increasing the learning rate to a maximum value in the early stages of training, and then gradually decreasing it as training progresses. This strategy helps the model converge quickly in the early stages and allows for fine-tuning in the later stages of training, resulting in better performance.

[0060] (1) AdamW optimizer AdamW is an improvement on the Adam optimizer that decouples weight decay from parameter updates, which helps improve the model's generalization ability for each parameter. The parameter update method is as follows: in, and These are the bias correction estimates corresponding to the first and second moments of the parameters, respectively. Is The learning rate at any given time can be dynamically adjusted using a learning rate scheduling strategy. It is the weight decay coefficient. This represents the updated parameters at time t.

[0061] (2) Preheating and gradual decay Learning rate warm-up is the process of gradually increasing the learning rate at the beginning of training, while gradual decay involves decreasing the learning rate after a certain period to help the model converge better. This model employs a strategy of linear warm-up followed by linear decay, with its learning rate... At time step The calculation method is as follows: Preheating stage: During the preheating stage (before...) (Step), the learning rate increases linearly from 0 to the preset maximum value. : The warm-up phase helps stabilize the initial training process of the model and prevents the model from diverging due to an excessively high learning rate at the beginning of training.

[0062] Decay Phase: After the warm-up period, the learning rate decreases from... The learning rate gradually decreases until training ends. A common decay method is linear decay, where the learning rate drops to 0 at the end of training. in, This is the total number of training steps. Let t represent the learning rate at time t, where t represents time t. This indicates the number of steps in the preheating phase.

[0063] Experimental Results and Analysis Dataset This invention uses the AIComprehend Dataset, which involves 5,201 independent interactions between 72 students and 300 different questions. This data not only covers basic user and question information but also includes detailed interaction processes of student responses, providing a rich information foundation for in-depth research into students' learning patterns and knowledge mastery.

[0064] Experimental setup In the specific experiments of this invention, a BERT-based-uncased pre-trained model was selected to embed the exercise text, and a single-layer bidirectional GRU network with a hidden state dimension of 128 was integrated on top of it. Furthermore, a multi-head attention mechanism was introduced to improve the model's learning ability and prediction accuracy. To further enhance the model's performance, a 256-dimensional fully connected layer was used to incorporate the exercise difficulty score into the model.

[0065] All experiments were conducted in a high-performance computing environment equipped with NVIDIA CUDA. Model training used the AdamW optimizer with an initial learning rate of 1e-6 and a weight decay of 0.01. The learning rate followed a linear decreasing strategy after a warm-up period, and the batch size was uniformly set to 32. After processing the input data by the BERT tokenizer, the dimensions of input_ids, attention_mask, and token_type_ids were all [16, 512]. The sequence output dimension obtained after processing was [16, 512, 768], while the output dimensions of the GRU layer and the multi-head attention layer were [16, 512, 256], respectively. The difficulty score was passed through a fully connected layer to obtain a dimension of [16, 1, 256], and then combined with the output of the attention layer to obtain the final combined output dimension of [16, 512, 256]. Ultimately, the model output logits match the dimensions of the training labels [16, 1]. When evaluating performance, AUC and accuracy (ACC) are the main evaluation metrics.

[0066] Comparative Experiments and Analysis To verify the advantages of the Natural Language Understanding-Based Knowledge Tracking (NLKT) model and test its accuracy in predicting students' reading comprehension performance, this invention designed a series of comparative experiments to compare and analyze NLKT with currently popular knowledge tracking models. These experiments aim to explore in depth the innovations of NLKT on existing classic knowledge tracking models and their effects. Four knowledge tracking models were selected for comparison in the experiments: The groundbreaking aspect of the DKT-LSTM model lies in its pioneering application of deep learning to track students' knowledge status and its use of LSTM to reveal the dynamic evolution of knowledge points over time.

[0067] SAKT introduces a self-attention mechanism, which strengthens the model's focus on historical interactions. It learns through an independent encoder structure of skill points and can assign appropriate weights to previous answers during prediction.

[0068] EKT, a model that combines the text information of exercises, uses Word2Vec to encode the exercise words, and uses a variant of RNN—bidirectional LSTM—to track the student's knowledge state, proposes a dual strategy that includes Markov properties and attention mechanisms.

[0069] PEBG is a model that uses a bipartite graph pre-training embedding method to learn the low-dimensional embedding of a problem through edge information, which covers multiple dimensions such as problem difficulty, problem-skill relationship, problem similarity, and skill similarity.

[0070] Table 1 As shown in Table 1, the NLKT model demonstrates superior performance in a series of comparative experiments, validating its significant improvement over existing knowledge tracking models. Compared to the highest performance among the baseline models, the ACC is improved by 1.5%, and the AUC is improved by 1.73%. The NLKT model successfully improves the performance of predicting student reading comprehension exercises through the embedding of high-level semantic features and fine-grained difficulty estimation.

[0071] Ablation Experiments and Analysis To accurately evaluate the effectiveness of each feature vector in the NLKT model, this invention employs an ablation experiment method to explore the changes in model performance after removing certain key features. By comparing the original model with its variants, the contributions of the exercise text embedding features and the question difficulty coefficient embedding features are quantitatively evaluated. Table 2 shows the ablation experiment results of the embedding components.

[0072] Table 2 The ablation experiments in Table 2 show that the removal of text embedding features and question difficulty information leads to a significant decrease in model performance (NLKT_text & diff), indicating the important role of complex text semantic features (i.e., integrated text vectors) and question difficulty metadata (i.e., integrated question difficulty coefficients) in understanding the dynamics of exercise content and its interaction with students. The integration of these embedding components provides a mechanism that allows the model to establish a closer connection between question representation and student knowledge state. When the exercise text embedding feature (NLKT_text) is removed alone, ACC and AUC decrease, but the decrease is smaller compared to the variant that completely removes both features. This indicates that the question difficulty coefficient embedding feature alone has a positive impact on the model. Removing only the question difficulty coefficient embedding feature (NLKT_diff) also leads to a decrease in performance, but it is still higher than the variant that removes both features simultaneously, indicating that the exercise text embedding feature is indispensable in capturing the complex interaction between students and exercises.

[0073] Table 3. Results of network layer ablation experiments Table 3 shows the ablation experiment results: after removing the CNN layer (NLKT_CNN), the model, relying on GRU and multi-head attention mechanisms, achieves high ACC and AUC, indicating that even without local feature extraction from the CNN layer, the GRU joint attention mechanism is sufficient to capture dynamic changes in knowledge state. The variant without the multi-head attention layer (NLKT_MHA) demonstrates the ability of CNN and GRU to maintain model performance without the assistance of attention mechanisms. However, compared to the complete NLKT model, the performance of these two variants is lower, particularly in the AUC metric, indicating the complementary role of these layers in the model and their joint value in accurately tracking student knowledge state.

[0074] Text Embedding Comparison This experimental section aims to evaluate and compare the performance of BERT, Word2Vec, GloVe, and QuesNet in constructing text embeddings for reading comprehension questions.

[0075] (1) Unsupervised clustering was performed using the embedding vectors of paragraphs, questions and answer options to evaluate whether the model could naturally classify related texts based on the text content. The evaluation was based on the silhouette coefficient.

[0076] The silhouette coefficient is a metric for measuring clustering effectiveness, encompassing two key factors: cohesion and separation. Cohesion... It describes how close a data point is to other elements in its class, while the separation degree describes how close a data point is to other elements in its class. This measures the distance between the data point and the nearest group of non-same-class elements. The silhouette coefficient is calculated using these two measures. It provides a means to quantify clustering quality: when When the distance within a class is less than the distance between classes, the clustering result is more compact. The value of S will approach 1. The closer it is to 1, the more distinct the cluster outline. Conversely, when... When the distance within a cluster is greater than the distance between clusters, it indicates that the clustering result is loose. The value of S will approach -1, and the closer it is to -1, the worse the clustering effect.

[0077] Figure 7 The results show that the above-mentioned text embedding techniques can all demonstrate clustering potential based on semantic information such as paragraph content, related questions, and answer options through unsupervised clustering. In this clustering task, QuesNet performs best, while BERT outperforms the static embedding techniques GloVe and Word2Vec, effectively capturing information about word co-occurrence.

[0078] (2) The ability of these techniques to preserve intra- and inter-sentence contextual information is evaluated by comparing the similarity of the embedding vectors of the same words in different sentences, using cosine similarity as the evaluation criterion. Specifically, for a given passage, two independent sentences are constructed by combining a question and two answer options (one correct answer and one randomly selected incorrect answer). Each method generates embedding vectors for these constructed sentences and calculates the cosine similarity between the embedding vectors.

[0079] Given two vectors A and B, the cosine similarity can be calculated using the following formula: in, and express and The model, and They represent and exist The components of position.

[0080] The experimental results are as follows: Figure 8 As shown, BERT is more effective at capturing contextual differences when comparing sentences with different answers, successfully generating embedding vectors that reflect these differences. This means that even if two sentences appear similar, the generated embedding vectors will have relatively low similarity if they contain correct and incorrect answers respectively.

[0081] In reading comprehension tasks, incorrect answers are sometimes not explicitly mentioned in the text, and sometimes appear as distractors, so the calculated similarity is always greater than zero. However, overall, the NLKT method, which combines BERT and sliding window techniques, outperforms other methods in generating exercise text embeddings.

[0082] (3) For a detailed comparison of the performance of BERT, Word2Vec, GloVe and QuesNet in predicting student knowledge state, see Table 4.

[0083] Table 4 shows the effectiveness of the three embedding encoding methods in the knowledge tracking model. The experimental results show that, for the prediction of the interaction of the knowledge tracking reading comprehension task, the best approach is to integrate the reading comprehension passage, question, and choice from the data as part of the input to the knowledge tracking model. Among the above text embedding generation methods, the BERT combined with sliding window strategy adopted in this invention performs the best.

[0084] Table 4 Comparative experiments further validated the unique advantages of the Multi-DDiff algorithm. As shown in Table 5, which compares the average accuracy rate with a custom difficulty coefficient, the Multi-DDiff algorithm utilizes deeper data analysis, including students' past answering skills and the knowledge components of the questions, compared to traditional methods that only use the average accuracy rate for difficulty assessment. This comprehensive difficulty assessment method demonstrates higher accuracy in knowledge tracing tasks, significantly improving the prediction of students' correct answers. Both ACC and AUC metrics show that the Multi-DDiff algorithm outperforms traditional methods, highlighting the importance of finding a more refined balance between answering skills and question difficulty assessment.

[0085] The Test-Aware Attention-Based Convolutional Neural Network (TACNN) framework is a method for predicting the absolute difficulty of English reading comprehension questions. It focuses on modeling the grammatical and semantic aspects of the questions, such as sentence structure and vocabulary. While TACNN has unique advantages in understanding sentence structure and vocabulary, the Multi-DDiff algorithm, by dynamically and individually assessing question difficulty—considering students' historical answer records—more accurately reflects students' actual perception of difficulty for each question. This makes it more precise in predicting students' knowledge levels and better adapts to changes in student abilities and learning needs.

[0086] Table 5 This invention proposes a deep knowledge tracking model that combines natural language processing techniques and deep learning methods. By integrating deep semantic feature embedding (Sem-TextEG) of BERT-generated exercise text data and a multi-dimensional dynamic exercise difficulty algorithm (Multi-DDiff), the model can deeply understand the complex relationship between text data and student learning behavior. The design of a composite deep learning architecture further enhances the model's feature capture capability, enabling it to accurately capture and understand question content, student answer patterns, and the dependencies between them. Experimental results verify the effectiveness of the model and the performance advantages of integrating each component.

[0087] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for predicting the correctness of answers based on the NLKT knowledge tracing model, characterized in that, include: Obtain exercise text data and knowledge component characteristics; A knowledge tracking model based on natural language understanding is constructed. The knowledge tracking model based on natural language understanding includes a problem text understanding module, a problem difficulty coefficient processing module, and a deep knowledge tracking module. The problem text understanding module is constructed based on a pre-trained BERT model. The exercise text data is input into the knowledge tracking model based on natural language understanding. The exercise text understanding module extracts features from the exercise text data to obtain a comprehensive text vector. The exercise difficulty coefficient processing model calculates and generates a comprehensive difficulty coefficient based on knowledge component features. A comprehensive feature vector is generated based on the comprehensive text vector and the comprehensive difficulty coefficient. The deep knowledge tracking module performs deep knowledge tracking processing on the comprehensive feature vector to obtain the final predicted answer correctness result. The process of generating the overall difficulty coefficient of the problem includes: The overall difficulty level of the problem Generated based on a basic difficulty score and a weighted score that incorporates multiple knowledge components: in, Basic difficulty rating The difficulty-weighted scoring is applied to multiple knowledge components, where KC represents the corresponding knowledge component feature. Indicates performance adjustment items, The basic difficulty score is obtained based on the accuracy of the questions. The multi-knowledge-component difficulty weighted score is obtained by weighting and summing different knowledge-component features, including literal comprehension features. Characteristics of reasoning and understanding and characteristics of critical understanding , in These are the weights of literal comprehension features, inferential comprehension features, and critical comprehension features, respectively. The calculation method for performance adjustment items is as follows: in, and These represent the number of times a student succeeds and fails on a specific KC problem, respectively. It is a constant used when the denominator is not zero; The deep knowledge tracking module includes a CNN layer, a multi-head attention layer, a Bi-GRU layer, and a fully connected layer connected in sequence. The process involves using a CNN layer to extract features from the comprehensive feature vector, generating a fused feature vector. A multi-head attention layer then performs self-attention calculation and linear transformation on the fused feature vector to obtain attention output features. A Bi-GRU layer processes the attention output features at different time steps, and a fully connected layer converts the output of the Bi-GRU layer into the final predicted probability of the correct answer to the question.

2. The method according to claim 1, characterized in that, The process of feature extraction from exercise text data includes: The exercise text data is preprocessed and segmented, and includes paragraph information, question information, and option information. The paragraph information and question information are divided into paragraph sequences and question sequences, respectively. The paragraph sequences are then divided using a sliding window to generate several window segments, each of which contains a fixed number of tokens and has overlapping parts. For different window segments, the window segment, question sequence, and option information are concatenated to obtain several input sequences; The BERT model processes several input sequences to obtain several output vectors. The output vector is weighted using an attention mechanism, and a weighted sum is calculated based on the assigned weights to generate a comprehensive text vector.

3. The method according to claim 1, characterized in that, The weights of the literal comprehension features, reasoning comprehension features, and critical comprehension features were obtained through linear regression analysis.

4. The method according to claim 1, characterized in that, The process of generating the comprehensive feature vector includes: Obtain the user identifier and the exercise identifier, convert the user identifier and the exercise identifier into numerical indices through an embedding layer, and map the numerical indices into identifier feature vectors; The overall difficulty coefficient of the problem is normalized by using a fully connected layer to generate a high-dimensional difficulty vector. The comprehensive text vector, the identifier feature vector, and the high-dimensional difficulty vector are concatenated to generate an input sequence. The input sequence is then concatenated to generate a comprehensive feature vector.

5. The method according to claim 1, characterized in that, In the CNN layer, the comprehensive feature vector is processed by convolution, and the dimensionality of the comprehensive feature vector after convolution is reduced by max pooling. The dimensionality-reduced comprehensive feature vector is then fused and concatenated to generate a fused vector.

6. The method according to claim 1, characterized in that, The knowledge tracking model based on natural language understanding is optimized using a binary cross-entropy loss function: The binary cross-entropy loss function is: in, It is the number of training samples. It is the first The students in time step The actual answer result It represents the probability of the model predicting the correctness of the answer.

Images