Learning behavior and time window fused learning performance early prediction method
By employing a learning behavior prediction model with time window segmentation and self-cross-attention mechanism in the online education environment, the problem of low accuracy in predicting student performance caused by the complexity of learning behavior data is solved, and efficient prediction of early learning performance is achieved.
Patent Information
- Application Number
- CN202411338196.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-02-07
AI Technical Summary
In the online education environment, the high complexity and dynamism of student learning behavior data lead to low accuracy of student performance prediction models, making it difficult to detect students' tendency to fail early and affecting the effectiveness of teaching interventions.
Learning behavior data is divided into time windows, and the temporal connectivity relationships of similar and different types of learning behaviors are learned through self-attention and cross-attention mechanisms. The prediction accuracy of the model is optimized by combining the multivariate cross-entropy loss function.
The accuracy of early predictions of learning performance was improved. Validation results showed that the LBTT model performed well among multiple benchmark models, especially with a significant improvement in F1 score on the LSTM model.
Smart Images
Figure CN119398578A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of early prediction of learning performance, and in particular to an early prediction method of learning performance integrating learning behavior and time window. Background Art
[0002] In the online education environment, the interaction data between students and learning platforms presents the characteristics of high complexity, dynamics and nonlinear correlation. In order to effectively process these data and extract valuable features, the LBTT model proposed in this invention is mainly based on the following design ideas: First, the time dependence and time window changes of learning behaviors: divide the student learning behavior data according to specific time windows to capture the changing trends and laws of different learning behaviors. Second, the temporal connection relationship of similar learning behaviors: in different time windows, the same type of learning behaviors may have certain temporal correlations; by designing a network module based on the self-attention mechanism, the temporal connection relationship of similar learning behaviors in different time windows is learned. Third, the correlation between different types of behaviors: in the same time window, there may be some correlation between different types of learning behaviors; by designing a network module based on the cross-attention mechanism, the correlation between different types of behaviors in the same time window is fused and calculated.
[0003] Based on the above design ideas, the students' online learning behavior data is first cleaned, converted and standardized in the data preprocessing stage to form an input format suitable for deep learning. The model captures the time dependency and time window changes of learning behavior by setting different time windows, and further subdivides the data according to the behavior category to form a feature sequence. The core self-attention module and cross-attention module respectively learn the temporal correlation of similar behaviors in different time windows and the mutual influence between different behavior types, thereby extracting key features. These features are fused and input into the fully connected neural network, and finally achieve accurate prediction of students' academic performance. The design of this model not only takes into account the complex dynamics of learning behavior, but also pays attention to the deep connection between learning behaviors through the cross-attention mechanism, providing support for learning performance prediction based on learning behavior.
[0004] With the development of mobile Internet and digital technology, online education platforms break through the limitations of time and space and provide a ubiquitous learning environment. At the same time, high dropout rates and low success rates have become a problem that cannot be ignored. Predicting student grades can detect students' tendency to academic failure early, which helps teachers intervene in time. With the continuous development of online education, the generated data presents characteristics such as high dimension, dynamics and nonlinearity, which brings challenges to academic performance prediction modeling. The present invention proposes a student performance prediction model (LBTT) based on learning behavior, observes student learning behaviors in time windows, designs a network module to learn the temporal connection relationship of similar learning behaviors in different windows, and at the same time integrates and calculates the correlation characteristics between different types of behaviors, in order to improve the accuracy of early learning performance prediction. Summary of the invention
[0005] In view of this, the present invention provides an early prediction method for learning performance that integrates learning behavior and time window.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0007] An early prediction method for learning performance integrating learning behavior and time window comprises the following steps:
[0008] Step 1: Use time windows to construct non-overlapping patches for the learning behavior time series;
[0009] Step 2: Use type embedding to embed each learning behavior type sequence into a learning behavior type token;
[0010] Step 3: Apply self-attention to the entire temporal token of the same learning behavior;
[0011] Step 4: Integrate the information from different categories of learning behaviors by embedding the type token of the learning behavior, and then concatenate the output of the cross-attention layer with the time token obtained by embedding the time window of the learning behavior;
[0012] Step 5: Use the multivariate cross entropy loss function to evaluate the difference between the model's predicted scores and the actual scores.
[0013] Preferably, in step 1, given a student learning behavior time series x 1:L ={x1,x2,…,x L}∈R L×C and the corresponding learning behavior type z 1:C ={z1,z2,…,z C}∈R C×L , where L is the length of the learning behavior time series, x i for ith The value at the time point, C is the number of learning behavior types, z 1:C Indicates i th Learning behavior type sequence; the goal of the prediction model f is to learn the behavior time series x 1:L and the corresponding learning behavior type sequence z 1:C To predict the student's score y = {y1, y2}, it can be expressed as: y = f(x 1:L ,z 1:C ); Each input time series w i First, it is divided into non-overlapping time windows; the time window length is represented as P, and the division process will generate a series of Where N is the number of patches, Among them, LearningBehaviorPatchEmbed() uses trainable linear projection and position embedding to map each patch of length P to D dimensions; the time window of the learning behavior time series, that is, the patch is P lb ∈R N×D , containing N patch-scale time tokens of D dimensions.
[0014] Preferably, in step 2, type embedding is achieved through a simple linear mapping, which can be summarized as follows:
[0015] T lb,i =LearningBehaviorTypeEmbed(z (i) ),i∈{1,...,C}
[0016] Among them, the type of embedding layer LearningBehaviorEmbed: R C →R D is instantiated as a trainable linear projector; in this way, the embedded vector contains information about the entire sequence; T lb ={T lb,i},i∈{1,...,C} is a representation set representing multiple sequences of learning behavior types.
[0017] Preferably, in step 3, the embedding vector for learning the same learning behavior contains multiple sharding time tokens and type token in, Acts as a global token for the entire sequence, merging information from other time tokens; the specific process is:
[0018]
[0019] Among them, l∈{0,…,M-1} represents i th LBTT block; and are the time token of the learning behavior and the type token of the learning behavior, respectively. is the embedded P sp ,V sp ; [·,·] represents the combined calculation of learning time and learning behavior type sequence tokens along the sequence dimension; Self-Attn(·) represents multi-head self-attention on the time token and type token of the learning behavior; LN(·) represents layer normalization.
[0020] Preferably, in step 4, the conversion formula is:
[0021]
[0022] Among them, Cross-Attn() represents the multi-head cross attention between learning behavior categories, where the query It comes from i th The learning behavior type token of the LBTT block, key and value are the learning behavior type token T lb ; Cross-Attention Layer The output of will become the input of the next self-attention layer.
[0023] Preferably, in step 5, the multivariate cross entropy is defined as follows:
[0024]
[0025] Among them, y i is the output of the model th Value is the corresponding actual value; is the predicted value of the model; C is 2, i.e., pass and fail; i is th student.
[0026] Compared with the prior art, the present invention has achieved the following technical effects:
[0027] The present invention not only improves the accuracy of early learning performance prediction, but also proves the effectiveness of the present invention through experiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A framework diagram of an early prediction method for learning performance integrating learning behavior and time window of the present invention;
[0029] Figure 2The distribution diagram of the number of students in each grade of the OULAD dataset of the present invention for an early prediction method of learning performance integrating learning behavior and time window and the distribution diagram after merging grades in this article;
[0030] Figure 3 This is a graph showing the comparison of the prediction performance of students' performance in weeks 5 to 38 between the LBTT model of the present invention, which is an early prediction method for learning performance integrating learning behavior and time window, and five baseline models;
[0031] Figure 4 This is a graph of learning performance prediction results for different time windows of an early prediction method for learning performance that integrates learning behavior and time windows in the present invention, where P = [2, 4, 6, 8, 10, 12, 14, 16, 18, 20], P is the length of Patch; the prediction results are all the final scores of the students. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0033] like Figure 1 As shown, the present invention discloses an early prediction method of learning performance integrating learning behavior and time window, comprising the following steps:
[0034] Step 1: Use time windows to construct non-overlapping patches for the learning behavior time series;
[0035] Given a student's learning behavior time series x 1:L ={x1,x2,…,x L}∈R L×C and the corresponding learning behavior type z 1:C ={z1,z2,…,z C}∈R C×L , where L is the length of the learning behavior time series, x i for i th The value at the time point, C is the number of learning behavior types, z 1:C Indicates i th Learning behavior type sequence; the goal of the prediction model f is to learn the behavior time series x 1:L and the corresponding learning behavior type sequence z 1:C To predict the student's score y = {y1, y2}, it can be expressed as: y = f(x 1:L ,z 1:C); Each input time series w i First, it is divided into non-overlapping time windows; the time window length is represented as P, and the division process will generate a series of Where N is the number of patches, in,
[0036] LearningBehaviorPatchEmbed() maps each patch of length P to D dimensions using trainable linear projection and position embedding; learns the time window of the behavior time series, i.e., the patch is P lb ∈R N×D , containing N patch-scale time tokens of D dimensions;
[0037] Step 2: Use type embedding to embed each learning behavior type sequence into a learning behavior type token;
[0038] Type embedding is achieved through a simple linear mapping, which can be summarized as follows:
[0039] T lb,i =LearningBehaviorTypeEmbed(z (i) ),i∈{1,...,C}
[0040] Among them, the type of embedding layer LearningBehaviorEmbed: R C →R D is instantiated as a trainable linear projector; in this way, the embedded vector contains information about the entire sequence; T lb ={T lb,i},i∈{1,...,C} represents a set of representations of multiple learning behavior type sequences;
[0041] Step 3: Apply self-attention to the entire temporal token of the same learning behavior;
[0042] The embedding vectors for learning the same learning behavior contain multiple shard time tokens and type token in, Acts as a global token for the entire sequence, merging information from other time tokens; the specific process is:
[0043]
[0044] Among them, l∈{0,…,M-1} represents i th LBTT block; and are the time token of the learning behavior and the type token of the learning behavior, respectively. is the embedded P sp ,V sp ; [·,·] represents the combined calculation of learning time and learning behavior type sequence tokens along the sequence dimension; Self-Attn(·) represents multi-head self-attention on the time token and type token of the learning behavior; LN(·) represents layer normalization;
[0045] Step 4: Integrate the information from different categories of learning behaviors by embedding the type token of the learning behavior, and then concatenate the output of the cross-attention layer with the time token obtained by embedding the time window of the learning behavior;
[0046] The conversion formula is:
[0047]
[0048] Among them, Cross-Attn() represents the multi-head cross attention between learning behavior categories, where the query It comes from i th The learning behavior type token of the LBTT block, key and value are the learning behavior type token T lb ; Cross-Attention Layer The output of will become the input of the next self-attention layer;
[0049] Step 5: Use the multivariate cross entropy loss function to evaluate the difference between the model's predicted scores and the actual scores;
[0050] The multivariate cross entropy is defined as follows:
[0051]
[0052] Among them, y i is the output of the model th Value is the corresponding actual value; is the predicted value of the model; C is 2, i.e., pass and fail; i is th student.
[0053] Experiments and results:
[0054] 1. Dataset
[0055] The Open University Dataset (OULAD) records the operation behaviors of students in the online learning environment. It contains the demographic data set (DIS), registration information data set (RIS), classroom behavior data set (LPS) and student assessment data set (SAS) of 32,593 students in 7 different courses from 2013 to 2014. Among them, DIS records the basic information of students such as gender, age, home address, credits and final grades, as well as the final grades of the courses; RIS records the time information of students registering for courses; LPS records all the behaviors of students on the online learning platform in detail and divides them into categories; students' learning behaviors in the learning environment are divided into 20 categories. Finally, all the features of LPS and the final grade features in DIS are combined to construct the experimental data set. SAS is not within the selected data range.
[0056] After merging the classification labels of students' grades, the distribution of students in each grade is as follows: excellent (3024 people), passing (12360 people), failing (6706 people), and dropouts (7188 people).
[0057] like Figure 2 From the data distribution, it can be seen that the proportion of students distributed in each grade is not uniform. This imbalance may affect the establishment of the grade prediction model, thereby reducing the prediction accuracy of the model. The present invention combines students into two categories during analysis: pass and fail. Excellent and passing students are classified as pass categories, and students who fail and drop out are classified as fail categories.
[0058] OULAD includes information, assessments, quizzes, and course performance of students interacting with the VLE. Interactions with the VLE are further categorized into 20 different learning behaviors. For example, downloading or watching lectures, course content, or quizzes. The names of these behaviors are as follows: dataplus, dualpane, externalquiz, folder, forumng, glossary, homepage, htmlactivity, oucollaborate, oucontent, ouelluminate, ouwiki, page, questionnaire, quiz, repeatactivity, resource, sharedsubpage, subpage, url. The student behavior categories are shown in Table 1:
[0059] Table 1 Student behavior categories
[0060]
[0061] Aggregate the student learning behavior data by week. The present invention processes according to the data field, aggregates every 7 days as a week, and finally aggregates the total number of clicks of the statistical behavior type into 38 weeks. After preprocessing the data, the present invention finally obtains a time series data set containing 38 time steps, and the data set includes the click stream data of 29,278 students.
[0062] 2. Experimental setup:
[0063] The present invention uses Python language for data merging, cleaning, model training and reasoning. To prevent the model from overfitting, the present invention introduces an early stopping mechanism. The final parameter settings of each model are shown in Table 2:
[0064] Table 2 Experimental parameter settings
[0065]
[0066] 3. Experimental results:
[0067] Student performance prediction. The present invention verifies the effectiveness of the LBTT model through comparative experiments. The comparative experiments listed in Table 2 include the student performance prediction results of the LBTT prediction model for a study period of 5 to 38 weeks. Figure 3 As shown in the figure, compared with the baseline model, LBTT has higher Acc and F1 scores than the comparison model. Among them, the average Acc is improved by 3.30% and F1 is improved by 4.41% compared with other models at all learning time lengths. In particular, compared with the LSTM model, the average F1 score is improved by up to 10.73%, which is the highest improvement among all compared models. These improvements are due to LBTT's effective use of time windows to construct patches of different windows and fuse the correlation between different types of learning behaviors. LBTT's self-attention mechanism effectively combines the effective input of learning behaviors from temporal tokens of different time windows and type tokens of different types of behaviors, outputs the key tokens, and then identifies the type tokens of key learning behaviors through cross-attention. Compared with iTransformer, LBTT effectively integrates the temporal connection relationship information of learning behaviors in different time windows, and surpasses its prediction performance that only uses the correlation information between different types of behaviors.
[0068] Table 3. Complete results of the student performance prediction task
[0069]
[0070] Ablation experiment. The module ablation experiments shown in Table 3 were performed on the embedding layers of the same type of learning behavior in different time windows, namely the learning behavior time window and position information embedding layer, and the embedding layers of different types of behaviors, namely the learning behavior type embedding layer. The results of the ablation experiment show that the embedding operation designed by the LBTT model achieves the best performance. In comparison, the replacement embedding experiment shows that the instance norm in the original patching module may cause important distinguishing features to be ignored. Replacing the behavior type embedding layer may cause the model to ignore important correlations between different behavior types. In addition, the elimination experiment shows that for student performance prediction, both the temporal token of the learning behavior time window and the type token of the learning behavior type are indispensable.
[0071] Table 4. Ablation experiments. LP. and LT. are the abbreviations for the learned behavior time series and the learned behavior category series, respectively. P and T represent patch and category embeddings, respectively. PIN represents the complete patch embedding using instance norm. The results are the average of multiple experiments.
[0072]
[0073] Different lengths of time windows. This paper studies the effect of time window length, i.e., patch length, on prediction performance. The prediction time length is set to 38 weeks and the patch length P = [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]. The stride is set to be the same as the patch length, i.e., the time windows do not overlap with each other. Observation Figure 4 Experimental results show that the accuracy, F1, Recall and Precision scores do not change significantly with the change of Patch length, which shows the robustness of Patch hyperparameters. Therefore, the length of the time window selected for prediction depends on whether early prediction is to be performed.
[0074] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. An early prediction method for learning performance integrating learning behavior and time window, characterized in that: The following steps are involved: Step 1: Use time windows to construct non-overlapping patches for the learning behavior time series; Step 2: Use type embedding to embed each learning behavior type sequence into a learning behavior type token; Step 3: Apply self-attention to the entire temporal token of the same learning behavior; Step 4: Integrate the information from different categories of learning behaviors by embedding the type token of the learning behavior, and then concatenate the output of the cross-attention layer with the time token obtained by embedding the time window of the learning behavior; Step 5: Use the multivariate cross entropy loss function to evaluate the difference between the model's predicted scores and the actual scores.
2. The early prediction method of learning performance integrating learning behavior and time window according to claim 1, characterized in that: In step 1, given a student learning behavior time series x 1:L ={x1,x2,…,x L }∈R L×C and the corresponding learning behavior type z 1:C ={z1,z2,…,z C }∈R C×L , where L is the length of the learning behavior time series, x i for i th The value at the time point, C is the number of learning behavior types, z 1:C Indicates i th Learning behavior type sequence; the goal of the prediction model f is to learn the behavior time series x 1:L and the corresponding learning behavior type sequence z 1:C To predict the student's score y = {y1, y2}, it can be expressed as: y = f(x 1:L ,z 1:C ); Each input time series w i First, it is divided into non-overlapping time windows; the time window length is represented as P, and the division process will generate a series of Where N is the number of patches, in, LearningBehaviorPatchEmbed() maps each patch of length P to D dimensions using trainable linear projection and position embedding; learns the time window of the behavior time series, i.e., the patch is P lb ∈R N×D , containing N patch-scale time tokens of D dimensions.
3. The early prediction method of learning performance integrating learning behavior and time window according to claim 1, characterized in that: In step 2, type embedding is achieved through a simple linear mapping, which can be summarized as follows: T lb,i =LearningBehaviorTypeEmbed(z (i) ),i∈{1,...,C} Among them, the type of embedding layer LearningBehaviorEmbed: R C →R D is instantiated as a trainable linear projector; in this way, the embedded vector contains information about the entire sequence; T lb ={T lb,i },i∈{1,...,C} is a representation set representing multiple sequences of learning behavior types.
4. The early prediction method of learning performance integrating learning behavior and time window according to claim 1, characterized in that: In step 3, learning the embedding vector of the same learning behavior includes multiple sharding times and type in, Acts as a global token for the entire sequence, merging information from other time tokens; the specific process is: Among them, l∈{0,…,M-1} represents i th LBTT block; and are the time token of the learning behavior and the type token of the learning behavior, respectively. is the embedded P sp ,V sp ; [·,·] represents the combined calculation of learning time and learning behavior type sequence tokens along the sequence dimension; Self-Attn(·) represents multi-head self-attention on the time token and type token of the learning behavior; LN(·) represents layer normalization.
5. The early prediction method of learning performance integrating learning behavior and time window according to claim 1, characterized in that: In step 4, the conversion formula is: Among them, Cross-Attn() represents the multi-head cross attention between learning behavior categories, where the query It comes from i th The learning behavior type token of the LBTT block, key and value are the learning behavior type token T lb ; Cross-Attention Layer The output of will become the input of the next self-attention layer.
6. The early prediction method of learning performance integrating learning behavior and time window according to claim 1, characterized in that: In step 5, the multivariate cross entropy is defined as follows: Among them, y i is the model output i th Value is the corresponding actual value; is the predicted value of the model; C is 2, i.e., pass and fail; i is th student.