Student behavior-based student meta-cognitive competence evaluation model construction method

Abstract

The invention relates to a student behavior-based student meta-cognitive competence evaluation model construction method, and belongs to the technical field of artificial intelligence. The method comprises a preparation stage, a data collection and preprocessing stage, a training stage and a reasoning stage; in the preparation stage, an online learning platform is used for preparing various data of student learning behaviors in the course teaching process, and the student learning behaviors comprise operation type learning behaviors and text type learning behaviors. The operation type learning behaviors comprise starting or pausing video playing of a certain course, video fast forward or fast backward, full-screen or full-screen quitting, volume adjusting, online experiment completing and the like generated in the online learning process of the student. The text learning behaviors comprise WeChat group interaction, teaching network interaction, course statement completion, test paper evaluation, semester course summarization and the like; according to the invention, a meta-cognitive competence evaluation model based on student learning behaviors is designed and realized, and the method has the advantages of high automation degree, no invasion and good evaluation timeliness.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a method for constructing a student metacognitive ability evaluation model based on student behavior. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, teaching models and evaluation methods in the field of education are also constantly innovating. The traditional education evaluation system mainly focuses on the evaluation of students' knowledge and skills, and often ignores students' cognitive reflection and self-regulation ability in the learning process. Metacognitive ability, that is, the ability of individuals to understand and control their own cognitive processes, is considered to be one of the important factors affecting students' learning outcomes. Metacognitive ability includes how students monitor and regulate their own learning process, and how to evaluate their learning outcomes. Objective and accurate evaluation of students' metacognitive ability can broaden the dimension of student evaluation, help teachers provide more personalized and precise guidance, and thus enhance students' autonomous learning ability.

[0003] However, although the existing metacognitive ability evaluation methods, such as psychological scales and behavioral tasks, are easy to operate, they still have some shortcomings, resulting in the evaluation results being unable to truly reflect the students' actual metacognitive level: first, the evaluation process will cause intrusion to the person being evaluated and affect the effectiveness of their answers; second, the subjective psychological and behavioral factors of the person being evaluated during the evaluation process will also have an impact on the evaluation process and evaluation results; third, the existing metacognitive ability evaluation methods cannot achieve real-time evaluation of metacognitive ability; at the same time, the question settings of the existing metacognitive ability evaluation methods are relatively fixed, and it is impossible to conduct targeted evaluation in combination with specific subjects or specific courses.

[0004] Therefore, it is necessary to provide a method for constructing a student metacognitive ability evaluation model based on student behavior that is highly accurate, non-intrusive, timely, and targeted. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a method for constructing a student metacognitive ability evaluation model based on student behavior that is highly accurate, non-intrusive, timely and targeted.

[0006] The technical solution adopted by the present invention is: a method for constructing a student metacognitive ability evaluation model based on student behavior, the method comprising a preparation stage, a data collection and preprocessing stage, a training stage and a reasoning stage;

[0007] The preparation phase includes the following steps:

[0008] Use online learning platforms to prepare various data on students’ learning behaviors during course teaching;

[0009] The student learning behaviors include operational learning behaviors and text-based learning behaviors. The operational learning behaviors include starting or pausing a course video, fast forwarding or rewinding a video, full screen or exiting full screen, adjusting the volume, completing online experiments, etc. generated during the student's online learning process. The text-based learning behaviors include WeChat group interaction, teaching network interaction, completing course summaries, evaluating test papers, and semester course summaries.

[0010] Prepare operational learning behavior data. The tables used to store data in the online learning platform include the learning behavior table behavior. The learner's student ID in the learning behavior table is the unique primary key and has been hidden for desensitization processing;

[0011] Prepare text-based learning behavior data and summarize all text-based learning behavior data of each student during the semester;

[0012] The data collection and preprocessing phase includes the following steps:

[0013] Use Python regular expressions to remove non-text noise data such as special characters and emoticons in text-based learning behavior data. In addition, use the HIT stop word list to filter stop words, which refer to words with no actual meaning in the word segmentation results.

[0014] The sentence-partitioned corpus is segmented using the precise Jiaba segmentation model, with words separated by spaces.

[0015] Represent the text in a form that can be processed by computers, and use the Skip-gram model in Word2Vec for word embedding to preserve the associated information of text data;

[0016] After embedding, the operational behavior data is directly input into the evaluation model as standard data; due to the complexity of language, the text learning behavior data cannot retain all language information during the embedding process. The information quality evaluation system is designed in combination with the course content, and the text learning behavior data is introduced into the information quality evaluation system;

[0017] Introduce Bloom's cognitive hierarchy evaluation system into text-based learning behavior data, further explore the implicit information of text-based learning behavior data, use Bloom's cognitive hierarchy evaluation results as additional input to the model, and manually annotate the Bloom's cognitive hierarchy corresponding to text-based learning behavior data based on the word identification library;

[0018] Introduce text learning behavior data accumulated by online learning platforms in previous semesters, remove unusable data such as short and meaningless data, and finally obtain usable text learning behavior data. Mark the cognitive level of the preprocessed data, and mark them as six integers from 0 to 5 from low to high. Divide the data into 6 categories according to the marking results, and divide each level into 4:1 ratio and merge them into training set A and test set A;

[0019] Use training set A to train Bloom's cognitive hierarchy evaluation model, and use test set A to evaluate the model effect;

[0020] Select some questions from the MAI metacognitive awareness scale and improve them in combination with the course content, and finally form a metacognitive scale questionnaire, which includes primary and secondary dimensions. The metacognitive scale questionnaire is distributed at two time points, before the beginning of the semester and after the end of the semester, to evaluate students' metacognitive ability twice, and the reliability and validity of the metacognitive scale questionnaire is analyzed using statistical software to verify its evaluation effect;

[0021] The two evaluation results were statistically processed and the scale data were processed. The metacognitive ability evaluation results of all students on the secondary dimension of the metacognitive scale questionnaire were arranged from small to large. According to the three equal points, the metacognitive ability scores of students on this dimension were matched to the three evaluation results of "weak ability", "average ability" and "strong ability".

[0022] The training stage includes an input layer, an embedding layer, a feature extraction layer, a feature fusion layer, an adjustment layer and an output layer;

[0023] Input layer: The evaluation results of the metacognitive scale questionnaire before the beginning of the semester are used as the data set for training. According to the evaluation results, students of each level are divided into a ratio of 4:1 and mixed into training set B and test set B;

[0024] The embedding layer includes the following steps:

[0025] The sequence of a student's operational learning behavior is denoted as X = {x1, x2, x3, ..., x n}, use the Item2Vec method to convert each row into x i Embedded as the corresponding d-dimensional behavior vector v xi ∈R d , each student's operation learning behavior sequence X = {x1,x2,x3,...,x n} can be transformed into an operation-like behavior sequence vector v i ∈R n×d , where x iIt represents a learning behavior of operation-type learning behavior data, such as a single operation such as start watching, and X represents an operation sequence, such as a sequence consisting of multiple actions such as start watching, pause, and continue watching;

[0026] The text learning behavior data of a student is input into the Bloom's cognitive hierarchy evaluation system to evaluate the information quality, and four characteristics of Bloom's cognitive hierarchy, information volume, relevance, and emotional tendency are obtained. The data vector V is obtained by integrating it with the operational learning behavior sequence. i ∈R (n+4)×d ;

[0027] Segment the text-based learning behavior data, use the Word2Vec method on the segmentation results, and convert the n-word text into a d-dimensional word vector, denoted as W i ∈R n×d ;

[0028] The feature extraction layer includes the following steps:

[0029] Use a combination of self-attention mechanism and bidirectional gated recurrent unit network to achieve effective feature extraction;

[0030] The vector V obtained by the embedding layer i ∈R (n+4)×d and W i ∈R n×d Further input into the self-attention mechanism, the operation class learning behavior sequence vector V i The output in the self-attention mechanism is denoted as M i , the text class learning behavior sequence vector W i The output in the self-attention mechanism is denoted by N i , the calculation formulas of operation learning behavior sequence and text learning behavior sequence are as follows:

[0031] M i =Attention(V i W Q ,V i W K ,V i W V )

[0032] N i =Attention(W i W Q ,W i W K ,W i W V )

[0033] Among them, Attention is the self-attention mechanism function, W Q ,WK ,W V are learnable weight matrices used to generate queries, keys, and values, respectively;

[0034] N i Further input into the bidirectional gated recurrent unit network, the calculation is shown as follows:

[0035]

[0036]

[0037] Among them, GRU is a variant of recurrent neural network, which is used to process sequence data;

[0038] By connecting a convolutional neural network after the bidirectional gated recurrent unit network, the powerful characteristics of convolution are used to more effectively capture and extract the local semantic features of the text for the training of Bloom's cognitive hierarchy evaluation module.

[0039] The feature fusion layer includes the following steps:

[0040] The corresponding features of the three channels of operation learning behavior data channel, text learning behavior data channel and information quality features combined with course content are fused into three-channel features;

[0041] Perform dual-channel feature fusion on the operation behavior data channel and the text behavior data channel;

[0042] The adjustment layer includes the following steps:

[0043] The results of the metacognitive ability evaluation of students before the beginning of the semester and the changes in the scores of the metacognitive ability evaluation of students before the beginning of the semester and after the end of the semester were used as data sets for training to reflect the changes in students' metacognitive ability.

[0044] The target loss function MAE is used to adjust the evaluation model. The target loss function MAE formula is as follows:

[0045]

[0046] Among them, y i It is expressed as the change value of the real metacognitive ability evaluation score, Represents the node output value;

[0047] After training the adjustment node, all behavior data are re-input and the adjustment node output results are embedded into k dimensions;

[0048] The output layer includes the following steps:

[0049] The softmax function is used to calculate the probability of the classification results of students' metacognitive ability. The calculation formula is as follows:

[0050] P = softmax(W c Z+b c )

[0051] Among them, Z∈R k Represents the high-dimensional features of the output data of the adjustment layer, W c Represented as the softmax model weight matrix, b c Represented as a softmax model bias vector;

[0052] The target loss function Loss is used to measure the difference between the model prediction and the true label. The target loss function Loss formula is as follows:

[0053] Loss=-∑ s p si logp si

[0054] Where p si represents the probability that the evaluation result of student s belongs to category i;

[0055] The metacognitive ability evaluation model includes a basic training module and an adjustment training module;

[0056] The basic training module, adjustment training module and overall model algorithm steps are as follows:

[0057] Input: student operation learning behavior sequence matrix v, text learning behavior information quality evaluation result matrix c and text learning behavior matrix w, set the training batch to epoch

[0058] Output: Classification probability p

[0059] #Train Bloom's cognitive hierarchy evaluation module

[0060] 1.foreinrange(epoch):

[0061] 2.a W ,N=SelfAttention(W) / / Self-attention mechanism obtains weights and output matrix

[0062] 3.G=BiGRU(N) / / Feature extraction to obtain hidden layer feature vector

[0063] 4.p1=Convolution(G) / / Convolution to get Bloom's cognitive hierarchy classification probability

[0064] 5.Loss_Hierarchy=Loss_CE(p1,true_label_Hierarchy) / / Cross entropy loss function

[0065] 6.Update_parameters(Loss_Hierarchy) / / Gradient descent method to update parameters

[0066] #Training basic output module

[0067] 7.foreinrange(epoch):

[0068] 8.a V ,M=SelfAttention(V) / / Self-attention mechanism obtains weights and output matrix

[0069] 9.a W ,N=SelfAttention(W)

[0070] 10.U,G=BiGRU(M),BiGRU(N) / / Feature extraction to obtain hidden layer feature vector

[0071] 11.Z = Concatenate_and_Fc(U,G) / / vector linking and full connection

[0072] 12.p=Softmax(Z) / / After the Softmax function, the classification probability p is obtained

[0073] 13.Loss_Basic = Loss_CE(p,true_label_basic) / / Cross entropy loss function

[0074] 14.Update_parameters(Loss_Basic) / / Gradient descent method to update parameters

[0075] #Training and adjusting output module

[0076] 15.foreinrange(epoch):

[0077] 16. Repeat the training steps from line 8 to line 14

[0078] 17.c=Fully_connected(Z) / / The output value c of the adjustment module is obtained through the fully connected layer

[0079] 18.Loss_Adjust=Loss_MAE(c,true_value_adjust) / / MAE loss function

[0080] 19.Update_parameters(Loss_Adjust) / / Gradient descent method to update parameters

[0081] #Train the overall model

[0082] 20.p_final=Fully_connected(p,c) / / After basic output p and adjusted output c, access the fully connected layer

[0083] 21.Loss_Final = Loss_CE(p_final, true_label) / / Cross entropy loss function

[0084] 22.Update_parameters(Loss_Final) / / Gradient descent method to update parameters

[0085] The evaluation results of the metacognition scale questionnaire before the beginning of the semester were used as the data set. According to the evaluation results, the students of each level were divided into a ratio of 4:1 and mixed into training set B and test set B respectively;

[0086] Adjust the output module training. This module uses the metacognitive ability changes obtained from two metacognitive scale questionnaire evaluations as the data set for training. The normalized data of all students' metacognitive ability changes are randomly divided into a training set C and a test set C with a ratio of 4:1. The mean absolute error (MAE) is used to evaluate the training results.

[0087] The reasoning stage is based on the output of the basic module training results and the adjustment module training results after the fully connected layer. Since the basic output module and the adjustment output module in the adjustment layer of the previous model respectively fit the relationship between learning behavior and metacognitive ability, and the relationship between learning behavior and changes in metacognitive ability, the model already has implicit features corresponding to the above two types of information. During the second metacognitive scale questionnaire evaluation, after students have experienced one semester of course study, their metacognitive ability at this moment is based on their inherent metacognitive ability and the changes in metacognitive ability in this semester. Therefore, the evaluation result is used as the data set corresponding to the final output of the model.

[0088] Preferably, the online learning platform includes but is not limited to the Moodle online learning platform used by the course, the Touge practical teaching platform, the school teaching network, WeChat, Ketangpai and other channels.

[0089] Preferably, the information quality evaluation system includes three indicators: information quantity, relevance and emotional tendency:

[0090] The information index is used to extract the features of students' course summary documents. First, students' course summary documents are obtained. The number of topics is set to K, and the parameters α = 50 / K, β = 0.01. Each topic contains different features. All topic clustering results can fully reflect the students' summary information from the perspectives of knowledge points, practices, goals, and gains in the entire course cycle. By setting different K values, a better balance between feature probability and topic classification can be achieved.

[0091] The relevance index is used to evaluate the information quality of the extended thinking texts in the teaching network courses of students. Teachers publish extended thinking tasks in the teaching network and calculate the relevance in combination with the corresponding chapters;

[0092] The sentiment tendency index is used to evaluate the information quality of student course evaluation texts. The sentiment tendency of the test paper evaluation text is determined by the number of sentiment words in the comment text. The Hownet sentiment dictionary is used to match the course evaluation data, and the matching text is subjected to sentiment analysis to determine the sentiment tendency of the text and improve the sentiment recognition ability of the model.

[0093] Preferably, the metacognition scale questionnaire includes several questions, each of which is divided into five options: "very inconsistent", "inconsistent", "average", "consistent" and "very consistent", and each option is set with a corresponding score from low to high, among which the "very inconsistent" option is scored 1 point, and the "very consistent" option is scored 5 points.

[0094] Preferably, the self-attention mechanism in the feature extraction layer is used to adjust the model so that it focuses on key parts when processing information, while reducing the attention to irrelevant information, thereby improving efficiency and accuracy. Its essence is the mapping between query and key-value pairs, which is applied between the element Query of Target and all elements in Source. The operation steps are as follows:

[0095] Obtain the corresponding attention weight through Query: Calculate the similarity between the target element Q and each Key. The calculation formula is as follows:

[0096] f(Q,K)=QK T

[0097] Among them, Q represents Query, K represents Key, and T represents matrix transposition;

[0098] Weight normalization: Use the Softmax function to normalize the weights, and the normalized weights are recorded as a i , the calculation formula is as follows:

[0099]

[0100] Among them, Q represents Query, K represents Key, i represents the index currently calculating the weight, and j represents the index traversing all weights;

[0101] Weighted sum: The normalized weight a i The weighted sum of the corresponding key value is the attention, and the calculation formula is as follows:

[0102]

[0103] Among them, Q represents Query, K represents Key, V represents Value, and a i represents the normalized weight;

[0104] The self-attention mechanism focuses all attention on the input sequence itself. This mechanism is suitable for processing input sequences of variable length, and can establish global dependencies and expand the receptive field. The calculation formula is as follows:

[0105]

[0106] Among them, Q represents Query, K represents Key, V represents Value, and X∈R d represents a d-dimensional vector. X can represent the original source or basic data of any of Q, K, and V. represents the adjustment factor to avoid X·X T The inner product value is too large, and the softmax value is only 0 or 1. k The value is the key vector dimension.

[0107] Preferably, the bidirectional gated recurrent unit obtains sequence input information from both the sequential and reverse directions, and links the bidirectional hidden layers, which can better focus on the timing or context information. The calculation formula is as follows:

[0108] r t =σ(W r x t +U r h t-1 )

[0109] z t =σ(W z x t +U z h t-1 )

[0110]

[0111] The current time point is represented as t, where σ is the Sigmoid function, x t Input for the current time, ht-1 is the hidden state of the previous memory unit, r t To reset the gate, we selectively forget the old memory and ignore the hidden state of the previous moment according to the current input and historical context information. When the r value is close to 1, the model is more inclined to consider the previous hidden state, and when the r value is close to 0, the model is more inclined to ignore the previous hidden state. t To update the gate, represents the candidate hidden information to be updated, h t It represents the final output information of the hidden layer obtained after the reset gate and the update gate control. The matrix W and matrix U are the continuously updated weight matrices in the model.

[0112] The input information up to the current time point t is a vector X = {x1, x2, x3, ..., x t}, X represents a piece of text data, where x1 to x t Represents all words after word segmentation. The input of the gated recurrent unit includes the hidden layer output h of the previous moment. t-1 and the input x at this moment t , the output is the hidden state h at the current moment t , through the update gate z in the unit t With reset gate r t The setting controls the hidden state h t The information is updated and the features that are more important to the target are finally retained.

[0113] Preferably, the convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer. The input layer is responsible for receiving the original image data and passing the original image data to the convolutional layer. The convolutional layer is responsible for extracting features and transmitting them to the pooling layer. The pooling layer samples and screens the feature map output by the convolutional layer. After being processed by multiple convolutional layers and pooling layers, the dimension of the data has been greatly reduced. At this time, the output of the pooling layer is flattened into a one-dimensional vector and passed to the fully connected layer for processing. The fully connected layer is responsible for mapping the features extracted by the previous layer to the label space of the sample. The output layer is used to perform probability output for classification tasks or numerical prediction for regression tasks.

[0114] Select the text processing task as N = max{n}, where {n} is the set of all single sentence lengths of text i, embed all sentences into n-dimensional vectors, and sequentially link them to obtain the text matrix x i ∈R N×d , let the convolution kernel w∈R h×d , for x i To extract local features, the convolution process calculation formula is as follows:

[0115] s ji ∈f(W jX i:i+h-1 +b)

[0116] Among them, s ji Representation word W i The local features obtained after convolution, h represents the window width, W j ∈R h×k is the jth convolution kernel weight, b∈R is the bias, and the convolution result is the sentence feature map A feature map is a collection of local features extracted by convolution operations, reflecting a certain pattern or feature of the input data;

[0117] The feature information sampling formula is as follows:

[0118]

[0119] in, is the convolution feature, s j is the sentence feature graph;

[0120] Using k convolution kernels w can form k convolution features All Concatenate to form the convolutional feature vector Input S into the fully connected layer, and calculate the output layer result y by the following formula:

[0121] y=Softmax(W·S+b)

[0122] Among them, W is the weight matrix of the fully connected layer, S is the convolution feature vector, b is the bias, and y is the output layer result.

[0123] Beneficial effects of the present invention:

[0124] 1. The present invention constructs a metacognitive ability evaluation model based on students' learning behavior, realizes the automation and non-intrusiveness of the metacognitive ability evaluation process, and improves the timeliness of the evaluation results;

[0125] 2. The present invention combines pedagogy and psychology theories to introduce Bloom's cognitive hierarchy evaluation system, and uses the correlation between Bloom's cognitive hierarchy theory and metacognitive theory to strengthen the theoretical support of the model;

[0126] 3. The present invention designs an information quality evaluation system in combination with the course content, increases the amount of model input information, fits the actual evaluation scenario, and effectively improves the model evaluation effect;

[0127] 4. The present invention adopts a basic output module and an adjustment output module to capture the implicit features related to the basic metacognitive ability and the changes in metacognitive ability for the same learning behavior data input, thereby effectively improving the model evaluation effect.

[0128] In summary, the present invention designs and implements a metacognitive ability evaluation model based on students' learning behavior. The model has the advantages of high degree of automation, non-intrusiveness and good evaluation timeliness. By introducing Bloom's cognitive hierarchy evaluation system and information quality evaluation system, it mines the implicit information of text-based learning behavior data and increases the amount of input information, which effectively improves the evaluation effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 It is a schematic diagram of the architecture of the knowledge graph of the overall solution of the present invention.

[0130] Figure 2 It is a structural schematic diagram of the overall solution of the present invention.

[0131] Figure 3 It is a structural schematic diagram of the self-attention mechanism of the present invention.

[0132] Figure 4 It is a schematic diagram of the structure of the gated cycle unit of the present invention.

[0133] Figure 5 It is a schematic diagram of the structure of the convolutional neural network of the present invention.

[0134] In the figure: 1. Input layer A 2. Convolutional layer 3. Pooling layer 4. Fully connected layer 5. Output layer A. DETAILED DESCRIPTION

[0135] The following are specific embodiments of the present invention, and the technical solutions of the present invention are further described in conjunction with the accompanying drawings, but the present invention is not limited to these embodiments.

[0136] Embodiment 1:

[0137] like Figure 1-5 As shown, a method for constructing a student metacognitive ability evaluation model based on student behavior includes a preparation stage, a data collection and preprocessing stage, a training stage, and a reasoning stage;

[0138] The preparation phase includes the following steps:

[0139] S11: Using online learning platforms to prepare various data on students' learning behaviors during the course teaching process, the online learning platforms include but are not limited to the Moodle online learning platform used in the course, the Touge practical teaching platform, the school teaching network, WeChat, Ketangpai and other channels;

[0140] S12: The student learning behaviors include operational learning behaviors and text-based learning behaviors. The operational learning behaviors include starting or pausing a course video, fast forwarding or rewinding a video, full screen or exiting full screen, adjusting the volume, completing an online experiment, etc. generated during the student's online learning process. The text-based learning behaviors include WeChat group interaction, teaching network interaction, completing a course summary, evaluating a test paper, and a semester course summary.

[0141] S13: Prepare operational learning behavior data. The table used to store data in the online learning platform includes the learning behavior table behavior. The learner's student ID in the learning behavior table is the unique primary key and has been hidden for desensitization. The corresponding relationship between the behavior value in the table and the learning behavior of the Moodle platform and the Touge platform is shown in Table 1:

[0142] Table 1 Corresponding identifiers of operational learning behaviors

[0143]

[0144]

[0145] Table 2 shows the records of some of the student's Moodle online operation platform operation learning behaviors:

[0146]

[0147] S14: Prepare text-based learning behavior data, and summarize all text-based learning behavior data of each student during the semester, including but not limited to WeChat group text content, teaching network text content, evaluation test paper text content, and semester summary text;

[0148] The data collection and preprocessing phase includes the following steps:

[0149] S21: Use Python regular expressions to remove special characters, emoticons and other non-text noise data in text-based learning behavior data. In addition, use the HIT stop word list to filter stop words. The stop words refer to non-actual meaning words in the word segmentation results, such as modal particles, high-frequency pronouns, etc.;

[0150] S22: Segment the sentence-partitioned corpus using the precise Jiaba segmentation model, with words separated by spaces;

[0151] S23: Represent the text in a form that can be processed by computers, and use the Skip-gram model in Word2Vec to embed words to preserve the associated information of text data;

[0152] S24: Since the operation behavior data completely records the operation behavior sequence of students, the operation behavior data can be directly input into the evaluation model as standard data after being embedded; due to the complexity of language, the text learning behavior data cannot retain all language information during the embedding process. In order to make the evaluation model closer to the course corresponding to this article and enhance the interpretability of the evaluation process, an information quality evaluation system is designed in combination with the course content, and the text learning behavior data is introduced into the information quality evaluation system;

[0153] The information quality evaluation system includes three indicators: information quantity, relevance and emotional tendency:

[0154] S241: The information quantity index is used to extract the features of the course summary documents of students. First, the course summary documents of 72 students are obtained. The number of topics is set to K, the parameters α = 50 / K, β = 0.01, and each topic contains different features. The clustering results of all topics can fully reflect the summary information of students from the perspectives of knowledge points, practices, goals, and gains in the entire course cycle. By setting different K values, a good balance between feature probability and topic classification is achieved. The probability distribution of each topic is shown in Table 3:

[0155] Table 3 Topic model clustering results

[0156]

[0157]

[0158] It can be seen from the topic clustering model that topic 1 is related to the students' summary of their gains during the semester, topic 2 is related to the course concept standards, topic 3 is related to the knowledge points of software quality, reliability and software testing in the course, topic 4 is related to teamwork, topic 5 is related to the object-oriented characteristics of software engineering courses, topic 6 is related to code implementation, topic 7 is related to goal setting and knowledge point summary, topic 8 is related to software maintenance and problem solving, topic 9 is related to system implementation and course presentation, and topic 10 is related to software design and user experience. The topic clustering results fully reflect the students' summary information from the perspectives of knowledge points, practices, goals, gains, etc. during the entire course cycle. When K is set to 10, a good balance between feature probability and topic classification can be achieved.

[0159] S242: The relevance index is used to evaluate the information quality of the extended thinking text of the student teaching network course. The teacher publishes the extended thinking task in the teaching network and calculates the relevance in combination with the corresponding chapters;

[0160] This embodiment is based on the "Software Engineering" course. Based on this course, a software engineering course dictionary as shown in Table 4 is constructed. The knowledge points and corresponding concepts of each chapter are summarized and integrated into chapter documents. The interactive text content of each chapter of the students and the completed "thinking tasks assigned by the teacher" in text form are combined with the corresponding chapters to calculate the relevance;

[0161] Table 4 Software Engineering Course Dictionary

[0162]

[0163]

[0164] S243: The sentiment tendency index is used to evaluate the information quality of student course evaluation texts. The sentiment tendency of the test paper evaluation text is determined by the number of sentiment words in the comment text. The Hownet sentiment dictionary is used to match the course evaluation data, and sentiment analysis is performed on the matched text to determine the sentiment tendency of the text and improve the sentiment recognition ability of the model.

[0165] For example, a student's test paper evaluation text is: "I highly approve of this exam. The questions tested everyone's learning situation very well and also allowed me to discover my own shortcomings." Among them, "very", "approved", and "very good" are positive sentiment words, and "inadequate" is a negative sentiment word. The total number of sentiment words is 4. After word segmentation and removal of stop words, the total number of words in the text is 16, so its sentiment tendency score is 0.25.

[0166] S25: Introduce Bloom's cognitive hierarchy evaluation system to text-based learning behavior data, further explore the implicit information of text-based learning behavior data, and use Bloom's cognitive hierarchy evaluation results as additional input to the model. The Bloom's cognitive hierarchy corresponding to the text-based learning behavior data can be manually annotated based on the word identification library. The Bloom's cognitive hierarchy word identification library is shown in Table 5:

[0167] Table 5 Bloom's cognitive hierarchy word library

[0168]

[0169] S26: In order to increase the amount of training data, the text learning behavior data accumulated by the online learning platform in the past semesters were introduced, and the unusable data such as too short and meaningless data were removed. Finally, a total of 7248 usable text learning behavior data were obtained. The pre-processed data were labeled with cognitive levels, and six integers from 0 to 5 were marked from low to high. The data were divided into 6 categories according to the labeled results. Each level was divided into training set A and test set A in a ratio of 4:1. The labeling results are shown in Table 6:

[0170] Table 6 Statistics of Bloom's cognitive hierarchy labeling results

[0171]

[0172] S27: Use training set A to train Bloom's cognitive hierarchy evaluation model, and use test set A to evaluate the model effect. The model evaluation results are shown in Table 7:

[0173] Table 7 Bloom's cognitive hierarchy evaluation module performance

[0174]

[0175] The F1 value is an indicator used in machine learning and statistics to evaluate model performance, especially in classification tasks. It is the harmonic mean of precision and recall. The value of F1 score is between 0 and 1, where 1 indicates perfect prediction performance of the model. The calculation formula of F1 value is as follows:

[0176] FI value = 2*precision*recall / (precision+recall)

[0177] As can be seen from Table 7, Bloom's cognitive hierarchy evaluation module has achieved good evaluation results at all cognitive levels in training set A, which provides an explanatory basis for the integration of the results obtained by inputting the original input data of the overall metacognitive ability evaluation model into this module and the original data as the model input.

[0178] S28: Use a psychological scale to evaluate students' metacognitive ability, select some questions from the MAI metacognitive awareness scale and improve them in combination with the course content, and finally form a metacognitive scale questionnaire, which includes a primary dimension and a secondary dimension. The metacognitive scale questionnaire is distributed at two time points before the beginning of the semester and after the end of the semester, and the students' metacognitive ability is evaluated twice. The reliability and validity of the metacognitive scale questionnaire are analyzed using statistical software to verify its evaluation effect. In this embodiment, the metacognitive scale questionnaire contains 48 questions, each of which is divided into five options: "very inconsistent", "inconsistent", "average", "compliant" and "very consistent", and each option is set with a corresponding score from low to high, among which the "very inconsistent" option is scored 1 point and the "very consistent" option is scored 5 points. The questionnaire item settings are shown in Table 8:

[0179] Table 8 Metacognitive Ability Evaluation Scale Questionnaire

[0180]

[0181]

[0182] This embodiment is based on the second bachelor's degree course "Software Engineering" of a certain university in the spring semester of the 2022-2023 academic year. The metacognitive scale is distributed at two time points before the beginning of the semester and after the end of the semester to count the metacognitive ability level of each student at these two time points. There are 94 students in this course, all of whom participated in the first (beginning of the semester) and second (end of the semester) metacognitive ability evaluation. In the two questionnaire collection results, there are situations such as too long questionnaire filling time, wrong student number filling, and the same option selected for all questions in the questionnaire. If one of the two questionnaires filled out by the same student has the above situation, it is judged that the data is unavailable. It is concluded that there are 72 students whose metacognitive questionnaires are valid for both times. The evaluation scores of each secondary dimension of metacognition for some students in the first time are shown in Table 9:

[0183] Table 9 Evaluation scores of the secondary dimensions of some students' first metacognition

[0184]

[0185] Comparing the results of the second metacognition scale questionnaire evaluation with the results of the first evaluation, the changes in the metacognition secondary dimension abilities of some students are shown in Table 10:

[0186] Table 10 Changes in the secondary dimensions of metacognition of some students

[0187]

[0188] The results of both evaluations are summarized as shown in Table 11:

[0189] Table 11 Summary of the evaluation results of the two metacognition scale questionnaires

[0190]

[0191] S29: All the evaluation results of the two times are statistically processed and the scale data is processed. The metacognitive ability evaluation results of all students on the secondary dimension in the metacognitive scale questionnaire are arranged from small to large. According to the three-point division point, the metacognitive ability scores of students on this dimension are corresponded to the three evaluation results of "weak ability", "average ability" and "strong ability". The evaluation results of each metacognitive secondary dimension in the two evaluations are divided respectively. The division points are shown in Table 12:

[0192] Table 12 The division points of each secondary dimension in the evaluation results of metacognitive ability

[0193]

[0194]

[0195] The training stage includes an input layer, an embedding layer, a feature extraction layer, a feature fusion layer, an adjustment layer and an output layer;

[0196] S31: Input layer: The evaluation results of the metacognitive scale questionnaire before the beginning of the semester are used as the data set for training. According to the evaluation results, the students of each level are divided into a ratio of 4:1 and mixed into training set B and test set B respectively;

[0197] The embedding layer includes the following steps:

[0198] S32: Let a student's operational learning behavior sequence be denoted as X = {x1, x2, x3, ..., x n}, use the Item2Vec method to convert each row into x i Embedded as the corresponding d-dimensional behavior vector v xi ∈R d , each student's operation learning behavior sequence X = {x1,x2,x3,...,x n} can be transformed into an operation class behavior sequence vector v i ∈R n×d , where x i It represents a learning behavior of operation-type learning behavior data, such as a single operation such as start watching, and X represents an operation sequence, such as a sequence consisting of multiple actions such as start watching, pause, and continue watching;

[0199] The text learning behavior data of a student is input into the Bloom's cognitive hierarchy evaluation system to evaluate the information quality, and four characteristics of Bloom's cognitive hierarchy, information volume, relevance, and emotional tendency are obtained. The data vector V is obtained by integrating it with the operational learning behavior sequence. i ∈R (n+4)×d ;

[0200] Segment the text-based learning behavior data, use the Word2Vec method on the segmentation results, and convert the n-word text into a d-dimensional word vector, denoted as W i ∈R n×d ;

[0201] The feature extraction layer includes the following steps:

[0202] S33: Use a combination of self-attention mechanism and bidirectional gated recurrent unit network to achieve effective feature extraction;

[0203] S34: embed the vector V obtained by the embedding layer i ∈R (n+4)×d and W i ∈R n×d Further input into the self-attention mechanism, the operation class learning behavior sequence vector V i The output in the self-attention mechanism is denoted as Mi , the text class learning behavior sequence vector W i The output in the self-attention mechanism is denoted by N i , the calculation formulas of operation learning behavior sequence and text learning behavior sequence are as follows:

[0204] M i =Attention(V i W Q ,V i W K ,V i W V )

[0205] N i =Attention(W i W Q ,W i W K ,W i W V )

[0206] Among them, Attention is the self-attention mechanism function, W Q ,W K ,W V are learnable weight matrices used to generate queries, keys, and values, respectively;

[0207] S341: The self-attention mechanism in the feature extraction layer is used to adjust the model so that it focuses on key parts when processing information, while reducing the attention to irrelevant information, thereby improving efficiency and accuracy. Its essence is the mapping between query and key-value pairs, which is applied between the Query element of Target and all elements of Source. The operation steps are as follows:

[0208] Obtain the corresponding attention weight through Query: Calculate the similarity between the target element Q and each Key. The calculation formula is as follows:

[0209] f(Q,K)=QK T

[0210] Among them, Q represents Query, K represents Key, and T represents matrix transposition;

[0211] Weight normalization: Use the Softmax function to normalize the weights, and the normalized weights are recorded as a i , the calculation formula is as follows:

[0212]

[0213] Among them, Q represents Query, K represents Key, i represents the index currently calculating the weight, and j represents the index traversing all weights;

[0214] Weighted sum: The normalized weight a i The weighted sum of the corresponding key value is the attention, and the calculation formula is as follows:

[0215]

[0216] Among them, Q represents Query, K represents Key, V represents Value, and a i represents the normalized weight;

[0217] The self-attention mechanism focuses all attention on the input sequence itself. This mechanism is suitable for processing input sequences of variable length, and can establish global dependencies and expand the receptive field. The calculation formula is as follows:

[0218]

[0219] Among them, Q represents Query, K represents Key, V represents Value, and X∈R d represents a d-dimensional vector. X can represent the original source or basic data of any of Q, K, and V. represents the adjustment factor to avoid X·X T The inner product value is too large, and the softmax value is only 0 or 1. k The value is the key vector dimension;

[0220] N i Further input into the bidirectional gated recurrent unit network, the calculation is shown as follows:

[0221]

[0222] Among them, GRU is a variant of recurrent neural network, which is used to process sequence data;

[0223] S342: The bidirectional gated recurrent unit simultaneously obtains sequence input information from both the sequential and reverse directions, and links the bidirectional hidden layers to better focus on the timing or context information. The calculation formula is as follows:

[0224] r t =σ(W r x t +U r h t-1 )

[0225] z t =σ(W z xt +U z h t-1 )

[0226]

[0227] The current time point is represented as t, where σ is the Sigmoid function, x t Input for the current time, h t-1 is the hidden state of the previous memory unit, r t To reset the gate, we selectively forget the old memory and ignore the hidden state of the previous moment according to the current input and historical context information. When the r value is close to 1, the model is more inclined to consider the previous hidden state, and when the r value is close to 0, the model is more inclined to ignore the previous hidden state. t To update the gate, represents the candidate hidden information to be updated, h t It represents the final output information of the hidden layer obtained after the reset gate and the update gate control. The matrix W and matrix U are the continuously updated weight matrices in the model.

[0228] The input information up to the current time point t is a vector X = {x1, x2, x3, ..., x t}, X represents a piece of text data, where x1 to x t Represents all words after word segmentation. The input of the gated recurrent unit includes the hidden layer output h of the previous moment. t-1 and the input x at this moment t , the output is the hidden state h at the current moment t , through the update gate z in the unit t With reset gate r t The setting controls the hidden state h t The information is updated and the features that are more important to the target are finally retained.

[0229] S35: By connecting a convolutional neural network after the bidirectional gated recurrent unit network, the powerful characteristics of convolution are used to more effectively capture and extract local semantic features of the text for training the Bloom's cognitive hierarchy evaluation module;

[0230] S351: The convolutional neural network includes an input layer A1, a convolutional layer 2, a pooling layer 3, a fully connected layer 4 and an output layer A5. The input layer A1 is responsible for receiving the original image data and passing the original image data to the convolutional layer 2. The convolutional layer 2 is responsible for extracting features and transmitting them to the pooling layer 3. The pooling layer 3 samples and screens the feature map output by the convolutional layer 2. After being processed by multiple convolutional layers 2 and pooling layers 3, the dimension of the data has been greatly reduced. At this time, the output of the pooling layer 3 is flattened into a one-dimensional vector and passed to the fully connected layer 4 for processing. The fully connected layer 4 is responsible for mapping the features extracted by the previous layer to the label space of the sample. The output layer A5 is used for probabilistic output of the classification task or numerical prediction of the regression task.

[0231] S352: Select a text processing task and record it as N=max{n}, where {n} is the set of all single sentence lengths of text i, embed all sentences into n-dimensional vectors, and sequentially link them to obtain a text matrix x i ∈R N×d , let the convolution kernel w∈R h×d , for x i To extract local features, the convolution process calculation formula is as follows:

[0232] s ji ∈f(W j X i:i+h-1 +b)

[0233] Among them, s ji Representation word W i The local features obtained after convolution, h represents the window width, W j ∈R h×k is the jth convolution kernel weight, b∈R is the bias, and the convolution result is the sentence feature map A feature map is a collection of local features extracted by convolution operations, reflecting a certain pattern or feature of the input data;

[0234] The feature information sampling formula is as follows:

[0235]

[0236] in, is the convolution feature, s j is the sentence feature graph;

[0237] S353: Using k convolution kernels w can form k convolution features All Concatenate to form the convolutional feature vector Input S into the fully connected layer 4, and calculate the output layer A5 result y by the following formula:

[0238] y=Softmax(W·S+b)

[0239] Among them, W is the weight matrix of the fully connected layer, S is the convolution feature vector, b is the bias, and y is the output layer A5 result.

[0240] The feature fusion layer includes the following steps:

[0241] S36: fusing the corresponding features of the three channels of the operation learning behavior data channel, the text learning behavior data channel and the information quality features of the course content;

[0242] Perform dual-channel feature fusion on the operation behavior data channel and the text behavior data channel;

[0243] The adjustment layer includes the following steps:

[0244] S37: Using the evaluation results of students' metacognitive ability before the beginning of the semester and the changes in the scores of the two evaluations of students' metacognitive ability before the beginning of the semester and after the end of the semester as data sets for training, respectively, to reflect the changes in students' metacognitive ability;

[0245] The target loss function MAE is used to adjust the evaluation model. The target loss function MAE formula is as follows:

[0246]

[0247] Among them, y i It is expressed as the change value of the real metacognitive ability evaluation score, Represents the node output value;

[0248] After training the adjustment node, all behavior data are re-input and the adjustment node output results are embedded into k dimensions;

[0249] The output layer includes the following steps:

[0250] S38: The softmax function is used to calculate the probability of the classification results of students' metacognitive ability. The calculation formula is as follows:

[0251] P = softmax(W c Z+b c )

[0252] Among them, Z∈R k Represents the high-dimensional features of the output data of the adjustment layer, W c Represented as the softmax model weight matrix, b c Represented as a softmax model bias vector;

[0253] The target loss function Loss is used to measure the difference between the model prediction and the true label. The target loss function Loss formula is as follows:

[0254] Loss=-∑ s p si logp si

[0255] Where p si represents the probability that the evaluation result of student s belongs to category i;

[0256] S39: The metacognitive ability evaluation model includes a basic training module and an adjustment training module;

[0257] The basic training module, adjustment training module and overall model algorithm steps are as follows:

[0258] Input: student operation learning behavior sequence matrix v, text learning behavior information quality evaluation result matrix c and text learning behavior matrix w, set the training batch to epoch

[0259] Output: Classification probability p

[0260] #Train Bloom's cognitive hierarchy evaluation module

[0261] 1.fore inrange(epoch):

[0262] 2.a W ,N=SelfAttention(W) / / Self-attention mechanism obtains weights and output matrix

[0263] 3.G=BiGRU(N) / / Feature extraction to obtain hidden layer feature vector

[0264] 4.p1=Convolution(G) / / Convolution to get Bloom's cognitive hierarchy classification probability

[0265] 5.Loss_Hierarchy=Loss_CE(p1,true_label_Hierarchy) / / Cross entropy loss function

[0266] 6.Update_parameters(Loss_Hierarchy) / / Gradient descent method to update parameters

[0267] #Training basic output module

[0268] 7.foreinrange(epoch):

[0269] 8.a V,M=SelfAttention(V) / / Self-attention mechanism obtains weights and output matrix

[0270] 9.a W ,N=SelfAttention(W)

[0271] 10.U,G=BiGRU(M),BiGRU(N) / / Feature extraction to obtain hidden layer feature vector

[0272] 11.Z = Concatenate_and_Fc(U,G) / / vector linking and full connection

[0273] 12.p=Softmax(Z) / / After the Softmax function, the classification probability p is obtained

[0274] 13.Loss_Basic = Loss_CE(p,true_label_basic) / / Cross entropy loss function

[0275] 14.Update_parameters(Loss_Basic) / / Gradient descent method to update parameters

[0276] #Training and adjusting output module

[0277] 15.foreinrange(epoch):

[0278] 16. Repeat the training steps from line 8 to line 14

[0279] 17.c=Fully_connected(Z) / / The output value c of the adjustment module is obtained through the fully connected layer

[0280] 18.Loss_Adjust=Loss_MAE(c,true_value_adjust) / / MAE loss function

[0281] 19.Update_parameters(Loss_Adjust) / / Gradient descent method to update parameters

[0282] #Train the overall model

[0283] 20.p_final=Fully_connected(p,c) / / After basic output p and adjusted output c, access the fully connected layer

[0284] 21.Loss_Final = Loss_CE(p_final, true_label) / / Cross entropy loss function

[0285] 22.Update_parameters(Loss_Final) / / Gradient descent method to update parameters

[0286] The evaluation results of the metacognition scale questionnaire before the beginning of the semester are used as the data set. According to the evaluation results, the students of each level are divided into a ratio of 4:1 and mixed into training set B and test set B. The evaluation results of the basic output module are shown in Table 13:

[0287] Table 13 Basic output module effect evaluation

[0288]

[0289] Adjust the output module training. This module uses the metacognitive ability changes obtained from two metacognitive scale questionnaire evaluations as the data set for training. The normalized data of all students' metacognitive ability changes are randomly divided into a training set C and a test set C with a ratio of 4:1. The mean absolute error (MAE) is used to evaluate the training results. The evaluation results of the adjusted output module are shown in Table 14:

[0290] Table 14 Adjustment of output module results MAE value

[0291]

[0292] S41: The reasoning stage is based on the output of the basic module training results and the adjustment module training results after the fully connected layer. Since the basic output module and the adjustment output module in the adjustment layer of the previous model are fitted respectively on the relationship between learning behavior and metacognitive ability, and the relationship between learning behavior and changes in metacognitive ability, the model already has implicit features corresponding to the above two types of information. During the second metacognitive scale questionnaire evaluation, after students have gone through one semester of course study, their metacognitive ability at this moment is based on their inherent metacognitive ability and the changes in metacognitive ability in this semester. Therefore, the evaluation results are used as the data set corresponding to the final output of the model.

[0293] Comparative Example 1:

[0294] The method for constructing a student metacognitive ability evaluation model based on student behavior does not change the model structure of Example 1, and does not use Bloom's cognitive hierarchy evaluation results and information quality evaluation results as supplementary features.

[0295] Comparative Example 2:

[0296] The method for constructing a student metacognitive ability evaluation model based on student behavior does not change the model structure of Example 1, and only uses Bloom's cognitive hierarchy evaluation results as a supplementary feature.

[0297] Comparative Example 3:

[0298] The method for constructing a student metacognitive ability evaluation model based on student behavior does not change the model structure of Example 1, and only uses the information quality evaluation results as a supplementary feature.

[0299] The present invention compares and evaluates the results of Example 1 and Comparative Examples 1 to 3 using the F1 value and accuracy, and obtains the following conclusions:

[0300] Compared with Example 1, Example 2 has improved its accuracy in the evaluation results of conditional knowledge, debugging strategy and evaluation dimensions by more than 15%, and the lowest improvement in accuracy of all metacognitive secondary dimensions is 8.34%. The average value of the F1 value index has also improved, with the highest improvement in the information management strategy dimension, which has increased by 15.01%.

[0301] Compared with Example 1, Example 3 has higher evaluation accuracy and F1 average values ​​in each metacognition secondary dimension, and has stronger evaluation ability. Its accuracy in the evaluation results of conditional knowledge, information management strategy and evaluation dimensions has increased by more than 13%, and the lowest increase in accuracy in all metacognition secondary dimensions is 2.34%. The average value of the F1 value index in each evaluation result has also increased, and the highest increase in the information management strategy dimension has increased by 15.01%.

[0302] Compared with comparative example 1, the accuracy of Example 1 in the evaluation of each metacognitive secondary dimension is improved by more than 13%, and the lowest improvement in accuracy in other dimensions is 2.34%; the average value of the F1 value index in each evaluation result is improved by more than 20%, and the procedural knowledge dimension with the highest improvement is improved by 25.40%.

[0303] The present invention introduces Bloom's cognitive hierarchy evaluation system and information quality evaluation system to mine implicit information in operational learning behavior data and textual learning behavior data and increase the amount of input information, thereby effectively improving the model evaluation effect.

[0304] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a student metacognitive ability evaluation model based on student behavior, characterized by: The method comprises a preparation phase, a data collection and preprocessing phase, a training phase and an inference phase; The preparation phase includes the following steps: Use online learning platforms to prepare various data on students’ learning behaviors during course teaching; The student learning behaviors include operational learning behaviors and text-based learning behaviors. The operational learning behaviors include starting or pausing a course video, fast forwarding or rewinding a video, full screen or exiting full screen, adjusting the volume, completing online experiments, etc. generated during the student's online learning process. The text-based learning behaviors include WeChat group interaction, teaching network interaction, completing course summaries, evaluating test papers, and semester course summaries. Prepare operational learning behavior data. The tables used to store data in the online learning platform include the learning behavior table behavior. The learner's student ID in the learning behavior table is the unique primary key and has been hidden for desensitization processing; Prepare text-based learning behavior data and summarize all text-based learning behavior data of each student during the semester; The data collection and preprocessing phase includes the following steps: Use Python regular expressions to remove non-text noise data such as special characters and emoticons in text-based learning behavior data. In addition, use the HIT stop word list to filter stop words, which refer to words with no actual meaning in the word segmentation results. The sentence-partitioned corpus is segmented using the precise Jiaba segmentation model, with words separated by spaces. Represent the text in a form that can be processed by computers, and use the Skip-gram model in Word2Vec for word embedding to preserve the associated information of text data; After embedding, the operational behavior data is directly input into the evaluation model as standard data; due to the complexity of language, the text learning behavior data cannot retain all language information during the embedding process. The information quality evaluation system is designed in combination with the course content, and the text learning behavior data is introduced into the information quality evaluation system; Introduce Bloom's cognitive hierarchy evaluation system into text-based learning behavior data, further explore the implicit information of text-based learning behavior data, use Bloom's cognitive hierarchy evaluation results as additional input to the model, and manually annotate the Bloom's cognitive hierarchy corresponding to text-based learning behavior data based on the word identification library; Introduce text learning behavior data accumulated by online learning platforms in previous semesters, remove unusable data such as short and meaningless data, and finally obtain usable text learning behavior data. Mark the cognitive level of the preprocessed data, and mark them as six integers from 0 to 5 from low to high. Divide the data into 6 categories according to the marking results, and divide each level into 4:1 ratio and merge them into training set A and test set A; Use training set A to train Bloom's cognitive hierarchy evaluation model, and use test set A to evaluate the model effect; Select some questions from the MAI metacognitive awareness scale and improve them in combination with the course content, and finally form a metacognitive scale questionnaire, which includes primary and secondary dimensions. The metacognitive scale questionnaire is distributed at two time points, before the beginning of the semester and after the end of the semester, to evaluate students' metacognitive ability twice, and the reliability and validity of the metacognitive scale questionnaire is analyzed using statistical software to verify its evaluation effect; The two evaluation results were statistically processed and the scale data were processed. The metacognitive ability evaluation results of all students on the secondary dimension of the metacognitive scale questionnaire were arranged from small to large. According to the three equal points, the metacognitive ability scores of students on this dimension were matched to three evaluation results: "weak ability", "average ability" and "strong ability". The training stage includes an input layer, an embedding layer, a feature extraction layer, a feature fusion layer, an adjustment layer and an output layer; Input layer: The evaluation results of the metacognitive scale questionnaire before the beginning of the semester are used as the data set for training. According to the evaluation results, students of each level are divided into a ratio of 4:1 and mixed into training set B and test set B; The embedding layer includes the following steps: The sequence of a student's operational learning behavior is denoted as X = {x1, x2, x3, ..., x n }, use the Item2Vec method to convert each row into x i Embedded as the corresponding d-dimensional behavior vector Each student's learning behavior sequence X = {x1, x2, x3, ..., x n } can be transformed into an operation-like behavior sequence vector v i ∈R n×d , where x i It represents a learning behavior of operation-type learning behavior data, such as a single operation such as start watching, and X represents an operation sequence, such as a sequence consisting of multiple actions such as start watching, pause, and continue watching; The text learning behavior data of a student is input into the Bloom's cognitive hierarchy evaluation system to evaluate the information quality, and four characteristics of Bloom's cognitive hierarchy, information volume, relevance, and emotional tendency are obtained. The data vector V is obtained by integrating it with the operational learning behavior sequence. i ∈R (n+4)×d ; Segment the text-based learning behavior data, use the Word2Vec method on the segmentation results, and convert the n-word text into a d-dimensional word vector, denoted as W i ∈R n×d ; The feature extraction layer includes the following steps: Use a combination of self-attention mechanism and bidirectional gated recurrent unit network to achieve effective feature extraction; The vector V obtained by the embedding layer i ∈R (n+4)×d and W i ∈R n×d Further input into the self-attention mechanism, the operation class learning behavior sequence vector V i The output in the self-attention mechanism is denoted as M i , the text class learning behavior sequence vector W i The output in the self-attention mechanism is denoted by N i , the calculation formulas of operation learning behavior sequence and text learning behavior sequence are as follows: M i =Attention(V i W Q ,V i W K ,V i W V ) N i =Attention(W i W Q ,W i W K ,W i W V ) Among them, Attention is the self-attention mechanism function, W Q ,W K ,W V are learnable weight matrices used to generate queries, keys, and values, respectively; N i Further input into the bidirectional gated recurrent unit network, the calculation is shown as follows: Among them, GRU is a variant of recurrent neural network, which is used to process sequence data; By connecting a convolutional neural network after the bidirectional gated recurrent unit network, the powerful characteristics of convolution are used to more effectively capture and extract the local semantic features of the text for the training of Bloom's cognitive hierarchy evaluation module. The feature fusion layer includes the following steps: The corresponding features of the three channels of operation learning behavior data channel, text learning behavior data channel and information quality features combined with course content are fused into three-channel features; Perform dual-channel feature fusion on the operation behavior data channel and the text behavior data channel; The adjustment layer includes the following steps: The results of the metacognitive ability evaluation of students before the beginning of the semester and the changes in the scores of the metacognitive ability evaluation of students before the beginning of the semester and after the end of the semester were used as data sets for training to reflect the changes in students' metacognitive ability. The target loss function MAE is used to adjust the evaluation model. The target loss function MAE formula is as follows: Among them, y i It is expressed as the change value of the real metacognitive ability evaluation score, Represents the node output value; After training the adjustment node, all behavior data are re-input and the adjustment node output results are embedded into k dimensions; The output layer includes the following steps: The softmax function is used to calculate the probability of the classification results of students' metacognitive ability. The calculation formula is as follows: P=softmax(W c Z+b c ) Among them, Z∈R k Represents the high-dimensional features of the output data of the adjustment layer, W c Represented as the softmax model weight matrix, b c Represented as a softmax model bias vector; The target loss function Loss is used to measure the difference between the model prediction and the true label. The target loss function Loss formula is as follows: Loss=-∑ s p si logp si Where p si represents the probability that the evaluation result of student s belongs to category i; The metacognitive ability evaluation model includes a basic training module and an adjustment training module; The basic training module, adjustment training module and overall model algorithm steps are as follows: Input: student operation learning behavior sequence matrix v, text learning behavior information quality evaluation result matrix c and text learning behavior matrix w, set the training batch to epoch Output: Classification probability p #Train Bloom's cognitive hierarchy evaluation module 1.for e in range(epoch): 2.a W ,N=SelfAttention(W) / / Self-attention mechanism obtains weights and output matrix 3.G=BiGRU(N) / / Feature extraction to obtain hidden layer feature vector 4.p1=Convolution(G) / / Convolution to get Bloom's cognitive hierarchy classification probability 5.Loss_Hierarchy=Loss_CE(p1,true_label_Hierarchy) / / Cross entropy loss function 6.Update_parameters(Loss_Hierarchy) / / Gradient descent method to update parameters #Training basic output module 7.for e in range(epoch): 8.a V ,M=SelfAttention(V) / / Self-attention mechanism obtains weights and output matrix 9.a W ,N=SelfAttention(W) 10.U,G=BiGRU(M),BiGRU(N) / / Feature extraction to obtain hidden layer feature vector 11.Z = Concatenate_and_Fc(U,G) / / vector linking and full connection 12.p=Softmax(Z) / / After the Softmax function, the classification probability p is obtained 13.Loss_Basic = Loss_CE(p,true_label_basic) / / Cross entropy loss function 14.Update_parameters(Loss_Basic) / / Gradient descent method to update parameters #Training and adjusting output module 15.for e in range(epoch):

16. Repeat the training steps from line 8 to line 14 17.c=Fully_connected(Z) / / The output value c of the adjustment module is obtained through the fully connected layer 18.Loss_Adjust=Loss_MAE(c,true_value_adjust) / / MAE loss function 19.Update_parameters(Loss_Adjust) / / Gradient descent method to update parameters #Train the overall model 20.p_final=Fully_connected(p,c) / / After basic output p and adjusted output c, access the fully connected layer 21.Loss_Final = Loss_CE(p_final, true_label) / / Cross entropy loss function 22.Update_parameters(Loss_Final) / / Gradient descent method to update parameters The evaluation results of the metacognition scale questionnaire before the beginning of the semester were used as the data set. According to the evaluation results, the students of each level were divided into a ratio of 4:1 and mixed into training set B and test set B respectively; Adjust the output module training. This module uses the metacognitive ability changes obtained from two metacognitive scale questionnaire evaluations as the data set for training. The normalized data of all students' metacognitive ability changes are randomly divided into a training set C and a test set C with a ratio of 4:

1. The mean absolute error (MAE) is used to evaluate the training results. The reasoning stage is based on the output of the basic module training results and the adjustment module training results after the fully connected layer. Since the basic output module and the adjustment output module in the adjustment layer of the previous model respectively fit the relationship between learning behavior and metacognitive ability, and the relationship between learning behavior and changes in metacognitive ability, the model already has latent features corresponding to the above two types of information. During the second metacognitive scale questionnaire evaluation, after students have experienced one semester of course study, their metacognitive ability at this moment is based on their inherent metacognitive ability and the changes in metacognitive ability in this semester. Therefore, the evaluation result is used as the data set corresponding to the final output of the model.

2. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The online learning platform includes but is not limited to the Moodle online learning platform used in the course, the Touge practical teaching platform, the school teaching network, WeChat, Ketangpai and other channels.

3. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The information quality evaluation system includes three indicators: information quantity, relevance and emotional tendency: The information index is used to extract the features of students' course summary documents. First, students' course summary documents are obtained. The number of topics is set to K, and the parameters α = 50 / K and β = 0.

01. Each topic contains different features. The clustering results of all topics can fully reflect the students' summary information from the perspectives of knowledge points, practices, goals, and gains during the entire course cycle. By setting different K values, a better balance between feature probability and topic classification can be achieved. The relevance index is used to evaluate the information quality of the extended thinking texts in the teaching network courses of students. Teachers publish extended thinking tasks in the teaching network and calculate the relevance in combination with the corresponding chapters; The sentiment tendency index is used to evaluate the information quality of student course evaluation texts. The sentiment tendency of the test paper evaluation text is determined by the number of sentiment words in the comment text. The Hownet sentiment dictionary is used to match the course evaluation data, and the matching text is subjected to sentiment analysis to determine the sentiment tendency of the text and improve the sentiment recognition ability of the model.

4. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The metacognition scale questionnaire includes several questions, each of which is divided into five options: "very inconsistent", "inconsistent", "average", "consistent" and "very consistent", and each option is set with a corresponding score from low to high, among which the "very inconsistent" option is scored 1 point and the "very consistent" option is scored 5 points.

5. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The self-attention mechanism in the feature extraction layer is used to adjust the model so that it focuses on key parts when processing information, while reducing the attention to irrelevant information, thereby improving efficiency and accuracy. Its essence is the mapping between query and key-value pairs, which is applied between the Target element Query and all elements in the Source. The operation steps are as follows: Obtain the corresponding attention weight through Query: Calculate the similarity between the target element Q and each Key. The calculation formula is as follows: f(Q,K)=QK T Among them, Q represents Query, K represents Key, and T represents matrix transposition; Weight normalization: Use the Softmax function to normalize the weights, and the normalized weights are recorded as a i , the calculation formula is as follows: Among them, Q represents Query, K represents Key, i represents the index currently calculating the weight, and j represents the index traversing all weights; Weighted sum: The normalized weight a i The weighted sum of the corresponding key value is the attention, and the calculation formula is as follows: Among them, Q represents Query, K represents Key, V represents Value, and a i represents the normalized weight; The self-attention mechanism focuses all attention on the input sequence itself. This mechanism is suitable for processing input sequences of variable length, and can establish global dependencies and expand the receptive field. The calculation formula is as follows: Among them, Q represents Query, K represents Key, V represents Value, and X∈R d represents a d-dimensional vector. X can represent the original source or basic data of any of Q, K, and V. represents the adjustment factor to avoid X·X T The inner product value is too large, and the softmax value is only 0 or 1. k The value is the key vector dimension.

6. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The bidirectional gated recurrent unit obtains sequence input information from both the sequential and reverse directions at the same time, and links the bidirectional hidden layers to better focus on the timing or context information. The calculation formula is as follows: r t =σ(W r x t +U r h t-1 ) z t =σ(W z x t +U z h t-1 ) The current time point is represented as t, where σ is the Sigmoid function, x t Input for the current time, h t-1 is the hidden state of the previous memory unit, r t To reset the gate, we selectively forget the old memory and ignore the hidden state of the previous moment according to the current input and historical context information. When the r value is close to 1, the model is more inclined to consider the previous hidden state, and when the r value is close to 0, the model is more inclined to ignore the previous hidden state. t To update the gate, represents the candidate hidden information to be updated, h t It represents the final output information of the hidden layer obtained after the reset gate and the update gate control. The matrix W and matrix U are the continuously updated weight matrices in the model. The input information up to the current time point t is a vector X = {x1, x2, x3, ..., x t }, X represents a piece of text data, where x1 to x t Represents all words after word segmentation. The input of the gated recurrent unit includes the hidden layer output h of the previous moment. t-1 and the input x at this moment t , the output is the hidden state h at the current moment t , through the update gate z in the unit t With reset gate r t The setting controls the hidden state h t The information is updated and the features that are more important to the target are finally retained.

7. The method for constructing a student metacognitive ability evaluation model based on student behavior according to claim 1 is characterized by: The convolutional neural network comprises an input layer A (1), a convolutional layer (2), a pooling layer (3), a fully connected layer (4) and an output layer A (5). The input layer A (1) is responsible for receiving original image data and passing the original image data to the convolutional layer (2). The convolutional layer (2) is responsible for extracting features and transmitting them to the pooling layer (3). The pooling layer (3) samples and screens the feature map output by the convolutional layer (2). After being processed by multiple convolutional layers (2) and pooling layers (3), the dimension of the data has been greatly reduced. At this time, the output of the pooling layer (3) is flattened into a one-dimensional vector and passed to the fully connected layer (4) for processing. The fully connected layer (4) is responsible for mapping the features extracted by the previous layer to the label space of the sample. The output layer A (5) is used to output probabilities for classification tasks or to make numerical predictions for regression tasks. Select the text processing task as N = max{n}, where {n} is the set of all single sentence lengths of text i, embed all sentences into n-dimensional vectors, and sequentially link them to obtain the text matrix x i ∈R N×d , let the convolution kernel w∈R h×d , for x i To extract local features, the convolution process calculation formula is as follows: s ji ∈f(W j X i:i+h-1 +b) Among them, s ji Representation word W i The local features obtained after convolution, h represents the window width, W j ∈R h×k is the jth convolution kernel weight, b∈R is the bias, and the convolution result is the sentence feature map A feature map is a collection of local features extracted by convolution operations, reflecting a certain pattern or feature of the input data; The feature information sampling formula is as follows: in, is the convolution feature, s j is the sentence feature graph; Using k convolution kernels w can form k convolution features All Concatenate to form convolutional feature vector Input S into the fully connected layer (4), and calculate the output layer A (5) result y using the following formula: y=Softmax(W·S+b) Among them, W is the weight matrix of the fully connected layer, S is the convolution feature vector, b is the bias, and y is the output layer A result.

Images