A teacher informationization teaching micro-ability evaluation method based on a deep learning model

Abstract

The application discloses a kind of based on deep learning model's teacher informationization teaching micro ability evaluation method, its characteristics are that the method includes: constructing teacher micro ability classification framework, design teacher micro ability classification standard and evaluation standard, from mobile listening and commenting class system collection commenting class data, clean and transform commenting class text, commenting class text is classified standard and micro ability level evaluation to micro ability, construct based on pre-training deep learning teacher micro ability classification model, construct based on attention mechanism deep learning teacher micro ability evaluation model, by prediction and weighted average teacher micro ability classification prediction result and micro ability level evaluation result are obtained, and teacher micro ability analysis result is visualized to teacher individual portrait etc.Steps.Compared with prior art, the present application has multi-angle scientific evaluation teacher classroom ability, improves data authenticity and accuracy, reduces artificial collection workload and analysis cost, and has higher research practical value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of education artificial intelligence and text mining technology, and in particular to a teacher informationization teaching micro-ability evaluation method based on a deep learning model. BACKGROUND

[0002] Teacher informationization teaching ability is an important content to measure the degree of integration of information technology and education teaching. As an analysis method that can feedback and promote teacher informationization teaching ability, ability evaluation has attracted the attention of researchers. The existing mainstream teacher micro-ability evaluation methods mainly have two kinds: one is large-scale research. Researchers design research questionnaires according to different evaluation situations and analyze the data collected to obtain the rough development level or development index of teacher micro-ability. This method has the disadvantages of strong subjectivity, large granularity and delayed feedback. The second is micro-ability certification based on sufficient evidence. This method can focus on ability-based and use evidence to evaluate the performance of teacher micro-ability, which is suitable for the terminal examination of teacher development project. However, collecting, analyzing and certifying teaching and research evidence for a group of teachers is a long-cycle, high-difficulty and high-cost project, which poses a great challenge to the burden of expert human resources. Therefore, it is necessary to design a teacher informationization teaching micro-ability analysis method based on deep learning model to achieve the purpose of easy evidence acquisition, automatic analysis and normal feedback. SUMMARY

[0003] The present application is a teacher informationization teaching micro-ability evaluation method based on a deep learning model, which uses a method of constructing a teacher micro-ability classification framework, designs a teacher micro-ability classification standard and an evaluation standard, collects evaluation data from a mobile evaluation class system, cleans and transforms the evaluation text, evaluates the micro-ability classification standard and the micro-ability level of the evaluation text, obtains the teacher micro-ability classification prediction result and the micro-ability level evaluation result through prediction and weighted average, and automatically and efficiently evaluates the teacher micro-ability. The method is simple, easy to obtain evidence, automatic analysis and normal feedback, reduces a large amount of manual analysis cost, is especially suitable for the terminal examination of teacher development project, is fair and objective, and has high research and practice value.

[0004] The technical scheme of the present application is: a teacher informationization teaching micro-ability evaluation method based on a deep learning model, which is characterized in that the method comprises the following steps:

[0005] 1) Constructing an informationization teaching teacher micro-ability classification framework based on the characteristics of multimedia teaching environment in primary and secondary schools. The framework combines actual classroom teaching situation and designs 13 micro-ability points from 4 dimensions of learning situation analysis, teaching design, teaching implementation and academic evaluation;

[0006] 2) On the basis of the teacher micro-ability classification framework in step 1, classification standards are formulated for the 13 teacher micro-ability points respectively, and corresponding evaluation standards are formulated for each micro-ability point;

[0007] 3) Collect teacher evaluation data from the mobile evaluation system, perform desensitization processing, and each evaluation data has the following attributes: course ID, teacher ID, course time, evaluation person name ID, actual time of the barrage, video time of the barrage, barrage content and barrage length;

[0008] 4) For the attribute missing and repeated data collected in step 3, use data cleaning tools for transformation and processing. At the same time, use the basic heuristic method of attribute subset selection to detect the data collected in step 3, delete the data with low correlation or redundant attributes. The basic heuristic method of attribute subset selection includes the following algorithms: step-by-step forward selection, step-by-step backward deletion, combination of step-by-step forward selection and step-by-step backward deletion, and decision tree induction;

[0009] 5) Label the teacher evaluation text data processed in step 4, the labeling work is mainly divided into two parts, the first part of the labeling content is the teacher micro-ability classification labeling corresponding to the teacher evaluation text, and the second part of the labeling content is the teacher micro-ability evaluation labeling corresponding to the teacher evaluation text;

[0010] 6) Construct a deep learning model based on pre-training to complete the classification of the teacher micro-ability corresponding to the teacher evaluation text.

[0011] 7) Construct a deep learning model based on attention mechanism to complete the evaluation of the teacher micro-ability corresponding to the teacher evaluation text.

[0012] 8) For each teacher, extract all evaluation data matching the teacher ID from the mobile evaluation system according to the teacher ID to form the evaluation set of each teacher, first use the deep learning model based on pre-training in step 6 to classify all data in the set, then use the deep learning model based on attention mechanism in step 7 to evaluate the classified evaluation text.

[0013] 9) For each teacher, use the weighted average method to calculate the score value of each micro-ability point in the set, and use keyword extraction technology to extract keywords from the teacher's evaluation text set according to different micro-ability points, and use visualization tools to represent them.

[0014] The specific process of step 1) is:

[0015] One micro-ability point is designed from the learning situation analysis dimension, which is learning situation analysis;

[0016] From the perspective of instructional design, three micro-competency points are designed: instructional resource design, learning process design, and learning activity design.

[0017] Six micro-competency points were designed from the perspective of teaching implementation: classroom introduction, classroom teaching and interaction, summary and improvement, learning method guidance, group learning organization and presentation and exchange support.

[0018] Three micro-competency points were designed from the perspective of academic assessment: assessment design and application, accompanying data collection, and assessment result analysis.

[0019] The specific process of step 2) is as follows:

[0020] Learning Situation Analysis: The lesson evaluation text contains analysis of students' learning situation, including their experience, knowledge reserves, learning abilities, learning styles, and learning conditions. This micro-ability assessment evaluates whether teachers can: accurately determine appropriate teaching objectives; provide a scientific basis for overcoming teaching difficulties, selecting teaching strategies, and designing teaching activities; and provide a reference for dynamically adjusting teaching content and methods during teaching.

[0021] Instructional resource design: The lesson evaluation text contains content reflecting the teacher's design of digital educational resources. This micro-competency assessment evaluates whether the teacher can: skillfully use information retrieval methods; enrich the media forms of educational resources; judge the applicability of resources according to the teaching theme; and ensure the scientific nature and timeliness of digital educational resources.

[0022] Learning process design: The evaluation text contains content that reflects the effective integration of learning content, learning strategies, learning environment and other elements in the design of the learning process. This micro-ability evaluation assesses whether the teacher can stimulate students' learning interest, promote student participation, help students master learning methods, develop students' abilities and thinking, promote students' transfer and application of relevant knowledge and methods in real situations, encourage inquiry-based learning, create an equal and open classroom learning atmosphere, and develop students' essential character and values.

[0023] Learning activity design: The evaluation text contains content that reflects the design of learning activities based on curriculum standards and learning objectives, and makes reasonable use of information technology to support the design. This micro-ability assessment evaluates whether teachers can integrate resources from schools, families, and society to expand students' learning space, provide tools or platforms for students to carry out collaborative learning and process recording, support teachers in process tracking, behavior analysis and timely intervention, and improve students' ability to use online resources and technical tools for learning.

[0024] Classroom Introduction: The lesson evaluation text contains content demonstrating the teacher's reasonable use of information technology to design and implement the classroom teaching introduction. This micro-ability assessment evaluates whether the teacher can: arouse students' interest and stimulate their learning motivation; make students feel personally involved and identify with the learning objectives; establish a connection between the content being learned and students' existing knowledge; help build learning confidence and gain a sense of accomplishment; and lay a good foundation for classroom learning.

[0025] Classroom instruction and interaction: The evaluation text contains content reflecting the teacher's use of appropriate information technology to design and optimize classroom instruction and interaction, including explanation, inspiration, demonstration, guidance, and evaluation. This micro-ability assessment evaluates whether the teacher can: help visualize and visualize knowledge and skills; establish connections between knowledge in multiple ways; help students understand key points and critical issues; provide rich learning support for students' participation in knowledge understanding and construction; pay attention to students' different needs; and stimulate students' thinking activities such as perception, memory, imagination, and creation.

[0026] Summary and Improvement: The lesson evaluation text contains content demonstrating the teacher's reasonable use of information technology resources or tools to conduct classroom summary and improvement activities. This micro-ability assessment evaluates whether the teacher can: improve the effectiveness of knowledge consolidation; promote students' overall understanding and application of the learned knowledge and skills; help students more intuitively understand and discover the connections between knowledge; help teachers identify problems in teaching activities and provide targeted guidance; and help students experience and master summarization methods such as connection, induction, and comparison in the activities.

[0027] Learning Method Guidance: The lesson evaluation text contains teaching methods that demonstrate the application of information technology tools or resources to support writing expression, reasoning calculation, research and analysis. This micro-ability assessment evaluates whether the teacher can: clearly and accurately demonstrate and guide the methods; create richer and more appropriate scenarios for the experience, acquisition and transfer of methods; help to check students' mastery of methods; and improve the effectiveness of teacher feedback and guidance.

[0028] Group Learning Organization: The lesson evaluation text contains information demonstrating the teacher's use of information technology to organize and manage learning groups. This micro-competency assessment evaluates whether the teacher can: enrich the ways in which learning groups are formed; increase the participation of group members in learning activities; ensure that group members have equal access to learning resources; understand and effectively promote the progress of group learning in a timely manner; promote timely communication and in-depth interaction among group members; and promote the accumulation and sharing of collective wisdom.

[0029] Demonstration and Communication Support: The lesson evaluation text contains information demonstrating the teacher's use of information technology to support discussions, debates, and presentations of results both inside and outside the classroom. This micro-ability assessment evaluates whether the teacher can: enhance student participation and enthusiasm; expand the time and space for presentations and exchanges; showcase and share process-based learning outcomes in real time; achieve critical thinking and in-depth interaction based on outcomes; and promote the accumulation and consolidation of process-based outcomes.

[0030] Evaluation Design and Application: The evaluation text contains evaluation rubrics designed and applied in teaching. This micro-competency assessment evaluates whether teachers can: improve the refinement of teaching objectives and processes; help students accurately understand learning objectives and evaluation requirements; guide students to adjust their learning processes and strategies in a timely manner; promote students' reflection on learning processes and outcomes; support students in conducting self-evaluation and peer evaluation activities; maintain consistency in evaluation standards and improve the scientific nature of evaluation; and increase students' participation and enthusiasm in the learning process.

[0031] Accompanying data collection for evaluation: The evaluation text contains information demonstrating that teachers use technological tools to collect student learning process information in real time and comprehensively. This micro-ability evaluation assesses whether teachers can: collect student learning behaviors from multiple channels to comprehensively reflect the learning process; record the learning process and learning outcomes in real time and in an orderly manner; optimize the data collection process and enrich the data types; help to identify learning problems in a timely manner and implement targeted interventions; and provide rich data support for the comprehensive evaluation of students' qualities.

[0032] Evaluation Result Analysis: The evaluation text contains information demonstrating that the teacher used appropriate methods or information technology tools to present and interpret the evaluation results. This micro-competency assessment evaluates whether the teacher can: identify the problems in students' learning; explore the potential connections between data to find possible reasons for students' learning problems; and provide evidence to support adjustments to the teaching plan and the implementation of targeted guidance.

[0033] The specific process of step 5) is as follows:

[0034] Teacher Micro-competency Classification and Annotation: A portion of the lesson evaluation texts obtained in step 4) is randomly selected and manually annotated according to the teacher micro-competency classification criteria established in step 2). At least two annotators are selected for each lesson evaluation text, and the Kappa coefficient of the annotation results is calculated. Lesson evaluation texts with a Kappa coefficient less than 0.81 are re-annotated by other annotators. For lesson evaluation texts that do not meet the Kappa coefficient requirement, the category with the most annotations is selected as the classification result. After data annotation is completed, 5% of the 13 categories are sampled for quality control. An accuracy rate of 90% is required to pass the annotation check and proceed to the teacher micro-competency level annotation step. Otherwise, categories that do not reach 90% accuracy must be re-annotated by annotators until the required accuracy rate is achieved before proceeding to the teacher micro-competency level annotation step.

[0035] Teacher Micro-competency Level Labeling: The evaluation text data obtained from the teacher micro-competency classification and labeling steps are labeled with teacher micro-competency levels according to the teacher micro-competency point classification criteria in step 2). The threshold for teacher micro-competency level labeling is [0,5], where a larger value indicates a higher level and vice versa. At least two data labelers are selected to label each evaluation text data. The difference in labeling scores between the labelers is calculated. For evaluation text data with a score difference greater than 0.8, other data labelers are selected for labeling again. For evaluation text data that do not meet the score difference requirement, the average value of the labelers' scores is selected as the level labeling result. A quality check is performed on 5% of the 13 categories of labeling results. The mean square error must be less than 0.64 to pass the labeling check and proceed to step 6). Otherwise, the categories with a mean square error greater than 0.64 need to be relabeled by different data labelers until the mean square error is less than the threshold before proceeding to step 6.

[0036] The specific process of step 6) is as follows:

[0037] Text preprocessing: Text preprocessing operations include removing specified symbols from the cleaned and labeled data, text segmentation, removal of stop words, and filtering of low-frequency words. For the lesson evaluation annotation text that passed the annotation detection in step 5, firstly, spaces and emoticons in the lesson evaluation text are removed. Then, Chinese word segmentation tools are used for text segmentation. Next, a Chinese stop word list is used to optimize the word segmentation results. Then, a low-frequency word threshold is set to remove low-frequency words in the word segmentation. Finally, the text data is divided into training set, validation set, and test set in a 6:2:2 ratio. The Chinese word segmentation tools include: jieba word segmentation tool, LAC (Lexical Analysis of Chinese) tool, etc.; the Chinese stop word list includes: Harbin Institute of Technology Chinese stop word list, Baidu Chinese stop word list, etc.

[0038] Model pre-training: Build a deep learning model based on pre-training, select pre-trained educational text data as the pre-training corpus, and input the pre-training corpus into the pre-trained deep learning model BERT-wwm for model pre-training. The pre-training tasks are two tasks of predicting words and predicting the next sentence. The prototype of the deep learning model based on pre-training is BERT-wwm. The pre-trained educational texts include educational news in the THUCNews dataset, educational news on the Toutiao client, etc.

[0039] Model training: On the basis of model pre-training, this invention connects BERT-wwm to a recurrent convolutional neural network (Recurrent CNN, RCNN), and finally connects to the fully connected layer of the neural network as the classification output. The model is simply called BERT-wwm-RCNN. At this time, BERT-wwm retains the model parameters obtained after model pre-training, and the parameters of the recurrent convolutional neural network and the fully connected layer are initialized using the Xavier initialization method. Input the pre-processed training set into BERT-wwm-RCNN for model training. During the training process, cross-entropy is selected as the training error of the model, and the Adam weight decay method with increased offset correction is used for optimization training. After reaching the training end condition, the training ends and enters the model evaluation and verification step. The training end condition is to reach the model training iteration times and the error drops to the threshold, etc. In this step, the hyperparameters of the model BERT-wwm-RCNN are adjusted using the pre-processed validation set. The method for adjusting hyperparameters is grid search;

[0040] Model evaluation and verification: After the model training is completed, the performance of the model is tested using the pre-processed test set. The test methods include using classification accuracy, precision, recall, and F1 Score, etc. After selecting one or more test methods, a passing rate is set for each test method. Only when all test methods pass the passing rate can the model pass the standard and be used for the subsequent steps. If it fails to meet the standard, incremental training is required. The incremental training method is to label more text data for training, reduce the learning rate, and increase the model iteration times, etc.

[0041] The specific process of step 7) is as follows:

[0042] Model Training: Construct a bidirectional long short-term memory network with an attention mechanism (BiLSTM Model of Attention Mechanism), and connect the fully connected layer to the model to obtain the regression output. All parameters of the model are initialized using Xavier. Select the preprocessed text in step 6), and input the training set into the dual long short-term memory network with an attention mechanism. During the training process, use the mean squared error as the training error, and at the same time use the Adam optimizer to optimize the training process. After reaching the training end condition, end the training and enter the model evaluation and verification step. The training end condition is to reach the model training iteration times and the error drops to the threshold, etc. In this step, use the validation set after text preprocessing in step 6) to adjust the hyperparameters of the bidirectional long short-term memory network with an attention mechanism. The method for adjusting hyperparameters is grid search;

[0043] Model Evaluation and Verification: After the model training in step 7) is completed, use the test set after text preprocessing in step 6) to perform the performance test of the model. The test methods include using methods such as mean absolute error loss, mean squared error loss, and Huber loss. After selecting one or more test methods, set the passing rate for each test method. Only when all test methods pass the passing rate can the model pass the standard and be used in the subsequent steps. If it cannot pass the standard, incremental training needs to be carried out. The incremental training method is to label more text data for training, reduce the learning rate, and increase the model iteration times, etc.

[0044] Compared with the prior art, the present invention is closely combined with the actual classroom teaching situation in primary and secondary schools, scientifically evaluates the teacher's classroom ability from multiple angles, uses the data of the real teacher's lesson observation and evaluation platform in data collection, which not only improves the data authenticity but also reduces the manual collection workload. In data preprocessing, data cleaning and data reduction methods are used to preprocess the data, and a rigorous data annotation method is adopted to provide accurate text training data for subsequent model analysis. In the teacher's micro-ability analysis model, a pre-training method is introduced, which integrates external educational knowledge and improves the accuracy of teacher's micro-ability classification. After the teacher's micro-ability analysis model training is completed, the subsequent teacher's micro-ability evaluation method is obtained from model analysis, which reduces a large amount of manual analysis cost and has high research and practical value. Brief Description of the Drawings

[0045] Figure 1 It is the flow chart of the present invention;

[0046] Figure 2 [[ID=IM16]]It is the visualization result of the teacher's micro-ability in Embodiment 1. Detailed Embodiment

[0047] Refer to Figure 1The present invention conducts teacher micro-ability assessment according to the following steps:

[0048] S1: Based on the characteristics of multimedia teaching environment in primary and secondary schools, a micro-ability classification framework for information-based teaching teachers is constructed. This framework combines actual classroom teaching scenarios and designs 13 micro-ability points from 4 dimensions: learning situation analysis, instructional design, instructional implementation, and academic assessment.

[0049] Design one micro-competency point from the perspective of learning situation analysis; this micro-competency point is learning situation analysis.

[0050] From the perspective of instructional design, three micro-competency points are designed: instructional resource design, learning process design, and learning activity design.

[0051] Six micro-competency points were designed from the perspective of teaching implementation: classroom introduction, classroom teaching and interaction, summary and improvement, learning method guidance, group learning organization and presentation and exchange support.

[0052] Three micro-competency points were designed from the perspective of academic assessment: assessment design and application, accompanying data collection, and assessment result analysis.

[0053] S2: Based on the teacher micro-ability classification framework in step S1, classification standards are formulated for 13 teacher micro-ability points, and corresponding evaluation dimensions are formulated for each micro-ability point. For details, please refer to the invention content.

[0054] S3: Collect teacher evaluation text data from the mobile lesson observation and evaluation system, and perform anonymization processing. Each evaluation text data has the following attributes: course ID, teacher ID, lesson start time, evaluator's name ID, actual time of bullet comments, bullet comment video time, bullet comment content, and bullet comment length.

[0055] S4: For the data with missing or duplicate attributes collected in step S3, use data cleaning tools to transform and process it. Simultaneously, use a basic heuristic method for attribute subset selection to detect and delete data with low relevance or redundant attributes. This basic heuristic method includes the following algorithms: stepwise forward selection, stepwise backward deletion, a combination of stepwise forward selection and stepwise backward deletion, and decision tree induction.

[0056] S5: Annotate the teacher evaluation texts processed in step S4. The annotation work is mainly divided into two parts. The first part is the annotation of the teacher evaluation text corresponding to the teacher's micro-ability classification. The second part is the annotation of the teacher evaluation text corresponding to the teacher's micro-ability evaluation. The specific process of this step is as follows:

[0057] Step S5-1: Teacher Micro-competency Classification and Annotation: Select a portion of the lesson evaluation texts obtained in Step S4 and manually annotate them according to the teacher micro-competency classification criteria in Step S2. For each lesson evaluation text, at least two data annotators are selected to annotate it, and the Kappa coefficient of the annotation results is calculated. For lesson evaluation texts with a Kappa coefficient less than 0.81, other data annotators will be selected for annotation again. For lesson evaluation data that does not meet the Kappa coefficient requirement, the category with the most annotations by annotators is selected as the classification result. After the data annotation is completed, 5% of the 13 categories of annotation results are sampled for quality control. An accuracy rate of 90% is required to pass the annotation check and proceed to Step S5-2. Otherwise, the categories that do not reach 90% accuracy must be re-annotated by annotators until the accuracy rate is achieved, proceeding to Step S5-2.

[0058] Step S5-2, Teacher Micro-ability Level Labeling: The lesson evaluation text data obtained in Step S5-1 is labeled with teacher micro-ability levels according to the teacher micro-ability point classification criteria in Step S-2. The threshold for teacher micro-ability level labeling is [0, 5], where a larger value indicates a higher level, and vice versa. At least two data labelers are selected to label each lesson evaluation text data. The difference in labeling scores between the labelers is calculated. For lesson evaluation text data with a score difference greater than 0.8, other data labelers are selected for labeling again. For lesson evaluation text data that do not meet the score difference requirement, the average value of the labeling scores is selected as the level labeling result. A quality check is performed on 5% of the 13 categories of labeling results. The mean square error (MSE) must be less than 0.64 to pass the labeling check and proceed to Step S6. Otherwise, the categories with an MSE greater than 0.64 need to be relabeled by different data labelers until the MSE is less than the threshold before proceeding to Step S6.

[0059] S6: Construct a pre-trained deep learning text classification model to classify the teacher evaluation data corresponding to the teacher's micro-ability points. The specific process of this step is as follows:

[0060] Step S6-1, Text Preprocessing: Text preprocessing includes removing specified symbols, text segmentation, stop word removal, and low-frequency word filtering from the cleaned and labeled data. For the lesson evaluation annotation text that passed the standard test in Step S5, firstly, spaces and emoticons are removed from the evaluation text. Then, Chinese word segmentation tools are used for text segmentation, including jieba and LAC (Lexical Analysis of Chinese). Next, a Chinese stop word list is used to optimize the segmentation results, including the Harbin Institute of Technology Chinese stop word list and Baidu Chinese stop word list. Then, a low-frequency word threshold is set to remove low-frequency words from the segmented text. Finally, the text data is divided into a training set, a validation set, and a test set in a 7:2:2 ratio.

[0061] Step S6-2, Model Pre-training: First, a pre-trained deep learning model is constructed, with BERT-wwm as the prototype. Second, pre-trained educational text data is selected as the pre-training corpus, including educational news from the THUCNews dataset and educational news from the Toutiao client. Next, the pre-training corpus is input into the constructed pre-trained deep learning model BERT-wwm for pre-training. The pre-training tasks are word prediction and next sentence prediction.

[0062] Step S6-3, Model Training: Based on the model from Step S62, this invention connects BERT-wwm to a Recurrent Convolutional Neural Network (RCNN), and finally connects it to a fully connected layer as the classification output. The model is abbreviated as BERT-wwm-RCNN. At this point, BERT-wwm retains the model parameters obtained after pre-training in Step S62, and the parameters of the Recurrent Convolutional Neural Network and the fully connected layer are initialized using the Xavier initialization method. The text data training set processed in Step S61 is input into BERT-wwm-RCNN for model training. During training, cross-entropy is selected as the model's training error, and the Adam weight decay method with added offset correction is used for optimization training. Training ends when the training termination condition is met, proceeding to Step S6-4. The training termination condition is reaching the required number of model training iterations and the error decreasing to a threshold. In this step, the hyperparameters of the BERT-wwm-RCNN model are adjusted using the text data validation set processed in Step S6-1. The hyperparameter adjustment method is grid search.

[0063] Step S6-4, Model Evaluation and Validation: After model training is complete, the performance of the model trained in Step S6-3 is tested using the text data test set processed in Step S6-1. The testing methods include classification accuracy, precision, recall, and F1 score. After selecting one or more testing methods, a pass rate is set for each method. Only when all testing methods pass the pass rate can the model be considered qualified and used in subsequent steps. If the model fails to meet the pass rate, it needs to return to Step S6-3 for incremental training. The incremental training method involves providing more labeled text data for training, reducing the learning rate, and increasing the number of model iterations.

[0064] S7: Construct a deep learning model based on the attention mechanism to evaluate the teacher's micro-ability points corresponding to the teacher's lesson evaluation data. The specific process of this step is as follows:

[0065] Step S7-1, Model Training: Construct a BiLSTM Model of Attention Mechanism based on an attention mechanism. The model is connected to a fully connected layer to obtain regression output. All model parameters are initialized using Xavier. The teacher evaluation data text from step S61 is selected, and its training set is input into the BiLSTM model of attention mechanism. During training, mean squared error is used as the training error, and the Adam optimizer is used to optimize the training process. Training ends when the training termination condition is met, and the process proceeds to step S72. The training termination condition is reaching the required number of model training iterations and the error decreasing to a threshold. In this step, the hyperparameters of the BiLSTM Model of Attention Mechanism are adjusted using the validation set of text data processed in step S61. The hyperparameter adjustment method is grid search.

[0066] Step S7-2, Model Evaluation and Validation: After the model training in Step S7-1 is completed, the model's performance is tested using the text data test set processed in Step S6-1. The testing methods include Mean Absolute Error Loss, Mean Squared Error Loss, and Huber Loss. After selecting one or more testing methods, a pass rate is set for each method. Only when all testing methods pass the pass rate can the model be considered qualified and used in subsequent steps. If the model fails to meet the pass rate, it needs to return to Step S7-1 for incremental training. The incremental training method involves providing more labeled text data for training, reducing the learning rate, and increasing the number of model iterations.

[0067] S8: For each teacher, extract all evaluation data that match the teacher ID from the teacher evaluation system to form an evaluation data set for each teacher. First, use the pre-trained deep learning text classification model from step 6 to classify all data in the set. Then, use the deep learning model based on the attention mechanism from step 7 to evaluate the classified evaluation data.

[0068] S9: For each teacher, the score of each micro-ability point in the set is calculated using a weighted average method, and keywords are extracted from the teacher's evaluation text set according to different micro-ability points using keyword extraction technology, and then displayed using visualization tools.

[0069] The following example, using all the lesson evaluation data of a certain teacher, further illustrates the present invention: Example 1

[0070] 1) De-identify teacher evaluation data collected from the mobile lesson observation and evaluation system.

[0071] The system collected one evaluation data point from this teacher: {Course ID: 440764231, Teacher ID: 2341, Start Time: 2018-09-19 14:01:50, Actual Time of Comments: 2018-09-20 08:03:50, Video Time of Comments: 05:21, Comment Content: "The video is a bit long, but it doesn't convey much effective information to the students [facepalm]. I suggest that students watch it with questions in mind, and explore further if possible." Comment Length: 52}.

[0072] 2) Use data cleaning tools to transform and process the data.

[0073] For the data with missing or duplicate attributes collected in step 1), data cleaning tools are used to transform and process it. Simultaneously, the basic heuristic method for attribute subset selection is used to examine the data collected in step 1), deleting data with low relevance or redundant attributes.

[0074] 3) Annotation of teacher evaluation texts

[0075] The teacher evaluation texts processed in step 2) were annotated according to the teacher micro-ability classification framework. The annotation work was mainly divided into two parts. The first part was the annotation of the teacher evaluation text corresponding to the teacher micro-ability classification, and the second part was the annotation of the teacher evaluation text corresponding to the teacher micro-ability evaluation. For example, if the teacher's evaluation text was "The video is a bit long, but it does not convey much effective information to the students [facepalm]. It is recommended that students watch it with questions in mind and explore further", two data annotators were selected to annotate it. The two data annotators annotated its micro-ability as "instructional resource design" and the micro-ability evaluation was 2.8.

[0076] 4) Classification of Teachers' Micro-competency Points

[0077] A pre-trained deep learning model was constructed to classify teacher evaluation texts corresponding to teachers' micro-ability points. For the evaluation texts detected through annotation in step 3), spaces and emoticons were first removed. Then, LAC Chinese word segmentation was used for text segmentation. Next, the Baidu Chinese stop word list was used to optimize the segmentation results, and a low-frequency word threshold was set to remove low-frequency words. The text data was divided into training, validation, and test sets in a 6:2:2 ratio. The BERT-wwm model was then pre-trained. The pre-trained educational texts included educational news from the THUCNews dataset and educational news from the Toutiao client. The pre-training tasks were word prediction and next sentence prediction. Based on the model pre-training, the BERT-wwm model was connected to a recurrent convolutional neural network, and finally, a fully connected layer was connected to the neural network as the classification output. At this point, the BERT-wwm model retained the model parameters obtained after pre-training, and the parameters of the recurrent convolutional neural network and the fully connected layer were initialized using the Xavier initialization method. The preprocessed text training set is input for model training. Cross-entropy is used as the training error, and the Adam weight decay method with offset correction is employed for optimization. Training ends after the required number of iterations, and the model proceeds to evaluation and validation. In this step, the hyperparameters are adjusted using the preprocessed text validation set via grid search. After training, the model's performance is tested using the preprocessed text test set, employing classification accuracy. The model is considered successful only if the classification accuracy reaches 85%, and is then used in subsequent steps. If the accuracy is insufficient, incremental training is performed by providing more labeled text data.

[0078] 5) Evaluation of teachers' micro-competencies

[0079] A deep learning model based on an attention mechanism is constructed to evaluate teachers' micro-abilities corresponding to teacher evaluation texts. A bidirectional long short-term memory (LSTM) network based on an attention mechanism is used, with the model connected to fully connected layers to obtain regression output. All model parameters are initialized using Xavier. The preprocessed text from step 4) is selected, and its training set is input into the bidirectional LSM network based on an attention mechanism. Mean squared error is used as the training error during training, and the Adam optimizer is used to optimize the training process. Training ends after the required number of iterations, and the model evaluation and validation steps begin. In this step, the hyperparameters of the bidirectional LSM network based on an attention mechanism are adjusted using the validation set from step 4) using grid search. After model training, the model's performance is tested using the test set from step 4) using the mean absolute error loss method. Only when the absolute error is less than 0.64 can the model meet the standard and be used in subsequent steps. If it does not meet the standard, incremental training is required, using more labeled text data for training.

[0080] 6) For the teacher's lesson evaluation dataset, firstly, the pre-trained deep learning model from step 4) is used to classify all data in the dataset. Then, the attention-based deep learning model from step 5) is used to evaluate the classified lesson evaluation data. Three random data points from this teacher are selected, and the model's classification and evaluation results are shown below:

[0081] {“Bullet screen content”: “The steps before the demonstration are below, but the students in the back can’t see them clearly”, Micro-ability category: “Demonstration, communication, and support”, “Micro-ability evaluation”: “2”};

[0082] {“Comment”: “The experiment was conducted with goggles on, which is standard practice. I’ll wear goggles next time I do this experiment in science class too [grinning face]”, Micro-ability category: “Learning process design”, “Micro-ability assessment”: “4.2”};

[0083] {“Bullet Screen Content”: “You can ask questions about what conditions the experimental materials need to meet before making comparisons”, Micro-ability Category: “Classroom Teaching and Interaction”, “Micro-ability Assessment”: “3”}.

[0084] 7) Design a visual interface

[0085] See Figure 2 We used a weighted average method to calculate the score of each micro-ability point in the evaluation set for teacher Zhou Li, and used keyword extraction technology to extract keywords from the teacher's evaluation set according to different micro-ability points, and designed a visual interface to display them.

[0086] This invention constructs a micro-ability classification framework for teachers' information-based teaching. This framework, combined with actual classroom teaching scenarios, designs 13 micro-ability points across four dimensions: student learning analysis, instructional design, instructional implementation, and academic assessment. A pre-trained teacher evaluation text classification model is constructed, and existing pre-trained deep learning classification models are technologically innovated by introducing a deeper neural network structure and a superior training method, thus improving classification accuracy. A teacher micro-ability level evaluation model is also constructed, providing a reference for deep learning analysis in teacher micro-ability evaluation. After both models are trained, a large number of unlabeled teacher evaluation texts can be classified and their quality assessed, effectively improving efficiency while maintaining accuracy. Furthermore, this evaluation method clearly demonstrates the teacher's performance in each micro-ability point, providing valuable reference for subsequent teaching improvement and teacher research.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model, characterized in that... The method specifically includes the following steps: Step 1: Based on the characteristics of multimedia teaching environments in primary and secondary schools, construct a classification framework for teachers' micro-abilities in information-based teaching. This framework designs 13 micro-ability points for teachers from four dimensions: learning situation analysis, instructional design, instructional implementation, and academic assessment. Step 2: Based on the teaching micro-competency classification framework in Step 1, formulate classification criteria and evaluation criteria for each of the 13 teacher micro-competency points; Step 3: Collect teacher evaluation data from the mobile lesson observation and evaluation system, and anonymize it to obtain each evaluation data with the following attributes: course ID, teacher ID, lesson start time, evaluator's name ID, actual time of bullet comments, bullet comment video time, bullet comment content, and bullet comment length. Step 4: Use data cleaning tools to clean and transform the above evaluation data, removing data with missing or duplicate attributes. At the same time, the basic heuristic method of attribute subset selection is used for detection to delete data with low relevance or redundant attributes; Step 5: Annotate the teacher evaluation text data processed in Step 4. The annotations are the teacher micro-ability classification annotations and evaluation annotations corresponding to the teacher evaluation texts. Step 6: Construct a pre-trained deep learning model to classify the teacher evaluation texts corresponding to the teacher's micro-abilities; Step 7: Construct a deep learning model based on the attention mechanism to evaluate the teacher's micro-abilities corresponding to the teacher's lesson evaluation text; Step 8: Extract all evaluation data matching the teacher ID from the mobile lesson observation and evaluation system based on the teacher ID to form an evaluation set for each teacher. Use the pre-trained deep learning model built in Step 6 to classify all data in the evaluation set. Then use the deep learning model based on the attention mechanism built in Step 7 to evaluate the classified evaluation text. Step 9: Calculate the score of each micro-ability point in the evaluation set using a weighted average method. Use keyword extraction technology to extract keywords from the teacher's evaluation set according to different micro-ability points. Use visualization tools to create a micro-ability profile for the teacher and represent each of the 13 designed teacher micro-ability points one by one. The pre-trained deep learning model is a BERT-wwm trained on pre-trained educational text data as pre-training corpus. The BERT-wwm is then connected to a recurrent convolutional neural network, and finally connected to a fully connected layer of the neural network as the classification output, thus obtaining the pre-trained deep learning model. The attention-based deep learning model is a bidirectional long short-term memory network based on an attention mechanism. The regression output obtained by connecting the model to a fully connected layer is the attention-based deep learning model.

2. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... The design encompasses four dimensions: student learning analysis, instructional design, instructional implementation, and academic assessment, encompassing 13 micro-competency points for teachers. The learning situation analysis dimension includes one micro-competency point: learning situation analysis; the instructional design dimension includes three micro-competency points: instructional resource design, learning process design, and learning activity design; the instructional implementation dimension includes six micro-competency points: classroom introduction, classroom teaching and interaction, summary and improvement, guidance on learning methods, organization of group learning, and support for presentation and exchange; and the academic assessment dimension includes three micro-competency points: assessment design and application, accompanying collection of assessment data, and analysis of assessment results.

3. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... The teacher micro-ability classification and evaluation annotation requires at least two data annotators to annotate each lesson evaluation text, and the annotation quality of the annotators is verified by the Kappa coefficient and the annotation score difference. After the data annotation is completed, 5% of the 13 types of annotation results are sampled for quality checks.

4. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... Step 4 includes: performing text preprocessing on the cleaned and labeled data by removing specified symbols, text segmentation, removing stop words, and filtering low-frequency words.

5. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... The pre-trained deep learning model connects a recurrent convolutional neural network and a fully connected layer on the basis of the pre-trained model, is trained using pre-processed text, and finally sets a verification method to evaluate and verify the model.

6. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... The attention-based deep learning model is a bidirectional long short-term memory network based on the attention mechanism. It is trained using preprocessed text, and finally, a validation method is set to evaluate and validate the model.

7. The method for evaluating teachers' micro-ability in information-based teaching based on a deep learning model according to claim 1, characterized in that... The basic heuristic methods for selecting attribute subsets include: step-forward selection algorithm, step-back deletion algorithm, a combination of step-forward selection and step-back deletion algorithm, and decision tree induction algorithm.

Images