Multi-modal representation based intelligent evaluation method and evaluation system for teaching quality of teacher's classroom

Abstract

This invention designs an intelligent evaluation method and system for teacher classroom teaching quality based on multimodal representation. First, multimodal data related to teacher classroom teaching is acquired and preprocessed for different modalities. Second, for lecture videos, a teacher classroom behavior classification network is constructed and trained to obtain four typical teacher classroom behavior percentage indicators. For lecture courseware and teacher lecture speech-to-text transcripts, three teaching segment percentage indicators are calculated. For lecture speech, lecture speed, pronunciation clarity, and intonation acoustic indicators are calculated. Then, guided by educational evaluation theory, a mapping relationship is established to calculate the corresponding evaluation scores for each indicator. Finally, based on the obtained indicators and their calculated evaluation scores, a classroom teaching quality evaluation report is generated by a large language model under the guidance of prompt word engineering. This invention enables an objective, comprehensive, accurate, and scientific evaluation of teacher classroom teaching quality.

Description

Technical Field

[0001] This invention relates to the fields of educational informatization and artificial intelligence application technology, specifically to a method that utilizes computer vision, speech signal processing, and natural language processing technologies to conduct multi-dimensional quantitative analysis of teachers' classrooms, and combines a large language model to generate a classroom teaching quality evaluation report with interpretability and guidance value. Background Technology

[0002] With the rapid development of artificial intelligence technology, universities are increasingly demanding more precise and intelligent evaluation of classroom teaching quality. In other words, achieving an objective, comprehensive, and scientific evaluation of teachers' classroom teaching quality is of great practical significance for optimizing the classroom teaching environment and empowering teachers' professional growth.

[0003] Existing intelligent evaluation technologies for classroom teaching quality address the issues of low evaluation efficiency and narrow coverage to some extent by deploying cameras and microphones in classrooms and using computer vision and speech recognition technologies to replace manual data collection. However, compared to the complex cognitive and analytical capabilities of human experts, most mainstream classroom evaluation technologies are limited to simple statistics on superficial classroom data, making it difficult to deeply understand the internal logic of the teaching process. Furthermore, when faced with complex real-world classroom scenarios, they still face significant technical challenges, including uneven distribution of behavioral samples, lack of multimodal semantic understanding, limited speech evaluation indicators, and a lack of interpretability of results.

[0004] Existing technologies have the following main problems when dealing with the above content:

[0005] First, the accuracy of identifying negative behaviors is low. Because real classroom teaching data exhibits a significant "long-tail distribution" characteristic, positive behaviors such as lecturing and writing on the blackboard occupy most of the time, while negative behaviors such as playing on mobile phones have fewer samples, leading to frequent misjudgments and missed judgments in the identification of negative behaviors.

[0006] Second, there is a lack of joint understanding of multimodal semantic context. Existing systems often process video or audio in a fragmented manner, neglecting the teaching materials as the core carrier of teaching logic. Due to the lack of semantic analysis of the content of the teaching materials, the system cannot perceive whether the teacher is currently conducting "review and introduction," "explaining core knowledge points," or "summarizing the class." This results in the system only mechanically recording time, failing to reconstruct the teaching structure and rhythm of the class, making the evaluation of the core dimension of "instructional design" merely a formality.

[0007] Third, speech indicators are not fully utilized. Existing technologies mostly focus on superficial indicators such as word count per minute or volume, lacking in-depth quantitative methods based on professional acoustic features such as formant analysis, vowel space area, and dynamic pitch slope, thus failing to comprehensively evaluate the quality of teachers' classroom speech.

[0008] Fourth, the evaluation results are presented in a rigid format and lack meaningful improvement suggestions. Existing systems lack the ability to convert quantitative data into natural language, and cannot generate structured diagnostic reports that incorporate attribution analysis and targeted recommendations, unlike human experts, by combining educational principles. This significantly diminishes the application value of the evaluation results.

[0009] Therefore, how to deeply integrate multimodal data, solve the problem of small sample behavior recognition, construct a teaching process perception model that integrates semantic context, introduce professional acoustic evaluation indicators, and finally realize a classroom teaching quality evaluation report with interpretability and guidance value are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0010] In view of the aforementioned problems, the present invention aims to provide an intelligent evaluation method for teacher classroom teaching quality based on multimodal representation, comprising the following steps:

[0011] S1: Acquire multimodal data related to teachers' classroom teaching, including lecture videos, lecture audio, and lecture courseware, and preprocess the data for different modalities;

[0012] S2: For the preprocessed lecture videos, firstly, a teacher classroom behavior classification network with a channel attention mechanism and a feature projection layer is constructed for the training dataset; the network is trained using a composite loss function based on Focal Loss and Center Loss to obtain the feature weight parameters, classification mapping parameters, and class center feature vectors in the network; then, based on the trained network, the classroom behavior categories of each segment in the teacher's lecture video of a certain lesson to be evaluated are initially judged and corrected; finally, the proportion index of the four types of classroom behaviors in the lesson is obtained.

[0013] S3: For the preprocessed teaching materials, extract the nouns to form a terminology set, and then construct a vocabulary database of three typical teaching links. At the same time, for the text obtained by transcribing the teacher's speech, determine the type of teaching link to which each sentence belongs, and finally calculate the proportion of the three types of teaching links in the lesson to be evaluated.

[0014] S4: For the pre-processed lecture audio, calculate the three acoustic indicators of the teacher's speaking speed, pronunciation clarity, and intonation in the lesson to be evaluated; S5: Based on the teacher's classroom behavior ratio indicator, the teaching segment ratio indicator, and the three acoustic indicators, establish a mapping relationship under the guidance of educational evaluation theory to obtain the corresponding evaluation scores for each indicator.

[0015] S6: Based on the calculated indicators and their corresponding evaluation scores, and guided by the prompt word project, the large language model generates a classroom teaching quality evaluation report for the lesson to be evaluated.

[0016] Optionally, acquire multimodal data related to teachers' classroom teaching, including lecture videos, lecture audio, and teaching materials, and preprocess the data for different modalities. The specific steps are as follows:

[0017] S11: Acquire and preprocess the lecture video: Acquire the teacher's lecture video from a camera facing the podium in the classroom; first, segment the video using a fixed-duration sliding window, and use an object detection tool to detect each segment frame by frame, identifying the area where the teacher is located, and obtaining the corresponding image frame sequence; then, perform alignment and normalization operations on each image frame sequence, and input it into a SlowFast dual-path network to extract high-dimensional spatiotemporal feature vectors containing spatial semantics and motion details. All Composition of feature vector set, using Let m represent the number of video segments; finally, based on typical classroom behaviors, including lecturing, blackboard writing, patrolling, and using mobile phones, for each feature vector in the feature vector set... Label it to obtain its corresponding behavior category label. To form a tag set Thus, a training sample set is constructed. This data is used to train a neural network model for subsequent classification of teacher classroom behavior. Simultaneously, for the teacher's teaching video of a specific lesson to be evaluated, the same processing method is applied to obtain the corresponding feature vector set. ;

[0018] S12: Acquire and preprocess the lecture audio: The lecture audio can be separated from the lecture video; For the lecture audio of a certain lesson to be evaluated, the audio endpoint detection algorithm is first used to scan it, and then based on the short-time energy and zero-crossing rate characteristics of the audio, the background noise and silence gaps are determined and removed, only the effective audio segments carrying semantic information are retained, and the start and end times of each effective audio segment are recorded.

[0019] S13: Obtain and preprocess the teaching materials: The teaching materials can be provided by the teacher in advance or extracted from the teaching video; for the teaching materials of a certain lesson to be evaluated, convert the format to text, remove the garbled characters generated during the format conversion process, and use the stop word list to filter high-frequency non-semantic words, retaining only the text content with substantive semantics.

[0020] Optionally, for the preprocessed lecture videos, firstly, a teacher classroom behavior classification network including a channel attention mechanism and a feature projection layer is constructed for the training dataset; the network is trained using a composite loss function based on Focal Loss and Center Loss to obtain the feature weight parameters, classification mapping parameters, and category center feature vectors in the network; then, based on the trained network, a preliminary judgment and correction is made on the classroom behavior category of each segment in the teacher's lecture video of a lesson to be evaluated; finally, the proportion index of the four types of classroom behavior in the lesson is obtained; specifically including:

[0021] S21: Construct a teacher classroom behavior classification network, which includes a channel attention mechanism layer, a feature projection layer, and a classification layer; specifically, firstly, the training sample set... Each of them The channel attention mechanism layer is formed by sequentially passing through a first unbiased fully connected layer, a ReLU activation function, and a second unbiased fully connected layer.

[0022] ,

[0023] in, For the generated channel weight vector, and The parameters to be estimated are... The activation function is Sigmoid; then, and Element-wise multiplication is performed, and the result is passed sequentially through a fully connected layer, a batch normalization layer, a ReLU layer, and a Dropout layer. These layers form the feature projection layer, and the output is a logical vector corresponding to each behavior category. Ultimately, By classification mapping parameters The classification layer, combined with the Softmax function, outputs confidence scores for four categories of classroom behaviors: lecturing (c=1), blackboard writing (c=2), classroom observation (c=3), and using mobile phones (c=4). :

[0024]

[0025] c represents the various behavior categories. This represents the logical score of the i-th sample in the training sample set corresponding to the c-th type of classroom behavior;

[0027] S22: Based on the composite loss function, train the teacher classroom behavior classification network to determine the optimal parameter values; specifically, on the one hand, in order to solve... To address the issue of imbalanced samples across different categories, Focal Loss is employed.

[0028]

[0029] in, For focusing parameters; on the other hand, in order to make Samples belonging to the same category should be as close as possible in the feature space, while samples belonging to different categories should be as far apart as possible. Center Loss is introduced to provide auxiliary supervision constraints in the feature space.

[0030]

[0031] in, For category The central feature vector;

[0032] The final joint loss function is:

[0033]

[0034] in, To balance the hyperparameters of the two loss terms, the Adam optimizer is used based on the error backpropagation method. Iterative optimization is performed by minimizing the composite loss function to obtain the feature weight parameters in the teacher classroom behavior classification network. and Classification mapping parameters and category center feature vector Thus, the training is completed;

[0035] S23: The feature vector set corresponding to the teacher's teaching video of the lesson to be evaluated. Input the trained network into the network to obtain the confidence score. The preliminary judgment results, based on the highest confidence level, determined the classroom behavior category to which each video segment belonged. ,in for

[0036]

[0037] S24: Yes Statistical smoothing was performed to further refine the data; specifically, for the t-th video segment of this lesson, the refined category of classroom behavior was determined. The category that appears most frequently in the preliminary category judgment results for itself, the k segments before it, and the k segments after it is determined.

[0038]

[0039] in, This is an operation to find the mode (the value that appears most frequently in the data);

[0040] S25: Merge the corrected video clips with continuous and identical classroom behavior categories, record the precise start and end times of each behavior event in the lesson, and finally, sum up all time periods for each of the four behaviors—lecturing, writing on the blackboard, patrolling, and using a mobile phone—in the lesson to obtain the cumulative duration of each of the four classroom behaviors in one lesson. 、 、 、 This means dividing by the total duration of the lesson. ,get:

[0041]

[0042]

[0043]

[0044]

[0045] in, 、 、 The indicator is the percentage of positive teaching behaviors (lecturing, blackboard writing, and classroom observation). The percentage of negative teaching behaviors (playing on mobile phones) in the total duration of a class.

[0046] Compared to existing methods, this embodiment constructs a teacher behavior classification network that includes a channel attention mechanism and a feature projection layer, and builds a joint loss function of Focal Loss and Center Loss to achieve joint gradient optimization. This effectively overcomes the dual challenges of scarce negative classroom behavior samples and high inter-class feature similarity, significantly improving the network's discriminative power and robustness in complex scenarios. At the same time, through a correction strategy, the influence of outliers that may exist in the initial judgment results is effectively eliminated, ensuring the temporal continuity of the same teacher's classroom behavior category and the accuracy of the statistical results.

[0047] Optionally, for the preprocessed teaching materials, nouns are extracted to form a terminology set, and then a thesaurus of three typical teaching segments is constructed. Simultaneously, for the text obtained from the transcription of the teacher's lecture, the type of teaching segment to which each sentence belongs is determined. Finally, the proportion of the three types of teaching segments in a lesson is calculated, specifically including:

[0048] S31: Based on the word segmentation tool, the preprocessed text content of a lesson's teaching materials is sequentially segmented, deduplicated, and tagged with parts of speech to select nouns and form a terminology set. ;

[0049] S32: Assume that there are three typical teaching segments in a lesson, namely, instructional preparation and design. Teaching Implementation and Interaction Teaching Summary and Extension ; respectively, a general feature word database is pre-set for the above three types of teaching links, among which, The vocabulary includes: {goal, today we, this lesson, class, hello everyone, open, textbook, page number, book, last lesson, yesterday's test, summary, this question, review, summarize, homework, practice, test, paper, class dismissed, preview, after class, next lesson, copy down, organize, understand, key points, remember, review, recall, remember, this lesson, today, start, prepare}, The lexicon includes: { Definition, Techniques, Who wants to try, Why, For example, Meaning, Attention, Look at the blackboard, Look here, Raise your hand, Who wants to answer, Discussion, Try it, Understand, Comprehend, Observe, Continue} The vocabulary includes: {test, summary, this question, explanation, summary, homework, practice, quiz, test, class dismissal, preview, after class, next class, copy down, organize, understand, key points, remember};

[0050] S33: Input the preprocessed effective speech segments of this lesson into the streaming automatic speech recognition model for transcription, and output the transcribed text of the teacher's speech in this lesson. ;

[0051] S34: Yes Each sentence in the text is compared with... The keywords are matched; if the sentence matches one of the thesaurus entries, then the sentence is classified as belonging to a certain category. The sentence is marked with the corresponding thesaurus name. If the sentence does not match any of the three thesauruses, then... Mark as empty;

[0052] S35: Preparation and Design of Statistics Teaching Teaching Implementation and Interaction Teaching Summary and Extension The number of successfully matched sentences in the corresponding steps are as follows: ,and The total number of sentences in the text is The percentage indicators of the above three types of teaching links in this lesson are respectively

[0053]

[0054]

[0055]

[0056] Compared with traditional text analysis methods, this step constructs a complete lexicon for each teaching segment, especially in the teaching implementation and interaction lexicon, which includes professional terms involved in a particular lesson. In addition, the KMP algorithm is used as the underlying matching operator to achieve accurate location and rapid locking of keywords in the transcribed text of the teacher's lecture, which greatly improves the clarity of the boundaries of the teaching segment.

[0057] Optionally, for the preprocessed lecture audio, three acoustic indicators are calculated: the teacher's speaking speed, pronunciation clarity, and intonation, specifically including:

[0058] S41: Calculate the teacher's speaking rate based on the number of phonemes: Input all valid speech segments of a pre-processed lesson into a phoneme recognition tool to identify the phonemes; then, sum the number of phonemes contained in all valid speech segments in the lesson and divide by the total duration of all valid speech segments to obtain the phoneme-level speaking rate. ,

[0059]

[0060] Where I represents the number of effective audio segments in a lesson. Let be the number of phonemes contained in the i-th valid speech segment. This is the total duration of all valid audio segments in this lesson;

[0061] S42: Calculate the pronunciation clarity of the teacher's lecture based on the vowel space area: First, select vowel phonemes from all phonemes identified in S41; then, calculate the first formant frequency of each vowel phoneme based on the linear predictive coding method. Second resonance frequency Using the standard formant reference values ​​of the three vowel phonemes / i / , / u / , and / a / as the initial values ​​for clustering, the K-Means algorithm was used to cluster all vowel phonemes into three classes. Then, the average formant value of the vowel phonemes in each class c was calculated.

[0062]

[0063]

[0064]

[0065] in, and respectively For class / i / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / u / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / a / the first The first and second formant frequencies of each vowel phoneme. The total number of samples for this type of vowel phoneme is given; finally, the area of ​​the vowel space is calculated using the geometric determinant formula. As an indicator of the clarity of a teacher's pronunciation. ,

[0066] ;

[0067] S43: Determine the teacher's tone of voice based on the slope of the fundamental frequency at the end of the sentence:

[0068] First, set the threshold. For preprocessed valid speech segments and Silence duration between Perform a judgment; if If the silence is determined to be an intra-sentence pause, the preceding and following segments are merged into a single sentence block; if If the silence is identified as a sentence boundary, the video is divided into two independent segments. This mechanism also segments the audio into a set of independent segments. ;

[0069] Next, for each chunk Lock its termination time The cut length is end window audio clips ;

[0070] Then, based on the autocorrelation method, we obtain The fundamental frequency change sequence of each sampling point in the data Where n is the number of sampling points, and is The product of the speech sampling frequency;

[0071] Next, the least squares method is used to process the sequence. Perform linear regression fitting and calculate the slope of the fitted line. ,

[0072]

[0073] in for The average value, for The average value of each element in the sequence; The sign and magnitude of the value are used as quantitative indicators to judge the teacher's tone of voice. If it exceeds 20 Hz / s, then it is judged To rise in pitch, corresponding to teacher questioning or heuristic teaching, when If it is below -20 Hz / s, then it is judged A falling tone corresponds to a statement or instruction; a tone in between is considered a level tone.

[0074] The intonation determination results of each language chunk in a lesson are statistically analyzed to obtain the percentage of rising, falling, and level intonation. Finally, the proportion of rising and falling intonation durations in the total lesson duration is obtained, i.e., the dynamic intonation proportion. express.

[0075] Compared to traditional evaluation methods that rely solely on subjective listening experience, this method introduces the calculation of phoneme-level speech rate, vowel space area, and the slope of sentence-final fundamental frequency. This maps abstract speech quality features into quantifiable physical indicators, effectively avoiding the randomness bias of subjective human evaluation and significantly improving the accuracy and objectivity of teacher's speech evaluation indicators.

[0076] Optionally, based on the teacher classroom behavior percentage index, the teaching segment percentage index, and three acoustic indicators, and guided by relevant educational evaluation theories and empirical research conclusions, a mapping rule is established to obtain the corresponding evaluation scores for each indicator, specifically including:

[0077] S51: For each proportion of teacher classroom behavior, construct corresponding mapping rules to obtain corresponding evaluation scores; specifically, for positive teaching behaviors of teachers (lecturing c=1, blackboard writing c=2, classroom observation c=3), preset the target threshold and base score, and calculate the teacher's positive behavior score.

[0078]

[0079] in, This is the base score for positive teaching behavior of type c, which satisfies ; The threshold for achieving the standard for positive teaching behavior of type c is [those that meet the following criteria]. ; The percentage of positive teaching behaviors of type c in the total duration of a lesson;

[0080] For negative teaching behaviors of teachers (such as using mobile phones), design a punishment function for negative teaching behaviors: set multi-level tolerance thresholds for duration. Used to filter accidental touches or for emergency viewing; circuit breaker threshold. This represents a serious teaching accident:

[0081]

[0082] in The percentage of negative teaching behaviors in the total duration of a lesson;

[0083] The final teacher classroom behavior scoring function combines positive teaching behavior scores and negative teaching behavior penalty functions. The following is a limit on the score. Within the interval:

[0084] ;

[0085] S52: For the three types of teaching segment percentage indicators, construct corresponding mapping rules to obtain corresponding evaluation scores; specifically, for teaching preparation and design... In this process, guided by Rosenshine's effective teaching principles, its mapping rules are designed as follows:

[0086]

[0087] in for The percentage of each link in the process for Evaluation scores for each stage;

[0088] For teaching implementation and interaction In this phase, guided by the "two-thirds rule" of the Flanders interactive analytics system, it is necessary to comprehensively consider its sum. The impact of the combined effects of various stages on teaching quality is mapped by the following rules:

[0089]

[0090] in for The percentage of each link in the process for Evaluation scores for each stage;

[0091] For teaching summary and extension The process, still guided by the "two-thirds rule" of the Flanders interactive analytics system, has its mapping rules designed as follows:

[0092]

[0093] in for The percentage of each link in the process for Evaluation scores for each stage;

[0094] Using the above rules, the result obtained in step S35 、 、 Mapping separately, we get 、 、 ;

[0095] S53: For the three acoustic indicators, corresponding mapping rules are constructed to obtain the corresponding evaluation scores; specifically, for the speech rate indicator, based on the principle that the speech rate of human natural language is usually stable at 10 to 15 phonemes per second, the mapping rule is designed as follows:

[0096]

[0097] in As an indicator of speech rate, Speech rate is used to evaluate the score;

[0098] For pronunciation clarity metrics, the average clarity of normal and clear lecture speech is typically around [value missing]. ( Within this range, the mapping rule for this conclusion is designed as follows:

[0099]

[0100] in As an indicator of speech rate, A score is given to evaluate pronunciation clarity;

[0101] Regarding the intonation indicator, based on the conclusion that varied intonation is a key element in maintaining students' classroom attention, its mapping rule is designed as follows:

[0102]

[0103] in For the proportion of dynamic intonation, Score the tone of voice.

[0104] Based on the above rules, the results obtained in steps S41, S42, and S43 will be... 、 、 Mapping separately, we get 、 、 ;

[0105] Compared to traditional subjective evaluation methods that rely on personal experience, this step constructs a corresponding quantitative mapping model guided by relevant educational evaluation theories and empirical research findings. This not only eliminates the random bias of subjective human scoring but also ensures the interpretability of automated evaluation scores.

[0106] Optionally, based on the calculated indicators and their corresponding evaluation scores, and guided by the cue word engineering, the large language model generates a classroom teaching quality evaluation report for the lesson to be evaluated, specifically including:

[0107] S61, Construct a dedicated prompt template for generating evaluation reports; this template, as a structured evaluation framework, guides the large language model in simulating the process of teaching supervisors providing teaching quality evaluations, and specifically includes the following key components:

[0108] Role Definition: The course domain big language model is assigned the professional role of a senior teaching supervisor of the course to be evaluated, and is required to generate an evaluation report that includes conclusions and targeted recommendations;

[0109] Input data: Input data includes 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 ;

[0110] Analysis principles: The evaluation analysis must be strictly based on the above-mentioned input data, and the evaluation report must be written in an objective, professional, and constructive tone.

[0111] Output content and format requirements: In terms of content, the generated report must include three components: an overall picture of classroom teaching, in-depth diagnosis from multiple perspectives, and targeted improvement suggestions; in terms of format, it must strictly follow the Markdown format for heading hierarchy.

[0112] S62, the result obtained from step S51 The result obtained from step S51 、 、 The result obtained from step S53 、 、 The result obtained from step S25 、 、 、 The result obtained from step S35 、 、 The results obtained from steps S41, S42, and S43 、 、 Enter this special prompt template;

[0113] S63, under the constraints of this dedicated prompt template, select a general large language model to simulate the evaluation process of teaching supervision, and generate a classroom teaching quality evaluation report for the lesson to be evaluated, including three parts: an overall portrait of classroom teaching, in-depth diagnosis from multiple perspectives, and targeted improvement suggestions.

[0114] Overall Classroom Teaching Profile: A summary text describing the teacher's overall performance in this lesson;

[0115] Multi-faceted in-depth diagnosis: The teaching quality is evaluated from three perspectives: teacher's teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone, and traceable attribution analysis is conducted.

[0116] Targeted improvement suggestions: Based on traceable causes, specific and actionable teaching adjustment suggestions are made regarding teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone.

[0117] This invention proposes an intelligent evaluation system for teacher classroom teaching quality based on multimodal representation, used to implement an intelligent evaluation method for teacher classroom teaching quality based on multimodal representation. The system includes:

[0118] The multimodal data acquisition and preprocessing module is used to acquire multimodal data of teachers in the classroom, including teaching videos, teaching audio, and teaching courseware, and to preprocess the data according to different modalities.

[0119] The classroom behavior analysis module is used to build and train a teacher classroom behavior classification network. The teaching video of a lesson to be evaluated is input into the trained network for preliminary judgment and correction, and finally the percentage of the four types of classroom behaviors in the lesson is obtained.

[0120] The teaching segment analysis module is used to extract three types of typical teaching segment vocabulary from the teaching courseware, and to determine the teaching segment type of each sentence based on the text obtained from the transcription of the teacher's speech, and finally obtain the percentage index of the three types of teaching segments in the lesson to be evaluated.

[0121] The lecture audio analysis module is used to extract the teacher's speaking speed, pronunciation clarity, and intonation from the lecture audio.

[0122] The intelligent scoring module establishes a mapping relationship based on the teacher's classroom behavior ratio, teaching segment ratio, and three acoustic indicators, under the guidance of educational evaluation theory, to obtain the corresponding evaluation scores for each indicator.

[0123] The report generation module, guided by the prompt word engineering, generates a classroom teaching quality evaluation report for the lesson to be evaluated based on the calculated indicators and their corresponding evaluation scores using a large language model.

[0124] The present invention also provides a computer-readable medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described thereon for intelligent evaluation of the quality of teachers' classroom teaching.

[0125] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method for intelligent evaluation of the quality of teachers' classroom teaching.

[0126] Compared with the prior art, the advantages of the present invention are as follows:

[0127] In the dimension of teacher behavior perception in complex classroom scenarios, this invention breaks through the limitations of appearance statistics and achieves accurate identification of teacher classroom behavior. By constructing a deep network incorporating channel attention and feature projection splitting, and introducing a composite loss function for joint gradient optimization, this method forces the model to learn behavioral representations with strong semantic discriminative power in the feature space. This enables the objective and accurate capture of classroom behavior in complex and dynamic classroom scenarios.

[0128] In terms of understanding the inherent logic of teaching, this invention extracts high-dimensional semantic features from courseware as strong prior knowledge and combines them with an efficient sequence matching strategy to achieve a comprehensive and accurate evaluation of teachers' classroom teaching.

[0129] Regarding the objectivity and accuracy of evaluation indicators, this invention introduces the slope calculation of phoneme-level speech rate, vowel space area, and sentence-final fundamental frequency, mapping abstract speech quality features into quantifiable physical indicators, effectively eliminating the randomness bias of manual evaluation, and ensuring the scientific nature and consistency of evaluation results.

[0130] In terms of interpretability and application value, this invention achieves a closed loop from theoretical quantification to intelligent diagnosis. Guided by relevant educational evaluation theories and empirical research findings, it utilizes prompt word engineering to drive a large language model to generate structured reports. By outputting causal analysis and improvement suggestions, it significantly enhances the system's interpretability and practical value.

[0131] In terms of the comprehensiveness and complementarity of evaluation dimensions, this invention achieves deep fusion of multimodal data. By integrating visual, auditory, and textual heterogeneous data, it overcomes the one-sidedness of traditional single-modal evaluation methods and constructs a comprehensive three-dimensional evaluation system. Attached Figure Description

[0132] Figure 1 This is a flowchart of the intelligent evaluation method for teacher classroom teaching quality based on multimodal representation proposed in this invention;

[0133] Figure 2 This is a schematic diagram of the teacher classroom behavior classification network structure designed in this invention;

[0134] Figure 3 This is an example of a teaching quality evaluation report generated by this invention;

[0135] Figure 4 This is a structural diagram of the intelligent evaluation system for teacher classroom teaching quality based on multimodal representation provided by the present invention. Detailed Implementation

[0136] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the following embodiments are for illustrative purposes only and do not constitute a limitation on the scope of the claims. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0137] In the following description, for the intelligent evaluation method and system for multimodal characterization of teacher classroom teaching quality of the present invention, this embodiment uses a 45-minute classroom lecture video of the college course "Signals and Systems" (including lecture audio, video in MP4 format, 1920×1080 resolution, 16kHz stereo) and lecture courseware as multimodal data.

[0138] Figure 1 This is a flowchart of a method and system for intelligent evaluation of teacher classroom teaching quality based on multimodal information processing, provided by an embodiment of the present invention, and is described in detail below.

[0139] Step S1: Acquire multimodal data related to the teacher's classroom teaching, including lecture videos, lecture audio, and lecture slides, and preprocess the data for different modalities. The specific implementation steps are as follows:

[0140] S11: Obtain the teaching video and preprocess it: Obtain the teacher's teaching video from the camera facing the podium in the classroom; First, segment the video using a sliding window with a certain set duration (6 - 10 seconds), and use an object detection tool (e.g., Faster R-CNN model) to perform frame-by-frame detection on each segment of the video, detect and segment the area where the teacher is located from it, and obtain the corresponding sequence of image frames; Then, perform alignment operations on each sequence of image frames, uniformly scale them to a size of 256 pixels and perform normalization processing, and input them into the SlowFast dual-path network to extract an 80-dimensional spatio-temporal feature vector containing spatial semantics and motion details ; It should be noted that the SlowFast dual-path network uses a low-sampling-rate slow path and a high-sampling-rate fast path to extract the static spatial semantics and high-frequency action features of the teacher respectively, and completes feature fusion through lateral connections; All constitute a feature vector set, denoted by , where m is the number of video segments; Finally, according to typical classroom behaviors, including teaching, writing on the blackboard, patrolling, and playing with mobile phones, each in the feature vector set is labeled to obtain its corresponding behavior category label , constituting a label set , thereby constructing a training sample set for training the neural network model for subsequent teacher classroom behavior classification; At the same time, for the teacher's teaching video of a certain class to be evaluated, use the same processing method as above to obtain the corresponding feature vector set .

[0141] S12: Obtain the teaching voice and preprocess it: The teacher's teaching voice can be separated from the teaching video; For the teacher's teaching voice of a certain class to be evaluated, first use the double-threshold method based on the short-time energy and zero-crossing rate of the voice to perform voice endpoint detection, determine and remove background noise and silent gaps, and only retain the effective voice segments carrying semantic information, and record the start and end times of each effective voice segment, and its set is denoted as .

[0142] S13: Obtain the teaching courseware and preprocess it: The teaching courseware can be provided by the teacher in advance or separated from the teaching video; For the teacher's teaching courseware of a certain class to be evaluated, convert its format to text, remove the garbled symbols generated during the format conversion process (e.g., structural particles or function words such as "de", "le", "shi", "zai", etc.), and use a stop word list to filter out high-frequency non-semantic words, and only retain the text content with substantial semantics.

[0143] Step S2: For the preprocessed lecture videos, firstly, for the training dataset, construct a teacher classroom behavior classification network that includes a channel attention mechanism and a feature projection layer; train the network using a composite loss function based on Focal Loss and Center Loss to obtain the feature weight parameters, classification mapping parameters, and category center feature vectors in the network; then, based on the trained network, perform preliminary judgment and correction on the classroom behavior categories of each segment in the teacher's lecture video of a lesson to be evaluated; finally, obtain the percentage index of the four types of classroom behaviors in the lesson. The specific implementation steps are as follows:

[0144] S21: Construct a teacher classroom behavior classification network, with the network structure as follows: Figure 2 As shown, it includes a channel attention mechanism layer, a feature projection layer, and a classification layer; specifically, the training sample set is first... Each of the 80 dimensions The input is sequentially passed through a first unbiased fully connected layer (compressing 80-dimensional features to 20-dimensionality), a ReLU activation function, and a second unbiased fully connected layer (restoring the 20-dimensional input to 80-dimensionality). This process explicitly models the inter-channel dependencies. These layers constitute the channel attention mechanism layer, followed by Sigmoid activation to generate channel weight vectors.

[0145] ,

[0146] in, For the generated channel weight vector, and The parameters to be estimated are... The activation function is Sigmoid; then, and Element-wise multiplication is performed to recalibrate the features. The multiplication result is then passed sequentially through a fully connected layer (mapping the 80-dimensional features to a 64-dimensional hidden space, enabling the network to extract more abstract and compact semantic features), a batch normalization layer (standardizing the feature distribution to prevent gradient vanishing and accelerate convergence), a ReLU layer, and a Dropout layer (with a dropout rate of 0.5 to enhance the model's generalization ability and prevent overfitting). These layers form the feature projection layer, outputting a 64-dimensional logical vector. Ultimately, By classification mapping parameters The classification layer, combined with the Softmax function, outputs a 4-dimensional vector representing the confidence scores of four types of classroom behaviors: lecturing (c=1), blackboard writing (c=2), classroom observation (c=3), and using a mobile phone (c=4). :

[0147]

[0148] c represents the various behavior categories;

[0149] S22: Based on the composite loss function, train the teacher classroom behavior classification network to determine the optimal parameter values; specifically, on the one hand, in order to solve... To address the issue of imbalanced samples across different categories, Focal Loss is employed.

[0150]

[0151] in, The focusing parameter can be set to 2. Focal Loss is chosen because traditional cross-entropy loss tends to predict all samples as a broad class in exchange for improved overall accuracy. Focal Loss introduces a focusing parameter... This dynamically reduces the weight of easily categorized samples (such as numerous "lecture" segments), forcing the model to shift the focus of gradient updates to difficult-to-identify samples like "playing on a mobile phone." Its mathematical expression is achieved through... This adjustment allows the model to pay more attention to misclassified samples with low prediction probabilities during training.

[0152] On the other hand, in order to Samples belonging to the same category should be as close as possible in the feature space, while samples belonging to different categories should be as far apart as possible. Center Loss is introduced to provide auxiliary supervision constraints in the feature space.

[0153]

[0154] in, For category The center feature vector; Center Loss learns the feature center of each class during training. And calculate the high-dimensional features of each sample. its category center The Euclidean distance forces similar samples to converge toward their respective centers in the feature space. This mechanism explicitly compresses intra-class distances, enhancing the discriminative power of features in the latent space, thereby effectively distinguishing visually similar behavioral categories.

[0155] The final joint loss function is:

[0156]

[0157] in, To balance the weights of the two loss terms, the hyperparameter can be set to 0.05. Based on the error backpropagation method, the Adam optimizer is used to iteratively optimize by minimizing the composite loss function, with the learning rate set to... With the batch size set to 16, feature weight parameters in the teacher classroom behavior classification network are obtained. and Classification mapping parameters and category center feature vector This completes the training. In other words, during training, the gradient flow from Focal Loss and the gradient flow from Center Loss converge at the feature projection layer, enabling the network to learn feature representations that are both semantically separable and spatially compact.

[0158] In some embodiments, the method proposed in this invention is compared with several comparative methods. These comparative methods include: SlowFast benchmark network, I3D, Two-Stream, and IDT. A five-fold cross-validation strategy is employed to ensure the objectivity and generalization ability of the results. For evaluation metrics, TOP-1 accuracy, recall, and F1 score are selected as evaluation metrics; the closer the metric value is to 1, the better the performance.

[0159] The results are shown in Table 1. It can be observed that the proposed method achieves the best performance in classifying teacher classroom behavior. Specifically, the TOP-1 accuracy of the proposed method is approximately 0.02 higher than the SlowFast benchmark network and approximately 0.10 higher than I3D. The performance improvement is attributed to the channel attention mechanism layer in the proposed method, which achieves adaptive recalibration of feature responses by explicitly modeling the dependencies between channels, thereby effectively enhancing the model's sensitivity to extracting key teaching features. Furthermore, the recall of the proposed method is approximately 0.06 higher than the SlowFast benchmark network and approximately 0.2 higher than I3D. These improvements are attributed to the construction of a joint loss function combining Focal Loss and Center Loss. By introducing a focusing parameter and minimizing intra-class distance, this strategy effectively alleviates the problems of scarce negative behavior samples and high inter-class feature similarity, significantly enhancing the network's discriminative ability and robustness in complex classroom scenarios.

[0160] Table 1. Performance comparison of the proposed method and the comparison method on the teacher classroom behavior dataset.

[0161] method TOP-1 Accuracy Recall rate F1 score The proposed method 0.8603 0.8315 0.8592 Slowfast (benchmark) 0.8375 0.7622 0.7918 I3D 0.7678 0.6322 0.6944 Two-Stream 0.6845 0.5801 0.6311 IDT 0.5033 0.4178 0.4539

[0162] S23: The feature vector set corresponding to the teacher's teaching video of the lesson to be evaluated. Input the trained network into the network to obtain the confidence score. The preliminary judgment results, based on the highest confidence level, determined the classroom behavior category to which each video segment belonged. ,in for

[0163]

[0164] S24: Yes Statistical smoothing was performed to further refine the data; specifically, for the t-th video segment of this lesson, the refined category of classroom behavior was determined. The category that appears most frequently in the preliminary category judgment results for itself, the k segments before it, and the k segments after it is determined.

[0165]

[0166] in, To extract the mode (the most frequent value in the data), k is typically set to 2. Statistical smoothing effectively filters out isolated erroneous predictions caused by transient interference, ensuring the continuity of behavioral states.

[0167] S25: Merge the corrected video clips with continuous and identical classroom behavior categories, record the precise start and end times of each behavior event within a lesson, and finally, sum up all time periods for each of the four behaviors—lecturing, blackboard writing, patrolling, and using a mobile phone—within the lesson to obtain the cumulative duration of each of the four classroom behaviors within a lesson. 、 、 、 This means dividing by the total duration of the lesson. ,get:

[0168]

[0169]

[0170]

[0171]

[0172] in, 、 、 The indicator is the percentage of positive teaching behaviors (lecturing, blackboard writing, and classroom observation). The percentage of negative teaching behaviors (playing on mobile phones) in the total duration of a class.

[0173] Step S3: For the preprocessed teaching materials, extract the nouns to form a terminology set, and then construct a vocabulary database for three typical teaching segments. Simultaneously, for the text obtained from the teacher's speech transcription, determine the type of teaching segment each sentence belongs to, and finally calculate the proportion of the three types of teaching segments in a lesson. The specific implementation steps are as follows:

[0174] S31: Using a word segmentation tool (e.g., jieba), the preprocessed text of a lesson's teaching materials is segmented, deduplicated, and tagged with parts of speech to select nouns. Furthermore, the importance weight of each noun in the candidate terminology set is calculated using term frequency statistics and inverse document frequency (TF-IDF) algorithms. Based on the weight ranking results, a preset number (usually 30) of high-weight nouns are selected to form the terminology set. For example, in a certain lesson, ={linear systems, nonlinear systems, homogeneity, superposition, decomposability...}.

[0175] S32: Assume that there are three typical teaching segments in a lesson, namely, instructional preparation and design. Teaching Implementation and Interaction Teaching Summary and Extension ; respectively, a general feature word database is pre-set for the above three types of teaching links, among which, The vocabulary includes: {goal, today we, this lesson, class, hello everyone, open, textbook, page number, book, last lesson, yesterday's test, summary, this question, review, summarize, homework, practice, test, paper, class dismissed, preview, after class, next lesson, copy down, organize, understand, key points, remember, review, recall, remember, this lesson, today, start, prepare}, The lexicon includes: { Definition, Techniques, Who wants to try, Why, For example, Meaning, Attention, Look at the blackboard, Look here, Raise your hand, Who wants to answer, Discussion, Try it, Understand, Comprehend, Observe, Continue} The vocabulary includes: {test, summary, this question, explanation, summary, homework, practice, quiz, test, class dismissal, preview, after class, next class, copy down, organize, understand, key points, remember};

[0176] S33: Input the preprocessed effective speech segments of this lesson into the streaming automatic speech recognition model for transcription, and output the transcribed text of the teacher's speech in this lesson. The streaming automatic speech recognition model used here is the Faster-Whisper model. The Whisper series of models are trained on large amounts of multilingual data and have excellent recognition capabilities in mixed scenarios involving Chinese, English, and Chinese technical terms, meeting the needs of university classroom teaching. Faster-Whisper, as an optimized version of Whisper, supports streaming recognition, significantly improving inference speed and reducing resource consumption while maintaining high accuracy, and supporting real-time transcription processing.

[0177] S34: Yes Each sentence in the text is compared with... The keywords are matched; if the sentence matches one of the thesaurus entries, then the sentence is classified as belonging to a certain category. The sentence is marked with the corresponding thesaurus name. If the sentence does not match any of the three thesauruses, then... Mark as empty;

[0178] S35: Preparation and Design of Statistics Teaching Teaching Implementation and Interaction Teaching Summary and Extension The number of successfully matched sentences in the corresponding steps are as follows: ,and The total number of sentences in the text is The percentage indicators of the above three types of teaching links in this lesson are respectively

[0179]

[0180]

[0181]

[0182] The above three indicators can quantify the intensity of teachers' investment in each teaching segment.

[0183] Step S4: Calculate three acoustic indicators for the preprocessed lecture audio: speech rate, pronunciation clarity, and intonation. The specific implementation steps are as follows:

[0184] S41: Calculate the teacher's speaking rate based on the number of phonemes: Input all valid speech segments of a preprocessed lesson into a phoneme recognition tool (e.g., the Wav2Vec 2-XLSR-53 model) to identify the phonemes; then, sum the number of phonemes contained in all valid speech segments of the lesson and divide by the total duration of all valid speech segments to obtain the phoneme-level speaking rate. ,

[0185]

[0186] Where I represents the number of effective audio segments in a lesson. Let be the number of phonemes contained in the i-th valid speech segment. This represents the total duration of all valid audio segments in this lesson.

[0187] S42: Calculate the pronunciation clarity of the teacher's lecture based on the vowel space area: First, select vowel phonemes from all phonemes identified in S41 (vowel phonemes can be identified using a method based on a Hidden Markov Model); then, calculate the first formant frequency of each vowel phoneme based on the Linear Predictive Coding (LPC) method. Second resonance frequency Specifically, this can be implemented using the Burg recursive algorithm module in the Praat software. (Select...) and The reason is that, from the perspective of pronunciation mechanism, Characterizes tongue height and mandibular opening. The vowel space area (VSA), which represents the forward and backward extension of the tongue, directly reflects the limit range of motion of the speaker's articulatory organs (such as the tongue, lips, and jaw). The larger the VSA value, the wider the range of the tongue's rise and fall and forward and backward extension in the oral cavity, the fuller the vowel pronunciation, and the higher the clarity of the sound.

[0188] Then, using the standard formant reference values ​​of the three vowel phonemes / i / (high front vowel, representing the highest point of tongue position), / u / (high back vowel, representing the furthest point of tongue position), and / a / (low vowel, representing the lowest point of tongue position / maximum mouth opening) as the initial values ​​for clustering, the K-Means algorithm was used to cluster all vowel phonemes into three classes. Then, the average formant value of the vowel phonemes in each class c was calculated.

[0189]

[0190]

[0191]

[0192] in, and respectively For class / i / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / u / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / a / the first The first and second formant frequencies of each vowel phoneme. The total number of samples for this type of vowel phoneme; finally, the geometric determinant formula is used to calculate... As an indicator of the clarity of a teacher's pronunciation. ,

[0193] ;

[0194] S43: Determine the teacher's tone of voice based on the slope of the fundamental frequency at the end of the sentence:

[0195] First, set the threshold. =0.7s, for preprocessed effective speech segments and Silence duration between Perform a judgment; if If the silence is determined to be an intra-sentence pause, the preceding and following segments are merged into a single sentence block; if If the silence is identified as a sentence boundary, the video is divided into two independent segments. This mechanism also segments the audio into a set of independent segments. ;

[0196] Next, for each chunk Lock its termination time The cut length is end window audio clips ;

[0197] Then, based on the autocorrelation method, we obtain The fundamental frequency change sequence of each sampling point in the data Where n is the number of sampling points, and is The product of the frequency and the speech sampling frequency; specifically, this can be achieved using the spectral autocorrelation operator in the Praat software, by extracting periodic features within a preset frequency search interval to obtain an effective voiced frequency sequence.

[0198] Next, the least squares method is used to process the sequence. Perform linear regression fitting and calculate the slope of the fitted line. ,

[0199]

[0200] in for The average value, for The average value of each element in the sequence; The sign and magnitude of the value are used as quantitative indicators to judge the teacher's tone of voice. If it exceeds 20 Hz / s, then it is judged To rise in pitch, corresponding to teacher questioning or heuristic teaching, when If it is below -20 Hz / s, then it is judged A falling tone corresponds to a statement or instruction; a tone in between is considered a level tone.

[0201] The intonation determination results of each language chunk in a lesson are statistically analyzed to obtain the percentage of rising, falling, and level intonation. Finally, the proportion of rising and falling intonation durations in the total lesson duration is obtained, i.e., the dynamic intonation proportion. express.

[0202] Step S5: Based on the teacher classroom behavior percentage index, the teaching segment percentage index, and three acoustic indicators, and guided by relevant educational evaluation theories and empirical research conclusions, establish mapping rules to obtain the corresponding evaluation scores for each indicator. The specific implementation steps are as follows:

[0203] S51: For each percentage of teacher classroom behaviors, corresponding mapping rules are constructed to obtain corresponding evaluation scores; specifically, for positive teaching behaviors of teachers (lecturing, blackboard writing, and classroom observation), the preset thresholds and base scores are shown in the table below:

[0204] Table 2 Mapping of Teachers' Positive Teaching Behaviors

[0205] Teaching behavior Qualification threshold base score Evaluation Guidance Explanation Teaching 0.60 40 Core classroom activities have the highest weight and ensure basic teaching assessment. writing on the blackboard 0.30 25 Supplementary teaching methods, with appropriate weight. Inspection 0.10 35 Assigning higher base scores encourages teachers to step down from the podium and pay attention to students' progress.

[0206] According to the table above, the formula for calculating teachers' positive behavior scores is as follows:

[0207]

[0208] in, The base score for positive teaching behaviors in category c is for lecturing (c=1), blackboard writing (c=2), and classroom observation (c=3). This represents the threshold for achieving the positive teaching behavior standard (category c). This represents the percentage of positive teaching behaviors (category c) in the total duration of a lesson.

[0209] For negative teaching behaviors of teachers (such as using mobile phones), design a punishment function for negative teaching behaviors: set multi-level tolerance thresholds for duration. =0.05, used to filter accidental touches or emergency checks; circuit breaker threshold. =0.15 represents a serious teaching accident:

[0210]

[0211] in The percentage of negative teaching behaviors in the total duration of a lesson;

[0212] The final teacher classroom behavior scoring function combines positive teaching behavior scores and negative teaching behavior penalty functions. The following is a limit on the score. Within the interval:

[0213] ;

[0214] S52: For the three types of teaching segment percentage indicators, construct corresponding mapping rules to obtain corresponding evaluation scores; specifically, for teaching preparation and design... In this segment, guided by Rosenshine's effective teaching principles (an effective classroom should begin with a brief review of prior knowledge to reduce cognitive load by activating old knowledge; according to prior research, the golden time for this segment typically accounts for 10%-15% of the early part of the class), its mapping rule is designed as follows:

[0215]

[0216] in for The percentage of each link in the process for Evaluation scores for each stage;

[0217] For teaching implementation and interaction In this process, guided by the "two-thirds rule" of the Flanders Interactive Analysis System (in a typical classroom, approximately two-thirds of the teacher's dominant discourse consists of direct-influence discourse such as lecturing and instructions, while approximately one-third consists of indirect-influence discourse such as praise, adoption of student opinions, and feedback), it is necessary to comprehensively consider both aspects. The impact of the combined effects of various stages on teaching quality is mapped by the following rules:

[0218]

[0219] in for The percentage of each link in the process for Evaluation scores for each stage;

[0220] For teaching summary and extension The process, still guided by the "two-thirds rule" of the Flanders interactive analytics system, has its mapping rules designed as follows:

[0221]

[0222] in for The percentage of each link in the process for Evaluation scores for each stage;

[0223] Using the above rules, the result obtained in step S35 、 、 Mapping separately, we get 、 、 ;

[0224] S53: For the three acoustic indicators, corresponding mapping rules are constructed to obtain the corresponding evaluation scores; specifically, for the speech rate indicator, based on the principle that the speech rate of human natural language is usually stable at 10 to 15 phonemes per second, the mapping rule is designed as follows:

[0225]

[0226] in As an indicator of speech rate, Speech rate is used to evaluate the score;

[0227] Regarding the indicators of articulation clarity, research shows that the vowel space area (VSA) reflects the range of motion of the speaker's articulatory organs (tongue, lips, and jaw). A larger VSA value means that the vowel pronunciation is more "full" and the articulation is clearer; conversely, a value that is too small means that the pronunciation is "unclear" or that there is serious coarticulation.

[0228] The mean VSA of normal and clear lecture audio is usually around [value missing]. ( Within the range of ), its mapping rules are designed as follows:

[0229]

[0230] in As an indicator of speech rate, A score is given to evaluate pronunciation clarity;

[0231] Regarding intonation, based on the conclusion that varied intonation is a key element in maintaining students' classroom attention, the alternation of rising intonation (usually corresponding to questions and emphasis) and falling intonation (corresponding to instructions and affirmations) can create a pleasant vocal melody; therefore, its mapping rule is designed as follows:

[0232]

[0233] in For the proportion of dynamic intonation, Score the tone of voice.

[0234] Based on the above rules, the results obtained in steps S41, S42, and S43 will be... 、 、 Mapping separately, we get 、 、 ;

[0235] Table 3 shows the comparison results between the evaluation scores of each indicator obtained using the method of this invention and the evaluation scores of three teaching supervisors. Classroom data from seven "Signals and Systems" instructors were selected as the analysis sample. The results for each indicator in the table are the average scores given by the seven instructors. It can be observed that the evaluation scores obtained by the method proposed in this invention match well with the evaluation scores given by the teaching supervisors, verifying the effectiveness and reliability of the proposed method.

[0236] Table 3. Comparison between the evaluation scores generated by the method proposed in this invention and the teaching supervision scores.

[0237] Classroom behavior Teaching preparation and design Teaching Implementation and Interaction Teaching Summary and Extension speech rate Clarity of expression tone of voice The method of the present invention 87.91 85.78 86.15 87.71 90.32 91.21 86.92 Supervisor 1 86.28 85.26 85.92 88.15 89.94 90.18 87.19 Supervisor 2 87.77 86.76 87.04 86.91 88.67 90.06 85.98 Supervision 3 88.69 86.12 85.82 87.23 90.04 89.92 87.05

[0238] Step S6: Based on the calculated indicators and their corresponding evaluation scores, and guided by the prompt word engineering, the large language model generates a classroom teaching quality evaluation report for the lesson to be evaluated. Specific implementation steps include:

[0239] S61, Construct a dedicated prompt template for generating evaluation reports; this template, as a structured evaluation framework, guides the large language model in simulating the process of teaching supervisors providing teaching quality evaluations, and specifically includes the following key components:

[0240] Role Definition: The course domain big language model is assigned the professional role of a senior teaching supervisor of the course to be evaluated, and is required to generate an evaluation report that includes conclusions and targeted recommendations;

[0241] Input data: Input data includes 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 ;

[0242] Analysis principles: The evaluation analysis must be strictly based on the above-mentioned input data, and the evaluation report must be written in an objective, professional, and constructive tone.

[0243] Output content and format requirements: In terms of content, the generated report must include three components: an overall picture of classroom teaching, in-depth diagnosis from multiple perspectives, and targeted improvement suggestions; in terms of format, it must strictly follow the Markdown format for heading hierarchy.

[0244] Based on the above description, a specific example of a structured prompt word template is as follows:

[0245] Role Definition

[0246] You are an "AI-powered teaching diagnosis expert system" based on educational psychology theories. Your task is to analyze a set of quantitative classroom teaching indicators and generate a professional "Multidimensional Diagnosis and Improvement Report on Teacher Classroom Teaching Quality".

[0247] Input data (quantitative scores)

[0248] Please analyze the rating data for the following 7 dimensions (each with a maximum score of 100 points):

[0249] 1. Behavioral and Interaction Dimension:

[0250] - Classroom behavior rating ( ): { 、 、 、 }

[0251] (This indicator reflects the percentage of positive teaching behaviors by teachers, as well as the deduction of points for negative behaviors such as using mobile phones.)

[0252] 2. Content and Structure Dimensions:

[0253] - Scoring of the lesson preparation and design stage ( ): { }

[0254] (Evaluate whether the time allocation for reviewing prior knowledge and introducing new lessons is reasonable)

[0255] - Scoring for the teaching summary and extension session ( ): { }

[0256] (The duration of the evaluation, summary, and feedback sessions, and the adequacy of teacher-student interaction)

[0257] - Direct teaching saturation ( ): { }

[0258] (Whether the assessment avoids "lecture-style" teaching and whether it leaves enough room for students to explore)

[0259] 3. Voice and Expression Dimension:

[0260] - Speech rate appropriateness score ( ): { }

[0261] (Evaluate whether the information transmission rate is within the cognitive processing limits of students' brains)

[0262] - Pronunciation clarity score ( ): { }

[0263] (Based on vowel space area (VSA) calculation, the fullness of pronunciation and the clarity of articulation are evaluated.)

[0264] - Tone of voice score ): { }

[0265] (Evaluate the range of intonation and rhythm to avoid monotonous, hypnotic speech)

[0266] Analysis instructions

[0267] Based on the scoring data above, please generate a diagnostic report that includes the following three components. Please ensure your tone is objective, professional, and constructive.

[0268] Module 1: Overall Classroom Profile

[0269] - Use concise language to summarize the teacher's overall performance in this lesson.

[0270] - Identify the "highlight dimension" (items with a score ≥ 90) that performed best.

[0271] - Identify the "weaknesses" (items with a score < 80) that urgently need improvement.

[0272] Module 2: Submodal Deep Diagnosis

[0273] Please conduct an attribution analysis combining data and theory:

[0274] -Regarding "teacher behavior": If If the score is low, please analyze whether it is due to triggering negative behavior penalties or insufficient patrol / interaction.

[0275] -Regarding "Teaching Content":

[0276] - if If the answer is low, please cite the *Rosenshine principle*, emphasizing the importance of "activating prior knowledge".

[0277] - if If the threshold is low, please cite the *Flanders Interaction Analysis Rules*, emphasizing the addition of "indirect impact" and feedback time.

[0278] - if Low, please warn of the drawbacks of "cramming" teaching.

[0279] -Regarding "Teacher's Voice":

[0280] - if If there is an anomaly, please refer to the *cognitive load theory* (too fast leads to overload, too slow leads to disorganization).

[0281] - if or If the articulation is low, please suggest articulation exercises or increasing emotional investment.

[0282] Module 3: Improvement Suggestions

[0283] - Provide 3 specific and actionable improvement suggestions.

[0284] - Format requirements: **"Problem identification + theoretical basis + specific actions to be taken"**.

[0285] Example: "To address the issue of speaking too fast, it is recommended to consciously pause for 2 seconds when explaining core concepts to give students a window for cognitive processing."

[0286] Output format requirements

[0287] - Do not output the original score repeatedly.

[0288] - Strictly follow Markdown formatting for heading hierarchy (using ####).

[0289] - The report should demonstrate "data-driven" professionalism.

[0290] S62, the result obtained from step S51 The result obtained from step S51 、 、 The result obtained from step S53 、 、 The result obtained from step S25 、 、 、 The result obtained from step S35 、 、 The results obtained from steps S41, S42, and S43 、 、 Input the corresponding slot in the dedicated prompt template to form the complete input text;

[0291] S63. Under the constraints of this dedicated prompt template, a general large language model is selected. Based on this input, and combined with its pre-trained semantic understanding capabilities, the large language model generates a classroom teaching quality evaluation report for the lesson to be evaluated, including three parts: an overall portrait of the classroom teaching, in-depth diagnosis from different perspectives, and targeted improvement suggestions.

[0292] Overall Classroom Teaching Profile: A summary text describing the teacher's overall performance in this lesson;

[0293] Multi-faceted in-depth diagnosis: The teaching quality is evaluated from three perspectives: teacher's teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone, and traceable attribution analysis is conducted.

[0294] Targeted improvement suggestions: Based on traceable causes, specific and actionable teaching adjustment suggestions are made regarding teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone.

[0295] Figure 3 This paper presents a classroom teaching quality evaluation report generated using the method proposed in this invention for a lesson taught by one of the teachers listed in Table 3. The report characterizes the overall classroom as "highly standardized and robust," and provides a granular explanation of the match between the standardization of teaching implementation and the emotional impact across three modalities: teacher behavior, teaching content, and vocal performance. Specifically, the report diagnoses the root cause of the lagging score in intonation engagement compared to vocal clarity: auditory fatigue due to a lack of fundamental frequency variation. Notably, the method proposed in this invention allows for in-depth reasoning based on authoritative theoretical frameworks such as the Rosenshine teaching principle and Flanders interaction analysis, ensuring the scientific validity and pathological basis of the evaluation conclusions. More importantly, the report transforms complex theoretical diagnoses into targeted and specific actionable steps; these suggestions directly correspond to the deep-seated interaction deficiencies revealed by quantitative analysis. Therefore, this report not only provides teaching and research management with objective evidence that transcends subjective evaluation but also offers teachers precise guidance and feedback for advancing from "standardized classrooms" to "intelligent classrooms."

[0296] like Figure 4 As shown, this invention proposes an intelligent evaluation system for teacher classroom teaching quality based on multimodal representation, used to implement an intelligent evaluation method for teacher classroom teaching quality based on multimodal representation. The system includes:

[0297] The multimodal data acquisition and preprocessing module is used to acquire multimodal data of teachers in the classroom, including teaching videos, teaching audio, and teaching courseware, and to preprocess the data according to different modalities.

[0298] The classroom behavior analysis module is used to build and train a teacher classroom behavior classification network. The teaching video of a lesson to be evaluated is input into the trained network for preliminary judgment and correction, and finally the percentage of the four types of classroom behaviors in the lesson is obtained.

[0299] The teaching segment analysis module is used to extract three types of typical teaching segment vocabulary from the teaching courseware, and to determine the teaching segment type of each sentence based on the text obtained from the transcription of the teacher's speech, and finally obtain the percentage index of the three types of teaching segments in the lesson to be evaluated.

[0300] The lecture audio analysis module is used to extract the teacher's speaking speed, pronunciation clarity, and intonation from the lecture audio.

[0301] The intelligent scoring module establishes a mapping relationship based on the teacher's classroom behavior ratio, teaching segment ratio, and three acoustic indicators, under the guidance of educational evaluation theory, to obtain the corresponding evaluation scores for each indicator.

[0302] The report generation module, guided by the prompt word engineering, generates a classroom teaching quality evaluation report for the lesson to be evaluated based on the calculated indicators and their corresponding evaluation scores using a large language model.

[0303] The present invention also provides a computer-readable medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described thereon for intelligent evaluation of the quality of teachers' classroom teaching.

[0304] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method for intelligent evaluation of the quality of teachers' classroom teaching.

[0305] Those skilled in the art will understand that the technical features described in the embodiments and claims of this invention, even if not explicitly described in the invention document, can still be combined and integrated in various forms without departing from the core ideas and principles of this application. All such combinations and integrations should be considered to fall within the protection scope covered by this application. Therefore, the protection scope of this invention is not limited to the embodiments listed in the specification, but should be defined by the appended claims and their legally equivalent content. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for intelligent evaluation of teacher classroom teaching quality based on multimodal representation, characterized in that, The method includes the following steps: S1: Acquire multimodal data related to teachers' classroom teaching, including lecture videos, lecture audio, and lecture courseware, and preprocess the data for different modalities; S2: For the preprocessed lecture videos, firstly, a teacher classroom behavior classification network with a channel attention mechanism and a feature projection layer is constructed for the training dataset; the network is trained using a composite loss function based on Focal Loss and Center Loss to obtain the feature weight parameters, classification mapping parameters, and class center feature vectors in the network; then, based on the trained network, the classroom behavior categories of each segment in the teacher's lecture video of a certain lesson to be evaluated are initially judged and corrected; finally, the proportion index of the four types of classroom behaviors in the lesson is obtained. S3: For the preprocessed teaching materials, extract the nouns to form a terminology set, and then construct a vocabulary database of three typical teaching links. At the same time, for the text obtained by transcribing the teacher's speech, determine the type of teaching link to which each sentence belongs, and finally calculate the proportion of the three types of teaching links in the lesson to be evaluated. S4: For the preprocessed lecture audio, calculate the three acoustic indicators of the teacher's speech rate, pronunciation clarity, and intonation in the lesson to be evaluated. S5: Based on the teacher classroom behavior ratio index, the teaching process ratio index, and three acoustic indicators, a mapping relationship is established under the guidance of educational evaluation theory to obtain the corresponding evaluation scores for each index. S6: Based on the calculated indicators and their corresponding evaluation scores, and guided by the prompt word project, the large language model generates a classroom teaching quality evaluation report for the lesson to be evaluated.

2. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S1 is as follows: S11: Acquire and preprocess the lecture video: Acquire the teacher's lecture video from a camera facing the podium in the classroom; first, segment the video using a fixed-duration sliding window, and use an object detection tool to detect each segment frame by frame, identifying the area where the teacher is located, and obtaining the corresponding image frame sequence; then, perform alignment and normalization operations on each image frame sequence, and input it into a SlowFast dual-path network to extract high-dimensional spatiotemporal feature vectors containing spatial semantics and motion details. All Composition of feature vector set, using Let m represent the number of video segments; finally, based on typical classroom behaviors, including lecturing, blackboard writing, patrolling, and using mobile phones, for each feature vector in the feature vector set... Label it to obtain its corresponding behavior category label. To form a tag set Thus, a training sample set is constructed. This data is used to train a neural network model for subsequent classification of teacher classroom behavior. Simultaneously, for the teacher's teaching video of a specific lesson to be evaluated, the same processing method is applied to obtain the corresponding feature vector set. ; S12: Acquire and preprocess the lecture audio: The lecture audio is separated from the lecture video; For the lecture audio of a certain lesson to be evaluated, the audio endpoint detection algorithm is first used to scan it, and then based on the short-time energy and zero-crossing rate characteristics of the audio, the background noise and silence gaps are determined and removed, only the effective audio segments carrying semantic information are retained, and the start and end times of each effective audio segment are recorded. S13: Obtain and preprocess the teaching materials: The teaching materials are provided by the teacher in advance or extracted from the teaching video; for the teaching materials of a certain lesson to be evaluated, convert the format to text, remove the garbled characters generated during the format conversion process, and use the stop word list to filter high-frequency non-semantic words, retaining only the text content with substantive semantics.

3. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S2 is as follows: S21: Construct a teacher classroom behavior classification network, which includes a channel attention mechanism layer, a feature projection layer, and a classification layer; specifically, the training sample set is first... Each of them The channel attention mechanism layer is formed by sequentially passing through a first unbiased fully connected layer, a ReLU activation function, and a second unbiased fully connected layer. ， in, For the generated channel weight vector, and The parameters to be estimated are... The activation function is Sigmoid; then, and Element-wise multiplication is performed, and the result is passed sequentially through a fully connected layer, a batch normalization layer, a ReLU layer, and a Dropout layer. These layers form the feature projection layer, and the output is a logical vector corresponding to each behavior category. Ultimately, By classification mapping parameters The classification layer, combined with the Softmax function, outputs confidence scores for four categories of classroom behaviors: lecturing (c=1), blackboard writing (c=2), classroom observation (c=3), and using mobile phones (c=4). : c represents the various behavior categories. This represents the logical score of the i-th sample in the training sample set corresponding to the c-th type of classroom behavior; S22: Based on the composite loss function, train the teacher classroom behavior classification network to determine the optimal parameter values; specifically, on the one hand, in order to solve... To address the issue of imbalanced samples across different categories, Focal Loss is employed. in, For focusing parameters; on the other hand, in order to make Samples belonging to the same category should be as close as possible in the feature space, while samples belonging to different categories should be as far apart as possible. Center Loss is introduced to provide auxiliary supervision constraints in the feature space. in, For category The central feature vector; The final joint loss function is: in, To balance the hyperparameters of the two loss terms, the Adam optimizer is used based on the error backpropagation method. Iterative optimization is performed by minimizing the composite loss function to obtain the feature weight parameters in the teacher classroom behavior classification network. and Classification mapping parameters and category center feature vector Thus, the training is completed; S23: The feature vector set corresponding to the teacher's teaching video of the lesson to be evaluated. Input the trained network into the network to obtain the confidence score. The preliminary judgment results, based on the highest confidence level, determined the classroom behavior category to which each video segment belonged. ,in for S24: Yes Statistical smoothing was performed to further refine the data; specifically, for the t-th segment of the video from this lesson, the refined classroom behavior categories were... The category that appears most frequently in the preliminary category judgment results for itself, the k segments before it, and the k segments after it is determined. in, This is an operation to find the mode (the value that appears most frequently in the data); S25: Merge the corrected video clips with continuous and identical classroom behavior categories, record the precise start and end times of each behavior event in the lesson, and finally, sum up all time periods for each of the four behaviors—lecturing, writing on the blackboard, patrolling, and using a mobile phone—in the lesson to obtain the cumulative duration of each of the four classroom behaviors in one lesson. 、 、 、 This means dividing by the total duration of the lesson. ,get: in, 、 、 The indicator is the percentage of positive teaching behaviors (lecturing, blackboard writing, and classroom observation). The percentage of negative teaching behaviors (playing on mobile phones) in the total duration of a class.

4. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S3 is as follows: S31: Based on the word segmentation tool, the preprocessed text content of a lesson's teaching materials is sequentially segmented, deduplicated, and tagged with parts of speech to select nouns and form a terminology set. ; S32: Assume that there are three typical teaching segments in a lesson, namely, instructional preparation and design. Teaching Implementation and Interaction Teaching Summary and Extension ; respectively, a general feature word database is pre-set for the above three types of teaching links, among which, The vocabulary includes: {goal, today we, this lesson, class, hello everyone, open, textbook, page number, book, last lesson, yesterday's test, summary, this question, review, summarize, homework, practice, test, paper, class dismissed, preview, after class, next lesson, copy down, organize, understand, key points, remember, review, recall, remember, this lesson, today, start, prepare}, The lexicon includes: { Definition, Techniques, Who wants to try, Why, For example, Meaning, Attention, Look at the blackboard, Look here, Raise your hand, Who wants to answer, Discussion, Try it, Understand, Comprehend, Observe, Continue} The vocabulary includes: {test, summary, this question, explanation, summary, homework, practice, quiz, test, class dismissal, preview, after class, next class, copy down, organize, understand, key points, remember}; S33: Input the preprocessed effective speech segments of this lesson into the streaming automatic speech recognition model for transcription, and output the transcribed text of the teacher's speech in this lesson. ; S34: Yes Each sentence in the text is compared with... The keywords are matched; if the sentence matches one of the thesaurus entries, then the sentence is classified as belonging to a certain category. The sentence is marked with the corresponding thesaurus name. If the sentence does not match any of the three thesauruses, then... Mark as empty; S35: Preparation and Design of Statistics Teaching Teaching Implementation and Interaction Teaching Summary and Extension The number of successfully matched sentences in the corresponding steps are as follows: ,and The total number of sentences in the text is The percentage indicators of the above three types of teaching links in this lesson are respectively ， ， 。 5. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S4 is as follows: S41: Calculate the teacher's speaking rate based on the number of phonemes: Input all valid speech segments of a pre-processed lesson into a phoneme recognition tool to identify the phonemes; then, sum the number of phonemes contained in all valid speech segments in the lesson and divide by the total duration of all valid speech segments to obtain the phoneme-level speaking rate. , in, The number of effective audio segments in a lesson. Let be the number of phonemes contained in the i-th valid speech segment. This is the total duration of all valid audio segments in this lesson; S42: Calculate the pronunciation clarity of the teacher's lecture based on the vowel space area: First, select vowel phonemes from all phonemes identified in S41; then, calculate the first formant frequency of each vowel phoneme based on the linear predictive coding method. Second resonance frequency Using the standard formant reference values ​​of the three vowel phonemes / i / , / u / , and / a / as the initial values ​​for clustering, the K-Means algorithm was used to cluster all vowel phonemes into three classes. Then, the average formant value of the vowel phonemes in each class c was calculated. in, and respectively For class / i / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / u / the first The first and second formant frequencies of each vowel phoneme. This represents the total number of samples for this type of vowel phoneme. and respectively For class / a / the first The first and second formant frequencies of each vowel phoneme. The total number of samples for this type of vowel phoneme is given; finally, the area of ​​the vowel space is calculated using the geometric determinant formula. As an indicator of the clarity of a teacher's pronunciation. , ; S43: Determine the teacher's tone of voice based on the slope of the fundamental frequency at the end of the sentence: First, set the threshold. For preprocessed valid speech segments and Silence duration between Perform a judgment; if If the silence is determined to be an intra-sentence pause, the preceding and following segments are merged into a single sentence block; if If the silence is identified as a sentence boundary, the video is divided into two independent segments; the audio is then divided into a set of independent segments using this mechanism. ; Next, for each chunk Lock its termination time The cut length is end window audio clips ; Then, based on the autocorrelation method, we obtain The fundamental frequency change sequence of each sampling point in the data Where n is the number of sampling points, and is The product of the speech sampling frequency; Next, the least squares method is used to process the sequence. Perform linear regression fitting and calculate the slope of the fitted line. , in for The average value, for The average value of each element in the sequence; The sign and magnitude of the value are used as quantitative indicators to judge the teacher's tone of voice. If it exceeds 20 Hz / s, then it is judged To rise in pitch, corresponding to teacher questioning or heuristic teaching, when If it is below -20 Hz / s, then it is judged A falling tone corresponds to a statement or instruction; a tone in between is considered a level tone. The intonation determination results of each language chunk in a lesson are statistically analyzed to obtain the percentage of rising, falling, and level intonation. Finally, the proportion of rising and falling intonation durations in the total lesson duration is obtained, i.e., the dynamic intonation proportion. express.

6. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S5 is as follows: S51: For each proportion of teacher classroom behavior, construct corresponding mapping rules to obtain corresponding evaluation scores; specifically, for positive teaching behaviors of teachers (lecturing c=1, blackboard writing c=2, classroom observation c=3), preset the target threshold and base score, and calculate the teacher's positive behavior score. in, This is the base score for positive teaching behavior of type c, which satisfies 00; The threshold for achieving the standard for positive teaching behavior of type c is [those that meet the following criteria]. ; The percentage of positive teaching behaviors of type c in the total duration of a lesson; For negative teaching behaviors of teachers (such as using mobile phones), design a punishment function for negative teaching behaviors: set multi-level tolerance thresholds for duration. Used to filter accidental touches or for emergency viewing; circuit breaker threshold. This represents a serious teaching accident: in The percentage of negative teaching behaviors in the total duration of a lesson; The final teacher classroom behavior scoring function combines positive teaching behavior scores and negative teaching behavior penalty functions. The following is a limit on the score. Within the interval: ； S52: For the three types of teaching segment percentage indicators, construct corresponding mapping rules to obtain corresponding evaluation scores; specifically as follows, for teaching preparation and design... In this process, guided by Rosenshine's effective teaching principles, its mapping rules are designed as follows: in for The percentage of each link in the process for Evaluation scores for each stage; For teaching implementation and interaction In this phase, guided by the "two-thirds rule" of the Flanders interactive analytics system, it is necessary to comprehensively consider its sum. The impact of the combined effects of various stages on teaching quality is mapped by the following rules: in for The percentage of each link in the process for Evaluation scores for each stage; For teaching summary and extension In this phase, still guided by the "two-thirds rule" of the Flanders interactive analytics system, the mapping rule is designed as follows: in for The percentage of each link in the process for Evaluation scores for each stage; Using the above rules, the result obtained in step S35 、 、 Mapping separately, we get 、 、 ; S53: For the three acoustic indicators, corresponding mapping rules are constructed to obtain the corresponding evaluation scores; specifically, for the speech rate indicator, based on the principle that the speech rate of human natural language is usually stable at 10 to 15 phonemes per second, the mapping rule is designed as follows: in As an indicator of speech rate, Speech rate is used to evaluate the score; For pronunciation clarity metrics, the average clarity of normal and clear lecture speech is typically around [value missing]. ( Within this range, the mapping rule for this conclusion is designed as follows: in As an indicator of speech rate, A score is given to evaluate pronunciation clarity; Regarding the intonation indicator, based on the conclusion that varied intonation is a key element in maintaining students' classroom attention, its mapping rule is designed as follows: in For the proportion of dynamic intonation, Score the tone of voice. Based on the above rules, the results obtained in steps S41, S42, and S43 will be... 、 、 Mapping separately, we get 、 、 .

7. The intelligent evaluation method for teacher classroom teaching quality based on multimodal representation according to claim 1, characterized in that, Step S6 is as follows: S61, Construct a dedicated prompt template for generating evaluation reports; this template, as a structured evaluation framework, guides the large language model in simulating the process of teaching supervisors providing teaching quality evaluations, and specifically includes the following key components: Role Definition: The course domain big language model is assigned the professional role of a senior teaching supervisor of the course to be evaluated, and is required to generate an evaluation report that includes conclusions and targeted recommendations; Input data: Input data includes 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 、 ; Analysis principles: The evaluation analysis must be strictly based on the above-mentioned input data, and the evaluation report must be written in an objective, professional, and constructive tone. Output content and format requirements: In terms of content, the generated report must include three components: an overall picture of classroom teaching, in-depth diagnosis from multiple perspectives, and targeted improvement suggestions; in terms of format, it must strictly follow the Markdown format for heading hierarchy. S62, the result obtained from step S51 The result obtained from step S51 、 、 The result obtained from step S53 、 、 The result obtained from step S25 、 、 、 The result obtained from step S35 、 、 The results obtained from steps S41, S42, and S43 、 、 Enter this special prompt template; S63, under the constraints of this dedicated prompt template, select a general large language model to simulate the evaluation process of teaching supervision, and generate a classroom teaching quality evaluation report for the lesson to be evaluated, including three parts: an overall portrait of classroom teaching, in-depth diagnosis from multiple perspectives, and targeted improvement suggestions. Overall Classroom Teaching Profile: A summary text describing the teacher's overall performance in this lesson; Multi-faceted in-depth diagnosis: The teaching quality is evaluated from three perspectives: teacher's teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone, and traceable attribution analysis is conducted. Targeted improvement suggestions: Based on traceable causes, specific and actionable teaching adjustment suggestions are made regarding teaching behavior, teaching process design, and teaching speed / pronunciation clarity / tone.

8. A teacher classroom teaching quality intelligent evaluation system based on multimodal representation, used to implement the teacher classroom teaching quality intelligent evaluation method based on multimodal representation as described in any one of claims 1-7, characterized in that, The system includes: The multimodal data acquisition and preprocessing module is used to acquire multimodal data of teachers in the classroom, including teaching videos, teaching audio, and teaching courseware, and to preprocess different modal data. The classroom behavior analysis module is used to build and train a teacher classroom behavior classification network. The teaching video of a lesson to be evaluated is input into the trained network for preliminary judgment and correction, and finally the percentage of the four types of classroom behaviors in the lesson is obtained. The teaching segment analysis module is used to extract three types of typical teaching segment vocabulary from the teaching courseware, and to determine the teaching segment type of each sentence based on the text obtained from the transcription of the teacher's speech, and finally obtain the percentage index of the three types of teaching segments in the lesson to be evaluated. The lecture audio analysis module is used to extract the teacher's speaking speed, pronunciation clarity, and intonation from the lecture audio. The intelligent scoring module establishes a mapping relationship based on the teacher's classroom behavior ratio, teaching segment ratio, and three acoustic indicators, under the guidance of educational evaluation theory, to obtain the corresponding evaluation scores for each indicator. The report generation module, guided by the prompt word engineering, generates a classroom teaching quality evaluation report for the lesson to be evaluated based on the calculated indicators and their corresponding evaluation scores using a large language model.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method of any one of claims 1-7 to intelligently evaluate the quality of teachers' classroom teaching.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements any one of the methods in claims 1-7 to intelligently evaluate the quality of teachers' classroom teaching.

Images