Classroom interaction quality assessment and teaching reflection system based on speech emotion recognition

Abstract

This invention discloses a classroom interaction quality assessment and teaching reflection system based on voice emotion recognition, belonging to the field of educational informatization technology. It includes a voice data acquisition module, a voice data preprocessing module, a voice emotion recognition module, a voice data segmentation module, an interaction depth quantification module, a classroom interaction quality index calculation module, an early warning signal generation module, and a teaching reflection auxiliary module. This invention constructs a multi-level classroom interaction quality assessment system. Through the identification of teaching links and a dynamic weight adjustment mechanism, it significantly improves the scientific nature of the assessment. Based on the current interaction state, it predicts future quality change trends and combines a dual-indicator early warning mechanism of declining slope and minimum point to support teachers in timely adjustments to teaching strategies during class. It also introduces counterfactual reasoning technology to conduct multi-dimensional causal contribution analysis of weak teaching links, improving the scientific nature and operability of reflection.

Description

Technical Field

[0001] This invention relates to the field of educational informatization technology, specifically to a classroom interaction quality assessment and teaching reflection system based on voice emotion recognition. Background Technology

[0002] With the deepening of educational informatization, intelligent assessment of classroom teaching quality has become one of the hot topics in educational technology research.

[0003] Traditional classroom observation and assessment methods largely rely on manual listening, observation records, or post-class questionnaires. In recent years, the development of speech recognition and affective computing technologies has provided new possibilities for classroom interaction analysis. By collecting speech data in the classroom, information such as the turn-by-turn dialogue, content, and emotional state of teacher-student interactions can be identified, thereby analyzing key teaching elements such as classroom atmosphere, student participation, and level of thinking. However, existing technologies still have the following limitations:

[0004] Analysis limited to a single dimension is insufficient to fully reflect the complexity of classroom interaction;

[0005] The evaluation method uses fixed weights, ignoring the differences in the requirements for interactive elements in different teaching stages;

[0006] Existing systems mostly perform post-event analysis and lack the ability to predict classroom development trends;

[0007] Teaching reflection often relies on teachers' subjective experience, making it difficult to quantify the specific impact of various teaching behaviors on classroom effectiveness.

[0008] Therefore, a method is needed to solve the above problems, which includes multi-dimensional integrated assessment, dynamic adjustment of teaching stage weights, real-time prediction and early warning, and teaching reflection based on causal reasoning. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a classroom interaction quality assessment and teaching reflection system based on voice emotion recognition.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows:

[0011] A classroom interaction quality assessment and teaching reflection system based on voice emotion recognition includes:

[0012] Voice data acquisition module: used to acquire raw voice data of classroom interaction collected synchronously from multiple channels, the raw voice data including sound source location information;

[0013] The voice data preprocessing module is used to preprocess the raw voice data, classify and label roles based on the sound source location information, output standardized voice data, and calculate the classroom enthusiasm coefficient.

[0014] Speech emotion recognition module: used to extract acoustic features from standardized speech data, use an attention-based speech emotion recognition network to perform emotion recognition, output emotion classification results and calculate classroom emotional atmosphere coefficient;

[0015] Voice data segmentation module: used to segment dialogue turns, silence types and teaching segments according to the standardized voice data, output silence types and corresponding time periods and calculate silence quality coefficient, output teaching segments and corresponding teacher speech duration ratio and calculate teacher-student speech power balance coefficient;

[0016] Interaction Depth Quantification Module: Used to identify thinking characteristics in the standardized speech data, output the proportion of dialogue rounds with different thinking patterns, and calculate the classroom interaction depth coefficient;

[0017] Classroom Interaction Quality Index Calculation Module: This module uses adaptive weighting of teaching segments and combines classroom enthusiasm coefficient, classroom emotional atmosphere coefficient, silence quality coefficient, teacher-student discourse power balance coefficient, and classroom interaction depth coefficient to calculate the classroom interaction quality index. When either the interaction depth coefficient or the emotional atmosphere coefficient is lower than a preset threshold, the quality is reset to zero.

[0018] Early warning signal generation module: used to build a classroom interaction state prediction model, predict future interaction state coefficients based on the current interaction state coefficients and calculate the classroom interaction prediction quality index and its trend index. If the trend index exceeds a preset threshold, an early warning signal and suggested intervention measures are generated.

[0019] The teaching reflection auxiliary module is used to calculate the deviation of the classroom interaction quality index in different teaching stages, extract the actual values ​​of each teaching behavior dimension in each stage, perform counterfactual reasoning based on the neurocausal model, calculate the causal contribution of different teaching behavior dimensions to the classroom interaction quality index, and output the attribution analysis results.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] 1. This invention constructs a multimodal and multi-level classroom interaction quality assessment system by integrating five dimensions: classroom enthusiasm coefficient, emotional atmosphere coefficient, silence quality coefficient, teacher-student discourse power balance coefficient, and interaction depth coefficient, thereby achieving a comprehensive quantification of classroom interaction from emotion, cognition, behavior to atmosphere.

[0022] 2. This invention uses a teaching process identification and dynamic weight adjustment mechanism, combined with the Delphi method of education experts to construct a weight matrix, making the quality index calculation more in line with teaching practice and significantly improving the scientific nature and pertinence of the assessment.

[0023] 3. This invention constructs a time-series state prediction model, which can predict future quality change trends based on the current interaction state. Combined with a dual-indicator early warning mechanism of the descent slope and the lowest point, it can achieve three levels of early warning: mild, moderate and severe. This allows teachers to adjust their teaching strategies in a timely manner during class to prevent the quality problems from worsening.

[0024] 4. This invention introduces a neural causal model and counterfactual reasoning technology, which can conduct multi-dimensional causal contribution analysis on weak teaching links, quantify the impact of teaching behavior on interaction quality, output interpretable attribution results, provide teachers with data-driven teaching improvement suggestions, and enhance the scientific nature and operability of reflection. Attached Figure Description

[0025] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0026] Figure 1 This is a schematic diagram of the structure of the present invention;

[0027] Figure 2 This is a step diagram of the present invention;

[0028] Figure 3 This is the early warning determination diagram of the present invention. Detailed Implementation

[0029] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0030] like Figure 1 As shown, the classroom interaction quality assessment and teaching reflection system based on voice emotion recognition includes a voice data acquisition module, a voice data preprocessing module, a voice emotion recognition module, an interaction depth quantification module, a classroom interaction quality index calculation module, a visualization module, and a teaching reflection auxiliary module.

[0031] like Figure 2 As shown, the specific implementation steps of the present invention include the following steps:

[0032] S1. Acquire raw classroom interactive audio data synchronously collected through a voice data acquisition module. The raw audio data includes sound source location information.

[0033] It should be specifically noted that in a standard classroom, n high-sensitivity microphones are deployed on both sides of the blackboard on the front wall, the two side walls, and the back wall to form a distributed microphone array. The distance between each microphone is m meters, where m ranges from 2 to 3 meters. The number of n is determined according to the length and width of the classroom to ensure sound field coverage of the classroom.

[0034] All microphones are connected to an edge computing host deployed at the back of the classroom via audio interfaces. The edge computing host is responsible for real-time voice acquisition and processing, and connects to a cloud server through the campus network to realize data storage and model updates. Teachers' mobile terminals are equipped with a supporting application to receive real-time visual dashboards and push notifications for post-class reflections.

[0035] It should be specifically noted that the voice data acquisition module adopts a high-precision clock synchronization protocol to ensure synchronized acquisition time of n microphone channels. The acquired raw data is buffered in a circular buffer, supporting real-time streaming processing and retrospective analysis. Based on time-of-arrival (TOA) sound source localization technology, it tracks the sound source location in real time to distinguish between teacher and student sound sources. Specifically:

[0036] The distance from the sound source to two different microphones is different, and the arrival time is different. The time difference multiplied by the propagation speed is the distance difference between the sound source and the two microphones. The time difference between a pair of microphones can determine that the sound source is located on a certain hyperbola. A third microphone can obtain another hyperbola. Multiple microphones can obtain multiple hyperbolas. The intersection of multiple hyperbolas is the precise location of the sound source.

[0037] S2. The original voice data is preprocessed by the voice data preprocessing module, and role classification and labeling are performed in combination with the sound source location information. Standardized voice data is output and the classroom enthusiasm coefficient is calculated.

[0038] It should be specifically noted that the preprocessing includes data cleaning and timestamp alignment. The data cleaning specifically includes noise reduction, echo cancellation, and speech activity detection. The noise reduction uses spectral subtraction combined with Wiener filtering to suppress stationary noise and improve the signal-to-noise ratio. Echo cancellation uses an adaptive filtering algorithm to eliminate linear echoes, which may exist in the classroom's sound amplification system. Speech activity detection is based on energy thresholds, zero-crossing rate, and spectral features, combined with neural networks, to accurately identify speech segments and non-speech segments. The timestamp alignment specifically involves adding a uniform timestamp to all speech segments that pass the speech activity detection, including the start time, end time, and duration.

[0039] The specific steps for classifying and labeling roles based on the sound source location information are as follows:

[0040] The sound source location is consistently located in the podium area, i.e. within a certain distance of the front wall, and in the non-student seating area of ​​the classroom. The voiceprint model matches the historical teacher's voiceprint, indicating a teacher role. The sound source location is located in the student seating area, and the voiceprint model does not match the historical classroom voiceprint, indicating an individual student role. Multiple channels detect overlapping speech with comparable energy, and the sound source locations are dispersed, indicating a collective role for the whole class.

[0041] The standardized voice data specifically includes complete information such as voice waveform data, start timestamp, end timestamp, role label, and sound source location coordinates.

[0042] The classroom engagement coefficient Dj is specifically the average of the ratio of the number of students who speak within each time window to the total number of students in the class, and the ratio of the number of times students speak voluntarily to the total number of times students speak when no teacher calls on them.

[0043] S3. The acoustic features of standardized speech data are extracted through the speech emotion recognition module, and the speech emotion recognition network based on the attention mechanism is used to perform emotion recognition, output the emotion classification results and calculate the classroom emotional atmosphere coefficient.

[0044] It should be specifically noted that the acoustic features include prosodic features, spectral features, and phonological features; among which, prosodic features are the most direct carrier of emotional expression, reflecting the tone and rhythm of speech, including fundamental frequency, energy, speech rate and pauses, and zero-crossing rate; among which, spectral features reflect the shape of the vocal tract and the mode of sound production, obtained through short-time Fourier transform or filter banks, including Mel frequency cepstral coefficients, Mel spectrogram, linear predictive coding coefficients, and spectral centroid, bandwidth, and roll-off point; among which, phonological features include jitter and flicker, as well as harmonic noise ratio.

[0045] The speech emotion recognition network based on the attention mechanism identifies and outputs the emotion state recognition result. The specific steps are as follows:

[0046] A1. Perform feature encoding;

[0047] The extracted acoustic features are input into the encoder, and the backbone network outputs a set of hidden state sequences containing contextual information. Each of them This represents the acoustic characteristics at time t;

[0048] A2. Calculate attention weights;

[0049] A learnable vector is generated from the global context, representing the emotional cues to be sought, and the calculation is performed at each time step. The relevance scores are normalized and transformed into a probability distribution, with a sum of 1.

[0050] A3. Perform context vector aggregation;

[0051] Based on the calculated weights, the hidden state sequence is weighted and summed to generate a global context vector. This vector focuses on the most emotionally discriminative information in the entire speech and ignores irrelevant sound segments.

[0052] A4. Output sentiment classification and confidence level;

[0053] The context vector is input into the fully connected layer, and the activation function outputs the probability distribution of each sentiment category. The category with the highest probability is taken as the recognition result, and the probability of this category is the confidence level.

[0054] The classroom emotional atmosphere coefficient Specifically:

[0055] ;

[0056] in The classroom emotional atmosphere coefficient measures the collective emotional state of students during the classroom teaching process. The total duration of positive emotional speech segments. The total duration of neutral emotional speech segments. This represents the total duration of the negative speech segment. The total voice duration, of which ; The value is a neutral sentiment weight, ranging from 0.3 to 0.7, with an optimal value of 0.5. The negative emotion penalty coefficient ranges from 0.8 to 1.2, with an optimal value of 1.0.

[0057] S4. The speech data segmentation module divides the dialogue rounds, silence types and teaching segments according to the standardized speech data, outputs the silence type and corresponding time period and calculates the silence quality coefficient, outputs the teaching segment and the corresponding teacher speech duration ratio and calculates the teacher-student speech power balance coefficient.

[0058] It should be specifically explained that the dialogue round division rule is as follows: based on role tags and timestamps, speaker switching events are detected. When the role tag of the current voice segment is different from that of the previous voice segment or the role is the same but the interval time is greater than the first set value, it is recorded as a new dialogue round. Each round is recorded, including round number, start time, end time and voice segment ID list. If the interval time is greater than the second set value, it is classified according to the context. If there is a question from the teacher, it is a thinking silence; if there is no clear guidance, it is an awkward silence; silence during the transition between segments is a transitional silence. The second set value is greater than the first set value.

[0059] The silent quality coefficient Specifically:

[0060] ;

[0061] in The silent quality coefficient measures the teaching value of silent periods in the classroom. For the duration of thoughtful silence, The duration of silence in awkward silences. This is a penalty coefficient, with a value > 1, indicating that the negative effect of awkward silence outweighs the positive effect of thoughtful silence.

[0062] It should be noted that, based on the characteristics of classroom speech flow, a Hidden Markov Model is used to automatically segment teaching segments. The input features include the proportion of teacher speech, the frequency of student speech, the duration of silence, and changes in speech rate. The Hidden Markov Model is trained on 500 labeled classroom data, with the number of states set to 5, corresponding to the five segments: introduction, new teaching, interaction, practice, and summary. The output is the segment label of each time point, as well as the start and end times of the segment.

[0063] The teacher-student power balance coefficient Specifically:

[0064] ;

[0065] in The teacher-student discourse power balance coefficient measures whether the ratio of teacher speaking time to student speaking time in the classroom is reasonable, reflecting the fairness of the distribution of discourse power in classroom interaction and the degree of student participation. The percentage of time teachers spend speaking. The target percentage of teacher discourse is set according to different teaching stages. This refers to the permissible range of fluctuations.

[0066] S5. Identify the thinking characteristics in the standardized voice data through the interaction depth quantization module, output the proportion of dialogue rounds with different thinking, and calculate the classroom interaction depth coefficient.

[0067] An end-to-end speech recognition model is used to transcribe each speech segment, generating timestamped text. Based on the transcribed text, semantic features of thought processes are identified, and thought language is labeled, specifically including:

[0068] Follow-up question marker: To deepen the understanding of the previous answer and request more details or reasons; Explanation marker: To elaborate on concepts, principles, or logical relationships, rather than simply restating them; Questioning marker: To raise objections, corrections, or critical thinking about existing viewpoints, premises, or conclusions; Connection marker: To establish connections between current knowledge points and prior knowledge, real-life situations, or other disciplines; Reflection marker: To reflect on the thinking process itself.

[0069] Mapping thought language markers to different thought processes specifically includes:

[0070] Lower-order thinking: No higher-order markers, simply stating facts or providing simple responses;

[0071] Intermediate thinking: Includes explanatory markers and expanded answers to questions;

[0072] Higher-order thinking includes questioning markers, connection markers, and reflection markers;

[0073] The number and percentage of discourse rounds for each thought process are statistically analyzed to generate thought distribution data, and the percentages of low-order, mid-order, and high-order thought processes are calculated.

[0074] The depth coefficient of classroom interaction Specifically, it is a weighted sum of the proportions of low-order thinking, mid-order thinking, and high-order thinking. The weighting coefficients are set by those skilled in the art based on big data samples and experience, with the weight of high-order thinking > the weight of mid-order thinking > the weight of low-order thinking.

[0075] S6. The classroom interaction quality index calculation module adopts adaptive weighting of teaching links and combines classroom enthusiasm coefficient, classroom emotional atmosphere coefficient, silence quality coefficient, teacher-student discourse power balance coefficient and classroom interaction depth coefficient to calculate the classroom interaction quality index. When either the interaction depth coefficient or the emotional atmosphere coefficient is lower than the preset threshold, the quality is reset to zero.

[0076] The classroom interaction quality index is specifically as follows:

[0077] ;

[0078] Where Z represents the classroom interaction quality index. This is the interaction depth coefficient. Emotional atmosphere coefficient As a classroom engagement coefficient, As a coefficient for balancing the voice of teachers and students, The silent quality coefficient; The weights are used as weighting coefficients. The weights of each dimension are dynamically adjusted based on the identified teaching stages. The weight matrix is ​​determined using the Delphi method by education experts and can be adjusted in the system configuration file.

[0079] When serious problems occur in the classroom, such as a complete lack of cognitive depth or an extremely negative emotional atmosphere, the quality index is directly reduced to zero to prevent high values ​​in other dimensions from masking the core problem. The threshold settings for the interaction depth coefficient and the emotional atmosphere coefficient are determined through statistical analysis of historical data.

[0080] S7. Construct a classroom interaction state prediction model through the early warning signal generation module, predict the future interaction state coefficient based on the current interaction state coefficient, and calculate the classroom interaction prediction quality index and its trend index. If the trend index exceeds the preset threshold, generate an early warning signal and suggested intervention measures.

[0081] It should be specifically noted that the construction steps of the classroom interaction state prediction model are as follows:

[0082] B1. Construct the classroom interaction state space and define the system state vector and control vector;

[0083] Where the state vector is

[0084] Each coefficient is a calculated value at time t, where t is the time window;

[0085] The control vectors can be combined as needed, such as question density, feedback density, average waiting time, and activity switching frequency.

[0086] B2. Perform state transition learning;

[0087] The state transition equation is learned based on historical classroom data. Long short-term memory network is used to model nonlinear state transition. The current state, historical state and control variables are input, and the mean square error between the predicted state and the actual state is minimized. It is updated offline every week and incorporates newly collected classroom data.

[0088] B3. Conduct quality trend prediction;

[0089] Based on the current state Based on the recent control variable sequence, the state sequence for the next k minutes is recursively predicted. For each predicted state and its coefficients, the classroom interaction prediction quality index Zy is calculated using the classroom interaction quality index, generating a quality change trend curve. ;

[0090] B4. Calculate trend indicators, including the downward slope and the lowest point;

[0091] The specific descent slope is: ,in This represents the total change over the next k minutes, measuring the overall trend and rate of change in the classroom interaction quality index over a future period. This indicates an improvement in the quality of classroom interaction. This indicates a decline in classroom quality. The larger the value, the faster the rate of increase or decrease. To predict the quality index of classroom interaction in the k-th minute, Let be the actual quality index at the current time t.

[0092] Different descent slope patterns correspond to different teaching problems: a continuous slow descent may indicate accumulated student fatigue and gradual distraction; a sudden sharp descent may indicate improper switching of teaching segments or a sudden increase in the difficulty of the problem; and a fluctuating descent may indicate unstable teaching strategies and fluctuating student participation.

[0093] The lowest point is specifically: ;in The lowest point indicates that the classroom interaction quality index will reach its minimum value within the predicted future time period. The smaller the minimum point, the greater the quality problems in teaching.

[0094] like Figure 3 As shown, the generation of the early warning signal specifically involves:

[0095] when or This is a mild warning; the quality of classroom interaction is showing a slight downward trend, and there may be intermittent quality issues that require attention.

[0096] when or The alert level is moderate; the quality of classroom interaction is deteriorating, and significant teaching problems may emerge. Intervention is recommended.

[0097] when or This is a severe warning sign, indicating a serious decline in the quality of classroom interaction and the potential for a serious teaching accident, requiring immediate intervention.

[0098] If multiple warning conditions are met simultaneously, only one warning signal will be generated, with the priority being severe warning > moderate warning > mild warning.

[0099] An intervention knowledge base is built, which includes common teaching intervention strategies. Based on the prediction results and the current status, suggested intervention measures are generated through rule matching. The intervention suggestions are output to the visual dashboard and teachers' mobile terminals along with the early warning signal.

[0100] For example, if the decline is predicted to be mainly due to a decrease in emotional atmosphere, it is recommended to increase positive feedback and intersperse engaging content; if the decline is predicted to be mainly due to a decrease in the depth of interaction, it is recommended to prepare follow-up questions to guide students to think deeply; if the decline is predicted to be mainly due to a decrease in classroom enthusiasm, it is recommended to design group discussions or individual speaking sessions.

[0101] After intervention measures are taken, future quality is re-predicted, and new observations are made. Whether it improves, calculate before and after intervention. The difference is used to quantify the intervention effect. The intervention-effect data is stored in a knowledge base to optimize future intervention recommendations.

[0102] S8. Calculate the deviation of the classroom interaction quality index for different teaching stages, extract the actual values ​​of each teaching behavior dimension within each stage, perform counterfactual reasoning based on the neural causal model, calculate the causal contribution of different teaching behavior dimensions to the classroom interaction quality index, and output the attribution analysis results.

[0103] It should be specifically noted that the deviation of the interaction quality index is as follows:

[0104] For each teaching segment, the average of the average classroom interaction quality index at all time points within the current segment is calculated. The difference between the average classroom interaction quality index at all time points in the entire lesson and the average of the classroom interaction quality index at all time points is the deviation of each segment. If the deviation is ≥0, it means that the current segment is higher than or equal to the average classroom level. If the deviation is <0, it means that the current segment is lower than the average classroom level. Segments that are lower than the preset threshold are marked as weak segments and enter in-depth diagnosis.

[0105] Based on educational psychology theories and expert knowledge, a causal graph of teaching behavior dimensions and classroom interaction quality index was constructed. A neural causal model was used to parameterize the causal graph, and the model was trained using data from 500 labeled classroom sessions.

[0106] For the identified weak links, the actual values ​​of each teaching behavior dimension within that link are extracted. For each behavior dimension, a counterfactual value is defined, which is either the teaching suggestion value or the average value of a high-quality classroom. Only one dimension's value is changed, while the other dimensions remain unchanged. The counterfactual classroom interaction quality index is predicted using a causal model, and the difference between the predicted counterfactual classroom interaction quality index and the actual classroom interaction quality index is the causal contribution. To avoid the randomness of a single counterfactual inference, multiple samplings are performed. For each behavior dimension, multiple points are sampled near its counterfactual value, and multiple causal effect values ​​are calculated. The average value is taken as the final contribution, and the standard deviation is calculated as a confidence index. The results are sorted from largest to smallest by the absolute value of the causal contribution, and the attribution analysis results for each teaching behavior dimension are output and sent to the teacher's mobile device.

[0107] It should be noted that the visualization dashboard displays in real time the ratio of teacher and student participation, changes in classroom emotional atmosphere, heat maps of classroom participation areas, quality change trend curves, and early warning signals.

[0108] The real-time dashboard for teacher-student discourse power includes a circular progress bar displaying the current percentage of teacher discourse power, the percentage of individual student speeches, and the percentage of whole-class reading aloud; and a historical trend line showing past changes in the proportion of discourse power in the classroom.

[0109] Classroom emotional atmosphere change chart: stacked area chart, with the horizontal axis representing time and the vertical axis representing the percentage of emotional categories; different emotional categories are distinguished by different colors.

[0110] The classroom participation area distribution heatmap is based on sound source localization data, dividing the classroom plane into grids, with color depth mapping the speaking frequency of each grid.

[0111] The quality change trend curve displays historical and predicted quality curves, with different colors marking prediction intervals. When an alert is triggered, a red marker flashes at the corresponding time point, and alert details and intervention suggestions are displayed on the side.

[0112] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A classroom interaction quality evaluation and teaching reflection system based on voice emotion recognition, characterized in that, include: Voice data acquisition module: used to acquire raw voice data of classroom interaction collected synchronously from multiple channels, the raw voice data including sound source location information; The voice data preprocessing module is used to preprocess the raw voice data, classify and label roles based on the sound source location information, output standardized voice data, and calculate the classroom enthusiasm coefficient. Speech emotion recognition module: used to extract acoustic features from standardized speech data, use an attention-based speech emotion recognition network to perform emotion recognition, output emotion classification results and calculate classroom emotional atmosphere coefficient; Voice data segmentation module: used to segment dialogue turns, silence types and teaching segments according to the standardized voice data, output silence types and corresponding time periods and calculate silence quality coefficient, output teaching segments and corresponding teacher speech duration ratio and calculate teacher-student speech power balance coefficient; Interaction Depth Quantification Module: Used to identify thinking characteristics in the standardized speech data, output the proportion of dialogue rounds with different thinking patterns, and calculate the classroom interaction depth coefficient; Classroom Interaction Quality Index Calculation Module: This module uses adaptive weighting of teaching segments and combines classroom enthusiasm coefficient, classroom emotional atmosphere coefficient, silence quality coefficient, teacher-student discourse power balance coefficient, and classroom interaction depth coefficient to calculate the classroom interaction quality index. When either the interaction depth coefficient or the emotional atmosphere coefficient is lower than a preset threshold, the quality is reset to zero. Early warning signal generation module: used to build a classroom interaction state prediction model, predict future interaction state coefficients based on the current interaction state coefficients and calculate the classroom interaction prediction quality index and its trend index. If the trend index exceeds a preset threshold, an early warning signal and suggested intervention measures are generated. The teaching reflection auxiliary module is used to calculate the deviation of the classroom interaction quality index in different teaching stages, extract the actual values ​​of each teaching behavior dimension in each stage, perform counterfactual reasoning based on the neurocausal model, calculate the causal contribution of different teaching behavior dimensions to the classroom interaction quality index, and output the attribution analysis results.

2. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 1, characterized in that: The standardized voice data specifically includes complete information of voice waveform data, start timestamp, end timestamp, role label, and sound source position coordinates; and the classroom initiative coefficient Specifically, the average value of the ratio of the number of students speaking in each time window to the total number of students in the class and the ratio of the number of times students voluntarily speak to the total number of times students speak without the teacher calling roll.

3. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 1, characterized in that: The classroom emotional atmosphere coefficient Specifically, Among them The classroom emotional atmosphere coefficient, The total duration of positive emotional speech segments, The total duration of neutral emotional speech segments, The total duration of negative speech segments, The total speech duration, wherein ; The neutral emotional weight; The negative emotional penalty coefficient.

4. The system according to claim 1, wherein the system further comprises: The specific rules for dividing dialogue rounds are as follows: Based on role labels and timestamps, speaker switching events are detected. When the role label of the current speech segment is different from that of the previous speech segment, or the role is the same but the interval time is greater than the first set value, it is recorded as a new dialogue round. If the interval time is greater than the second set value, it is classified according to the context. If there is a question from the teacher, it is a thinking silence; if there is no clear guidance, it is an awkward silence; silence during the transition between segments is a transitional silence, where the second set value is greater than the first set value.

5. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 1, characterized in that: the silence quality coefficient Specifically, ; wherein is the silence quality coefficient, is the thinking-type silence duration is the cold-field-type silence duration, is a penalty coefficient, and is greater than 1; The teacher-student discourse power balance coefficient Specifically, ; wherein The teacher-student discourse power balance coefficient, The teacher discourse time length proportion, The target teacher discourse proportion, The allowed fluctuation range; The depth coefficient of classroom interaction Specifically, it is a weighted sum of the proportions of low-order thinking, mid-order thinking, and high-order thinking. The weighting coefficients are set by those skilled in the art based on big data samples and experience, with the weight of high-order thinking > the weight of mid-order thinking > the weight of low-order thinking.

6. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 1, characterized in that: The classroom interaction quality index is specifically as follows: ； in As a classroom interaction quality index, This is the interaction depth coefficient. Emotional atmosphere coefficient As a classroom engagement coefficient, As a coefficient for balancing the voice of teachers and students, The silent quality coefficient; , , , , The weights are used as weighting coefficients, and the weights of each dimension are dynamically adjusted according to the teaching process. The weight matrix is ​​determined by the Delphi method of education experts and can be adjusted in the system configuration file.

7. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 1, characterized in that: The trend indicators include the downward slope and the lowest point; the downward slope is specifically: ,in This represents the total change over the next k minutes. This indicates an improvement in the quality of classroom interaction. This indicates a decline in classroom quality. The larger the value, the faster the rate of increase or decrease. To predict the quality index of classroom interaction in the k-th minute, Let be the classroom interaction quality index at the current time t; the minimum point is specifically: ;in This represents the lowest point, i.e., the minimum value reached by the classroom interaction quality prediction index.

8. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 7, characterized in that: Specifically, an early warning signal is generated when a trend indicator exceeds a preset threshold: When... or A mild warning; when or A moderate warning level is in effect; when or A severe alert has been issued. If multiple warning conditions are met simultaneously, only one warning signal will be generated, with the priority being severe warning > moderate warning > mild warning.

9. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 4, characterized in that: The deviation of the classroom interaction quality index is specifically as follows: For each teaching segment, the average of the average classroom interaction quality index at all time points within the current segment is calculated. The difference between the average classroom interaction quality index at all time points in the entire lesson and the average classroom interaction quality index of the current segment is the deviation of the classroom interaction quality index for each segment. If the deviation is ≥0, it means that the current segment is higher than or equal to the average classroom level. If the deviation is <0, it means that the current segment is lower than the average classroom level. Segments that are lower than the preset threshold are marked as weak segments and enter in-depth diagnosis.

10. The classroom interaction quality assessment and teaching reflection system based on voice emotion recognition according to claim 9, characterized in that: The causal contribution of each teaching behavior dimension to the quality index is calculated as follows: For the identified weak links, the actual values ​​of each teaching behavior dimension within that link are extracted. For each behavior dimension, a counterfactual value is defined, which is either the teaching suggestion value or the average value of a high-quality classroom. Only one dimension's value is changed, while the other dimensions remain unchanged. The counterfactual classroom interaction quality index is predicted using a causal model, and the difference between the counterfactual and actual classroom interaction quality indices is the causal contribution. Multiple sampling is performed, and for each behavior dimension, multiple points are sampled near its counterfactual value to calculate multiple causal effect values. The mean value is taken as the final contribution, and the standard deviation is calculated as the confidence index. The results are sorted from largest to smallest by the absolute value of the causal contribution, and the attribution analysis results for each teaching behavior dimension are output and sent to the teacher's mobile device.

Images