An AI-assisted classroom teaching behavior analysis and feedback method

Abstract

The application discloses an AI-assisted classroom teaching behavior analysis and feedback method, and relates to the technical field of artificial intelligence-assisted education. The method obtains a teacher area video stream, a student area video stream, a panoramic video stream, a teacher voice stream, environment sound field data and environment perception data, and aligns them according to a unified classroom time axis to obtain original multi-modal data of a classroom; generates a teacher teaching behavior feature sequence, a student learning state feature sequence, a classroom atmosphere feature, a teaching behavior score and a learning input degree score; based on the classroom atmosphere feature, interference correction is performed on a time sequence correlation result, and a candidate correlation feature pair is screened; directional time sequence causal analysis and reverse checking are performed on the candidate correlation feature pair to generate a directed weighted causal graph and a key path. The application can reduce false correlations caused by physical environment abnormalities, and improve the causal explainability and closed-loop optimization capability of classroom teaching feedback.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted education technology, specifically to an AI-assisted method for analyzing and providing feedback on classroom teaching behavior. Background Technology

[0002] Smart classrooms and recording studios are typically equipped with cameras, microphone arrays, teaching terminals, and environmental sensors to record video, audio, courseware, and interactive information during classroom teaching. Existing classroom analytics systems can identify student head-up rate, facial orientation, hand-raising behavior, teacher speaking speed, teacher volume, blackboard writing behavior, and interaction frequency, and generate classroom evaluation reports based on this information.

[0003] However, existing classroom analytics solutions still have the following shortcomings. The sampling frequencies for teacher videos, student videos, teacher audio, classroom sound field data, and environmental sensor data differ. Without standardized timeline alignment, changes in teacher behavior and student states may be incorrectly linked to the same analytical segment, leading to an unreliable data foundation for subsequent classroom teaching behavior analysis.

[0004] Existing solutions often directly interpret the synchronous changes between teacher behavior and student learning status as teaching influence. For example, when a teacher's speaking speed increases and students' attention decreases simultaneously, the system tends to suggest reducing the speaking speed. However, in real classrooms, student attention decreases can also be caused by excessively high carbon dioxide concentrations, abnormal temperatures, insufficient lighting, external noise, or partial visual obstruction. If the system does not adjust for the classroom physical environment as a confounding variable, it is easy to incorrectly attribute fluctuations in student status caused by environmental factors to teacher behavior. Existing classroom evaluations mostly remain at the stage of correlation statistics or static scoring, lacking directional temporal verification that points from teacher behavior to student status. Correlation can only indicate that two characteristics change together; it cannot prove that teacher behavior precedes changes in student status in time, nor can it prove that teacher behavior has a predictive gain on student status. The lack of a causal evidence chain reduces the credibility of feedback suggestions.

[0005] Existing classroom evaluation reports are typically generated once after a single lesson, lacking follow-up classroom tracking. The system struggles to determine whether teachers have adopted suggestions, whether student engagement has improved after adopting suggestions, and also finds it difficult to update the attribution analysis model and suggestion generation model based on actual feedback. Summary of the Invention

[0006] The purpose of this invention is to provide an AI-assisted classroom teaching behavior analysis and feedback method to solve the technical problems in existing classroom teaching behavior analysis, such as time mismatch of multimodal data, spurious correlation caused by environmental factors, confusion between correlation and causation, and lack of closed-loop verification of feedback suggestions.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] This invention provides an AI-assisted method for analyzing and providing feedback on classroom teaching behavior, comprising the following steps:

[0009] The system acquires teacher area video stream, student area video stream, panoramic video stream, teacher voice stream, environmental sound field data, and environmental perception data during classroom teaching. It then performs time alignment of the teacher area video stream, student area video stream, panoramic video stream, teacher voice stream, environmental sound field data, and environmental perception data according to a unified classroom timeline to obtain the original multimodal classroom data.

[0010] Based on the original multimodal classroom data, a sequence of teacher teaching behavior characteristics, a sequence of student learning status characteristics, and a sequence of classroom atmosphere characteristics are generated, and teaching behavior scores and learning engagement scores are generated simultaneously.

[0011] A time-series correlation result is generated based on the teacher's teaching behavior feature sequence and the student's learning state feature sequence. The time-series correlation result is then subjected to interference correction based on the classroom atmosphere feature. Candidate correlation feature pairs are selected, and a directional time-series causal analysis and reverse verification are performed on the candidate correlation feature pairs, pointing from the teacher's teaching behavior feature to the student's learning state feature, to generate a directed weighted causal graph and critical path.

[0012] Based on a pre-trained attribution analysis model, attribution results are generated according to the critical path, the teaching behavior score, the learning engagement score, and the classroom atmosphere characteristics; based on a pre-trained suggestion generation model, teaching improvement suggestions and learning guidance suggestions are generated according to the attribution results, the critical path, the teaching behavior score, the learning engagement score, and the classroom atmosphere characteristics.

[0013] The attribution analysis model and the suggestion generation model are updated based on the changes in teaching behavior scores and learning engagement scores after feedback.

[0014] Before generating the critical path, quality control is performed on the low-confidence time window based on the modality validity label. When the proportion of low-confidence time windows exceeds the low-confidence time window proportion threshold, the output of the critical path and the teaching improvement suggestions is blocked.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] This invention maps video, audio, sound field, and environmental sensor data to the same time window by unifying the classroom timeline, reducing time mismatch caused by differences in sampling frequencies of different modalities. A classroom comfort index characterizes the comprehensive impact of temperature, humidity, light intensity, and carbon dioxide concentration on classroom status, and the initial time-series correlation results are attenuated and corrected based on the quantified deviation of the classroom comfort index, reducing spurious correlations caused by environmental anomalies. By comparing the prediction residuals of the baseline prediction model and the extended prediction model, the directional influence of teacher teaching behavior characteristics on student learning status characteristics is confirmed by the prediction error reduction rate and the consistency of adjacent validation window directions, avoiding the generation of feedback suggestions based solely on correlation. Backward verification eliminates directional misjudgments caused by students' learning status inversely influencing teacher teaching behavior or common factors, improving the reliability of the directed weighted causal graph. Key paths are screened using the directed weighted causal graph and path contribution values, enabling feedback suggestions to correspond to specific teacher behavior elements, student status anomalies, and causal evidence chains. Score triggering conditions, key path hit conditions, and posterior contribution threshold conditions jointly control suggestion output, reducing invalid suggestions triggered by occasional anomalies and low-contribution features. The effectiveness of the recommendations is determined by measuring changes in teaching behavior scores, learning engagement scores, and classroom comfort index in subsequent comparable classrooms, and the model is updated accordingly to achieve a closed loop of teaching feedback. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the overall process of the method of the present invention.

[0019] Figure 2 This is a flowchart of the original multimodal data acquisition process in the classroom according to the present invention.

[0020] Figure 3 This is a flowchart of the time alignment process for the original multimodal data in the classroom according to the present invention.

[0021] Figure 4 This is a flowchart of the process for generating teacher teaching behavior feature sequences and teaching behavior scores according to the present invention.

[0022] Figure 5 This is a flowchart illustrating the process of generating student learning state characteristic sequences and learning engagement scores according to the present invention.

[0023] Figure 6 This is a flowchart of the classroom atmosphere characteristic calculation process of the present invention.

[0024] Figure 7 This is a flowchart of the environmental interference correction and candidate correlation feature pair screening process of the present invention.

[0025] Figure 8 This is a flowchart of the directional temporal causal analysis and reverse verification process of the present invention.

[0026] Figure 9 This is a flowchart of the directed weighted causal graph construction and critical path screening process of the present invention.

[0027] Figure 10 This is a flowchart of the logic gating process for generating attribution results and recommendations in this invention. Detailed Implementation

[0028] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0029] The following is in conjunction with the appendix Figures 1-10 The embodiments of the present invention will be described in detail below.

[0030] Example 1: This example provides an AI-assisted method for analyzing and providing feedback on classroom teaching behavior. The AI-assisted method for analyzing and providing feedback on classroom teaching behavior is deployed in a smart classroom server, edge computing device, or campus teaching quality analysis platform.

[0031] At the start of class, the multimodal data acquisition module activates the panoramic camera, directional microphone array, and environmental sensor group. The panoramic camera captures images of the entire classroom and outputs a panoramic video stream. Based on pre-calibrated classroom coordinates, the lectern area, blackboard area, and student seating areas, the multimodal data acquisition module divides the panoramic video stream into a teacher area video stream and a student area video stream. The teacher area video stream is used for subsequent extraction of teacher body movement features, blackboard writing behavior features, and facial expression features, while the student area video stream is used for subsequent extraction of student spatiotemporal trajectory features, individual attention features, group distribution features, and group movement features.

[0032] A directional microphone array acquires acoustic information from the classroom space. A multimodal data acquisition module performs beamforming on the acoustic information to amplify sound sources in the teacher's direction. The module then performs blind source separation on the amplified acoustic information to separate the teacher's speech stream and the ambient sound field data. The teacher's speech stream is used to extract speech rate, volume, pauses, and question semantics, while the ambient sound field data is used to extract classroom noise features and student response delay features.

[0033] The environmental sensor array collects data on light intensity, temperature, humidity, and carbon dioxide concentration to form environmental perception data. When the environmental sensor array is placed in the center of the student seating area, at the front of the classroom, or at the back of the classroom, the multimodal data acquisition module maps the sampling results from sensors at different locations to the same time window and forms environmental perception data using the mean, maximum deviation, or regional representative value.

[0034] Due to differences in video frame rate, audio sampling rate, and sensor sampling period, the multimodal data acquisition module uses a unified classroom timeline. This unified timeline uses the classroom start command, the bell detection time, or the teacher's initiation time as its zero point. The multimodal data acquisition module maps the teacher area video stream, student area video stream, panoramic video stream, teacher audio stream, environmental sound field data, and environmental perception data to a length of [length missing] according to the video frame time, audio window start time, and sensor sampling time. seconds, sliding step size Within a continuous overlapping time window of seconds.

[0035] Video frame data is mapped to consecutively overlapping time windows using either the closest temporal data method or linear interpolation. Audio data is mapped to consecutively overlapping time windows using audio window energy aggregation. Environmental perception data is mapped to consecutively overlapping time windows using either linear interpolation or window mean aggregation. When any modality data cannot be completed, the multimodal data acquisition module generates a modality validity marker for the corresponding time window.

[0036] Each consecutive overlapping time window serves as a unified calculation unit for subsequent analysis. This unified calculation unit includes video clips of the teacher's area, student's area, panoramic video clips, teacher's audio clips, environmental sound field clips, and environmental perception data within the same class period. If any modality is briefly missing within a specific time window, the multimodal data acquisition module generates a modality validity marker; if the missing proportion exceeds a preset proportion, the multimodal data acquisition module marks the corresponding time window as a low-confidence time window.

[0037] Through the above processing, the multimodal data acquisition module obtains the raw multimodal classroom data. The raw multimodal classroom data includes aligned multimodal data segments, time window numbers, modality validity markers, and sampling integrity markers.

[0038] The teacher teaching behavior feature extraction module receives the teacher's regional video stream and teacher's voice stream from the original multimodal classroom data and generates a sequence of teacher teaching behavior features.

[0039] For teacher body movement characteristics, the teacher teaching behavior feature extraction module extracts key points of the teacher's head, shoulders, elbows, wrists, and hips using a human posture estimation model, and calculates the teacher's torso tilt angle, upper limb extension range, hand movement speed, and teacher movement trajectory. The above calculation results are aggregated within each time window to obtain the teacher's body movement characteristics.

[0040] For blackboard writing behavior characteristics, the teacher teaching behavior characteristic extraction module uses the blackboard area, whiteboard area, or electronic screen area as the blackboard writing area. This module identifies new text, graphics, formula traces, erasure traces, and dwell time within the blackboard writing area, generating blackboard area change rate, blackboard switching frequency, blackboard retention time, and dwell time for key content. Blackboard writing behavior characteristics are used to determine whether teachers switch blackboards too quickly, do not spend enough time on key content, or have disorganized blackboard space.

[0041] For facial expression features, the teacher teaching behavior feature extraction module generates the degree of facial activity, the proportion of the teacher facing students, and the amplitude of facial expression changes based on facial key points, head posture, and the duration of time the teacher faces students in the teacher's regional video stream. For speech rhythm features, the module generates speech rate, volume, number of pauses, average pause duration, pitch variation, and question semantics based on the teacher's speech stream. The module arranges these features according to a unified classroom timeline, forming a sequence of teacher teaching behavior features.

[0042] The teacher teaching behavior feature extraction module compares the sequence of teacher teaching behavior features with the distribution of features from history classes in the same course to generate a teaching behavior score. The range of values ​​for the teaching behavior score is as follows: to The higher the teaching behavior score, the closer the teacher's teaching behavior is to the characteristic distribution of high-quality history classes in the same course. The characteristic distribution of history classes in the same course is a statistical distribution of teacher teaching behavior characteristics and student learning status characteristics in history classes of the same course. The quantile threshold of the characteristic distribution of history classes in the same course is calculated by the characteristic distribution of history classes in the same course according to the preset quantile.

[0043] The student learning status feature extraction module receives student area video stream, panoramic video stream and environmental sound field data, and generates a sequence of student learning status features.

[0044] For student spatiotemporal trajectory features, the student learning state feature extraction module performs multi-object detection and cross-frame tracking, associating individual students with seat mapping numbers to generate position changes, body posture changes, and seat departure states corresponding to each seat mapping number. For individual attention features, the student learning state feature extraction module identifies student facial orientation, head posture, and gaze direction, and determines the classroom attention target direction based on the teaching event. When the teaching event is a teacher lecture, the classroom attention target direction is the teacher's current position; when the teaching event is a blackboard derivation, the classroom attention target direction is the blackboard area or electronic screen area; when the teaching event is a group discussion, the classroom attention target direction is the corresponding discussion area. The student learning state feature extraction module generates attention concentration, attention stability, and attention shift frequency based on the degree of deviation between the student's gaze direction and the classroom attention target direction.

[0045] For group distribution and movement characteristics, the student learning state feature extraction module statistically analyzes student attention distribution, movement amplitude, head turning frequency, and group synchronous movement degree according to front row, middle row, back row, left side, right side, or other seating areas. For classroom noise characteristics, the module suppresses teacher speech from environmental sound field data, extracts student acoustic data and background acoustic data, and generates classroom noise energy, noise duration, and noise spatial distribution. For student response delay characteristics, the module calculates student response delay characteristics starting from the end of the teacher's question and ending at the moment when the student acoustic data first meets the response energy and semantic response conditions.

[0046] The student learning status feature extraction module compares the student learning status feature sequence with the feature distribution of historical classes in the same course to generate a learning engagement score. The range of the learning engagement score is as follows: to The higher the learning engagement score, the closer the student's learning status is to the distribution of historical high-engagement classroom characteristics in the same course.

[0047] Through the above processing, each time window corresponds to a teacher's teaching behavior feature vector and a student's learning status feature vector, forming the data foundation for subsequent correlation correction and directional temporal causal analysis.

[0048] Example 2: This example is a further optimization based on Example 1. In this example, the classroom atmosphere feature calculation module generates classroom comfort index, classroom activity index, classroom interaction intensity index and classroom concentration index based on the original multimodal data of the classroom.

[0049] The classroom comfort index is used to characterize the potential interference of the physical environment on students' learning state. The classroom atmosphere characteristic calculation module transforms temperature, humidity, light intensity, and carbon dioxide concentration data into state values ​​of the same dimensions using normalization mapping functions, and then performs weighted fusion.

[0050]

[0051] in, For time window The classroom comfort index under the current situation To standardize the time window numbering on the classroom timeline, The weighting is based on the effect of temperature. Temperature data The corresponding normalized mapping result, For time window The following temperature data, Humidity affects the weighting. For humidity data The corresponding normalized mapping result, For time window Humidity data below, The weighting is determined by the influence of light intensity. Light intensity data The corresponding normalized mapping result, For time window The following light intensity data, The weighting is determined by the influence of carbon dioxide concentration. For carbon dioxide concentration data The corresponding normalized mapping result, For time window The carbon dioxide concentration data is as follows. The value ranges for the influence weights of temperature, humidity, light intensity, and carbon dioxide concentration are all [missing information]. to Furthermore, the sum of the weights for the effects of temperature, humidity, light intensity, and carbon dioxide concentration is... .

[0052] Classroom comfort index normalized to to The numerical range, with the standard distribution baseline set as... The normalized mapping functions for temperature, humidity, light intensity, and carbon dioxide concentration data are established based on the historical normal distribution of classroom data or the school's preset comfort range; when the corresponding environmental perception data is within the preset comfort range, the corresponding normalized mapping result is no less than... When the corresponding environmental perception data deviates from the preset comfort range, the corresponding normalized mapping result decreases linearly according to the degree of deviation. .

[0053] The classroom activity index is generated from group movement characteristics; the classroom interaction intensity index is generated by fusing the speech energy of the teacher's speech stream and the acoustic energy of classroom noise characteristics; and the classroom focus index is generated from the statistical distribution of individual attention characteristics. Classroom atmosphere characteristics are not directly used to evaluate teachers, but rather serve as constraints for interference correction and feedback output.

[0054] The association-causal analysis module first generates an initial time-series correlation matrix based on the teacher's teaching behavior characteristic sequence and the student's learning state characteristic sequence. The module then calculates the initial mutual information value between each dimension of teacher's teaching behavior characteristic and each dimension of student's learning state characteristic; this initial mutual information value is used as an element in the initial time-series correlation matrix. The mutual information calculation is performed using a probabilistic estimation method.

[0055] After obtaining the initial time-series correlation matrix, the correlation causality analysis module reads the classroom comfort index and calculates the quantitative deviation of the classroom comfort index relative to the standard distribution baseline. When the quantitative deviation exceeds the preset safety baseline, an abnormal environmental condition is determined to exist. Abnormal environmental conditions include excessively high carbon dioxide concentration, temperature deviating from the comfortable range, insufficient lighting, or abnormal humidity. The preset safety baseline range is as follows: to The preferred value is The range of values ​​for the environmental sensitivity attenuation coefficient is as follows: to The preferred value is The preset threshold value range is as follows: to The preferred value is .

[0056] The correlation causality analysis module uses the quantified deviation of the classroom comfort index to perform multiplicative numerical decay on the initial time-series correlation matrix:

[0057]

[0058] in, These are the elements of the corrected time series correlation matrix. For specific dimensions of teacher teaching behavior characteristics, For specific dimensions of student learning status characteristics, Characteristics of teachers' teaching behavior Characteristics of students' learning status Initial values ​​of mutual information between them This is the environmental sensitivity attenuation coefficient. This represents the quantitative deviation of the classroom comfort index from the standard distribution baseline. It is a natural exponential function.

[0059] When the classroom physical environment is close to normal, the quantification deviation is small, and the initial correlation value is basically preserved. When the classroom physical environment is significantly abnormal, the quantification deviation increases, and the environmental decay coefficient output by the exponential environmental decay function reduces the initial correlation value, thereby reducing false teacher-student correlations caused by environmental anomalies. The correlation causality analysis module compares the elements of the corrected time-series correlation matrix with a preset correlation threshold. Feature combinations that exceed the preset correlation threshold are selected as candidate correlation feature pairs.

[0060] For example, the initial mutual information between the teacher's speaking speed characteristic and the characteristic of decreased attention among students in the back row is... Within the same time window, the classroom comfort index was: The baseline value of the standard distribution is Quantitative deviation is The environmental sensitivity attenuation coefficient is Based on the multiplicative numerical decay calculation, the elements of the corrected time-series correlation matrix are: .because Below the preset association threshold The correlation causality analysis module does not identify the teacher's speaking speed feature and the back row students' attention decline feature as candidate correlation feature pairs, thereby avoiding the misattribution of the student attention decline caused by abnormal carbon dioxide concentration to changes in the teacher's speaking speed.

[0061] Compared to schemes that only use correlation statistics, this embodiment adds classroom environment interference correction before entering causal analysis, so that student attention decline, response delay increase or group movement increase dominated by physical environment abnormalities will not be directly attributed to teacher teaching behavior.

[0062] Furthermore, in some preferred embodiments, after environmental interference correction, the candidate correlation feature pairs still only indicate a strong correlation between teacher teaching behavior characteristics and student learning status characteristics. To avoid misjudging correlation as causation, the correlation causation analysis module performs directional temporal causation analysis on the candidate correlation feature pairs.

[0063] The association-causal analysis module performs a stationarity assessment on each candidate pair of associated features. When the teacher's teaching behavior feature sequence or the student's learning status feature sequence exhibits a trend term or mean drift, the association-causal analysis module performs differencing on the corresponding sequence to ensure that the sequence meets the input requirements of the subsequent prediction model.

[0064] The correlation causal analysis module constructs a baseline prediction model. This baseline prediction model uses only historical window data of the student's learning status characteristics to predict the student's learning status characteristics in the current or subsequent windows.

[0065]

[0066] in, For the baseline prediction model in the time window The predicted output value of the student's learning status characteristics. To standardize the time window numbering on the classroom timeline, The maximum number of historical time windows. This refers to the sequence number of the historical time window. Characteristics of student learning status in history The characteristic values ​​of each time window For the historical first in the benchmark prediction model The autoregressive fit coefficients corresponding to each time window For the baseline prediction model in the time window The predicted residual term.

[0067] The association-causal analysis module constructs an extended prediction model. Based on the historical window of student learning status characteristics, the extended prediction model introduces a historical window of teacher teaching behavior characteristics:

[0068]

[0069] in, To extend the predictive model within the time window The predicted output value of the student's learning status characteristics. To standardize the time window numbering on the classroom timeline, The maximum number of historical time windows. This refers to the sequence number of the historical time window. To expand the historical data in the prediction model The autoregressive fit coefficients corresponding to each time window Characteristics of student learning status in history The characteristic values ​​of each time window Characteristics of teachers' teaching behavior in history Causal contribution coefficient under a time window Characteristics of teachers' teaching behavior in history The characteristic values ​​of each time window To extend the predictive model within the time window The predicted residual term.

[0070] The correlation causality analysis module calculates the sum of squared residuals for the baseline prediction model and the extended prediction model within a continuous validation time window, and also calculates the prediction error reduction rate.

[0071]

[0072] in, To extend the rate of reduction in prediction error of the prediction model relative to the baseline prediction model, To standardize the time window numbering on the classroom timeline, For a set of consecutive verification time windows, For the baseline prediction model in the time window The predicted residual term is below. To extend the predictive model within the time window The predicted residual term.

[0073] The maximum number of historical time window spans can be categorized as follows: to The set of continuous verification time windows contains no fewer than [number of time windows]. If the prediction error reduction rate is greater than zero, it indicates that the prediction error of student learning status has decreased after introducing the teacher's teaching behavior history window. The association causal analysis module further checks whether the direction of the prediction error reduction rate is consistent across multiple adjacent verification time windows. Only when the prediction error reduction rate is greater than zero and the direction is continuously consistent will the association causal analysis module determine the candidate association feature pair as a causal relationship pair pointing from teacher teaching behavior features to student learning status features.

[0074] To further improve the reliability of causal direction, the association causal analysis module also performs reverse verification. Reverse verification uses a historical window of student learning status characteristics to predict teacher teaching behavior characteristics. When the reduction rate of the reverse prediction error is not less than the reduction rate of the forward prediction error, the association causal analysis module marks the corresponding candidate association feature pair as a relationship to be confirmed, and does not write it into the directed weighted causal graph. The relationship to be confirmed is stored in the set of samples to be confirmed, and directional temporal causal analysis and reverse verification are performed again in subsequent comparable classroom records; when the same candidate association feature pair appears consecutively in subsequent comparable classroom records... When the rate of reduction in prediction error from teacher teaching behavior characteristics to student learning status characteristics is greater than the rate of reduction in reverse prediction error, the correlation causal analysis module will update the relationship to be confirmed into a causal relationship pair.

[0075] The correlation causal analysis module aggregates all confirmed causal relationships to construct a directed weighted causal graph. Teacher teaching behavior characteristics and student learning status characteristics serve as nodes in the topological network, causal relationship pairs serve as directed edges, the direction of the directed edges points from teacher teaching behavior characteristics to student learning status characteristics, and the weight of the directed edges is determined by the prediction error reduction rate.

[0076] In the directed weighted causal graph, the association causal analysis module traverses each topological path and calculates the path contribution value:

[0077]

[0078] in, The path contribution value for a specific topology path. This refers to the normalized adjustment weight coefficient corresponding to the average edge weight. This is the average edge weight of all directed edges on the topological path. The normalized adjustment weight coefficients corresponding to the path length. This represents the total number of directed edges contained in the topological path. The maximum allowed path length, The normalized adjustment weight coefficients corresponding to node importance. This is the cumulative importance value of all intermediate nodes in the topology path. The normalized adjustment weight coefficients corresponding to the degree of abnormality in students' learning status. The degree of abnormality in the student's learning status at the result node.

[0079] The normalized adjustment weight coefficients corresponding to the average edge weight, path length, node importance, and student learning status abnormality all have the following value ranges: to ,and The sum of the values ​​of the normalized adjustment weight coefficients is . The normalized adjustment weight coefficients are obtained through training with historical classroom samples or preset by the teaching and research management system. The maximum allowable path length ranges from [value missing]. to .

[0080] The path contribution value considers causal edge strength, path simplicity, node importance, and the degree of student learning status anomaly. The association causal analysis module sorts the path contribution values ​​from largest to smallest and selects the top-ranked topological paths that simultaneously connect teacher teaching behavior feature nodes with teaching behavior scores below a preset teaching behavior score threshold with student learning status feature nodes with student learning status anomaly levels above a preset student learning status anomaly threshold as critical paths. The degree of student learning status anomaly is determined by the extent to which student learning status features deviate from the median of the historical classroom feature distribution of the same course.

[0081] The pre-trained attribution analysis model was trained using historical classroom samples. These samples included sequences of historical teachers' teaching behaviors, sequences of historical students' learning states, characteristics of the historical classroom atmosphere, historical teaching behavior scores, historical learning engagement scores, historical critical paths, and manually verified historical attribution results. The pre-trained suggestion generation model was trained using historical classroom samples and manually verified historical teaching suggestions.

[0082] The teaching feedback generation module uses the teacher's teaching behavior characteristics, student learning status characteristics, edge weights, teaching behavior scores, learning engagement scores, and classroom atmosphere characteristics corresponding to the critical path as inputs to a pre-trained attribution analysis model. Based on the attribution analysis model, it outputs the posterior contribution rate of each teacher's teaching behavior element to the target outcome state. The target outcome state is the student's learning status represented by the learning engagement score. The attribution results include key teacher teaching behavior elements, affected student learning status characteristics, path contribution values, edge weights, classroom atmosphere background, and posterior contribution rates.

[0083] To avoid invalid suggestion output, the teaching feedback generation module employs multiple logical gating mechanisms. Specific teacher teaching behavior elements must simultaneously meet the following conditions before being input into the suggestion generation model:

[0084] Scoring trigger condition: The teaching score in a specific dimension is lower than the quantile threshold of the history class characteristic of the same course;

[0085] Critical path hit condition: A specific teacher instructional behavior element is located at the starting side of the critical path;

[0086] Posterior contribution threshold condition: The posterior contribution rate of a specific teacher's teaching behavior element is not lower than the median of the contribution values ​​of all teacher's teaching behavior elements on the same critical path.

[0087] When the above When all conditions are met, it is recommended that the model generate teaching improvement suggestions based on teacher teaching behavior elements, types of abnormal student learning states, and classroom atmosphere characteristics. These suggestions include adjusting blackboard writing time, controlling speaking speed, adjusting questioning intervals, restating key knowledge points, adjusting interaction rhythm, and providing reminders for students in designated seating areas. Learning guidance suggestions are output to the entire class or anonymous seating areas, without directly revealing students' real identities.

[0088] For example, a course on the derivation of mathematical function graphs follows... Second window length and Alignment is performed using a sliding step size of one second. Within a certain time window, the teacher's blackboard writing switching frequency is high, the dwell time on key formulas is short, while the student's attention shift frequency increases and the student response delay characteristic rises. The classroom comfort index is within the normal fluctuation range, so environmental interference correction will not significantly weaken the initial correlation value. The correlation causality analysis module selects the teacher's blackboard writing behavior characteristics and the student response delay characteristics as candidate correlation feature pairs.

[0089] The association causality analysis module performs directional temporal causality analysis on candidate association feature pairs. The baseline prediction model uses only a historical window of student response delay features to predict the current student response delay feature, with a residual sum of squares of... The extended prediction model incorporates historical windows into the teacher's blackboard switching frequency and the duration of key content dwell time, based on the baseline prediction model. The sum of squared residuals is... According to the calculation method for the prediction error reduction rate, the prediction error reduction rate of the extended prediction model relative to the baseline prediction model is... .

[0090] The set of continuous verification time windows includes Adjacent time windows. The prediction error reduction rates corresponding to the adjacent time windows are respectively , and , The reduction rates of all prediction errors were greater than zero and in the same direction. The correlation causality analysis module further performed reverse verification, and the reduction rates of the reverse prediction errors were respectively... , and All of these values ​​were lower than the corresponding positive prediction error reduction rate. Therefore, the correlation causal analysis module confirmed the causal relationship between the high frequency of teacher blackboard switching and the short dwell time on key content, pointing to the increased student response delay characteristics, and wrote it into the directed weighted causal graph.

[0091] Subsequently, the causal analysis module calculates the path contribution value in the directed weighted causal graph. Because the weights of edges in causal relationships are high, the degree of abnormality in student learning status is high, and the teacher's blackboard writing behavior score is lower than the quantile threshold for historical classroom features in the same course, the corresponding topological path is selected as the critical path. The teaching feedback generation module verifies the triple logical gating conditions: the teacher's blackboard writing behavior score is lower than the quantile threshold for historical classroom features in the same course, the blackboard writing behavior element is located on the starting side of the critical path, and the posterior contribution rate of the blackboard writing behavior element is not lower than the median contribution of teacher teaching behavior elements on the same critical path. After all three logical gating conditions are met, the teaching feedback generation module outputs teaching improvement suggestions: extend the dwell time on key conclusions when deriving complex formulas, reduce rapid screen switching, set short pauses and checking questions after key formulas appear, and retain the core image area.

[0092] Furthermore, in some embodiments, the feedback effect tracking module continues to track subsequent classes after the teaching feedback generation module outputs teaching improvement suggestions. To avoid evaluation bias caused by different course topics, different class levels, and different classroom environments, the feedback effect tracking module first determines whether the previous classes and subsequent classes constitute comparable classroom records.

[0093] The criteria for determining comparable classroom records include whether the course topics are similar, whether the proportion of teaching segments is similar, whether the student groups in the classes are similar, whether the class duration is similar, and whether the classroom comfort index is within a normal fluctuation range. Only subsequent classrooms that meet the comparability criteria are used to determine the effectiveness of teaching recommendations.

[0094] The feedback effect tracking module calculates the changes in learning engagement scores and teaching behavior scores in subsequent classes relative to previous classes, and determines whether to allow model updates based on the classroom comfort index.

[0095]

[0096] in, The trigger state value for updating the model instruction. This is an indicator function that outputs the condition when the condition within the parentheses is true. When the condition within the parentheses is not true, the output is: , This represents the change in learning engagement scores between subsequent lessons and earlier lessons. This represents the change in teaching behavior scores between subsequent lessons and earlier lessons. To improve the classroom comfort index for subsequent classes, This represents the lower limit of the normal fluctuation range for the classroom comfort index. This represents the upper limit of the normal fluctuation range for the classroom comfort index.

[0097] When the trigger state value of the model update command is This indicates that both teaching behavior scores and learning engagement scores improved in subsequent classes, and that the improvement occurred under normal physical conditions. The feedback effect tracking module generates a model update instruction to incrementally update the attribution analysis model and the suggestion generation model. When the trigger state value of the model update instruction is... In this process, the feedback effect tracking module does not perform automatic updates and marks subsequent classroom data as samples to be confirmed. These samples, after manual review or secondary verification using comparable classroom records, are used as training samples for the subsequent training of the attribution analysis model and the suggestion generation model, and are not directly used for automatic updates. Through this mechanism, the present invention avoids the misleading influence of abnormal environments or incomparable classrooms on model parameters.

[0098] When the same teacher subsequently teaches a similar course in the same class, the feedback effect tracking module determines whether the subsequent class constitutes a comparable classroom record with the previous class. If the teacher's blackboard writing performance score and the student's learning engagement score improve in the subsequent class, and the classroom comfort index remains within the normal fluctuation range, the feedback effect tracking module triggers a model update instruction. If the subsequent classroom environment is abnormal or the course topic is significantly different, the feedback effect tracking module blocks automatic updates.

[0099] In a real classroom, situations may arise where the teacher's image is obscured by equipment, students' faces are obscured, microphones experience temporary distortion, sensors are offline, or there is network jitter. The multimodal data acquisition module records modal validity markers for each time window. When the teacher's area video stream, student's area video stream, teacher's audio stream, environmental sound field data, or environmental perception data fall below a preset validity threshold, the multimodal data acquisition module marks the corresponding time window as a low-confidence time window. The preset validity threshold is that the confidence level of a single-modal data window is not lower than a certain threshold. to For any value in, the preferred value is The threshold for the proportion of low-confidence time windows ranges from 30% to 50%, with 40% being the preferred value.

[0100] If the proportion of low-confidence time windows exceeds the threshold, the association-causal analysis module blocks the output of critical paths, and the teaching feedback generation module blocks the output of teaching improvement suggestions and generates a data quality deficiency warning. If the proportion of low-confidence time windows does not exceed the threshold, the association-causal analysis module reduces the weight of low-confidence time windows in time-series association results, directional time-series causal analysis, and path contribution value calculation according to the ratio of the confidence level of the single-modal data window to the preset validity condition. Therefore, the AI-assisted classroom teaching behavior analysis and feedback method remains feasible in real classroom scenarios with incomplete data, while avoiding erroneous feedback caused by low-quality data.

[0101] To protect privacy, the AI-assisted classroom teaching behavior analysis and feedback method extracts student facial orientation, gaze direction, key human features, and acoustic characteristics using edge computing devices. Individual student characteristics are represented by seat mapping numbers or anonymous numbers. The teaching feedback report outputs class group suggestions or anonymous area suggestions, but does not output students' real names or identifiable facial images. The retention period, access permissions, and anonymization rules for the original video and audio are uniformly configured by the school administration.

[0102] Example 3: This example provides an AI-assisted classroom teaching behavior analysis and feedback system, including a multimodal data acquisition module, a teacher teaching behavior feature extraction module, a student learning status feature extraction module, a classroom atmosphere feature calculation module, a correlation causal analysis module, a teaching feedback generation module, and a feedback effect tracking module.

[0103] The multimodal data acquisition module collects video streams from the teacher's area, student's area, panoramic video stream, teacher's voice stream, environmental sound field data, and environmental perception data. It outputs raw multimodal classroom data aligned with a unified classroom timeline and identifies low-confidence time windows based on modality validity markers.

[0104] The teacher teaching behavior feature extraction module generates a sequence of teacher teaching behavior features and teaching behavior scores based on the original multimodal classroom data. The student learning status feature extraction module generates a sequence of student learning status features and learning engagement scores based on the original multimodal classroom data. The classroom atmosphere feature calculation module generates classroom atmosphere features based on the original multimodal classroom data.

[0105] The correlation causal analysis module corrects the temporal correlation results between the teacher's teaching behavior feature sequence and the student's learning status feature sequence based on classroom atmosphere characteristics, filters candidate correlation feature pairs, performs directional temporal causal analysis and reverse verification, and generates a directed weighted causal graph and critical path.

[0106] The teaching feedback generation module, based on a pre-trained attribution analysis model, generates attribution results according to critical paths, teaching behavior scores, learning engagement scores, and classroom atmosphere characteristics. It also generates teaching improvement suggestions and learning guidance suggestions based on a pre-trained suggestion generation model. The feedback effect tracking module, based on changes in teaching behavior scores and learning engagement scores after feedback, executes model update instructions for both the correlation causal analysis module and the teaching feedback generation module, and labels and manages samples awaiting confirmation.

[0107] The modules described above are deployed on the same server, or distributed across edge computing devices, campus servers, and the teaching management platform. Data is transmitted between the modules via the campus LAN, message queues, or database interfaces.

[0108] In some embodiments, an electronic device is also provided. The electronic device includes a processor, a memory, a communication interface, and a bus. The memory stores a computer program, and when the processor executes the computer program, it implements an AI-assisted classroom teaching behavior analysis and feedback method according to any embodiment of the present invention.

[0109] In some embodiments, a computer-readable storage medium is also provided. The computer-readable storage medium is a read-only memory, random access memory, magnetic disk, optical disk, solid-state drive, flash memory, or other medium capable of storing a computer program. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements an AI-assisted classroom teaching behavior analysis and feedback method as described in any embodiment of the present invention.

[0110] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0111] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An AI-assisted method for analyzing and providing feedback on classroom teaching behavior, characterized in that, Includes the following steps: The system acquires teacher area video stream, student area video stream, panoramic video stream, teacher voice stream, environmental sound field data, and environmental perception data during classroom teaching. It then performs time alignment of the teacher area video stream, student area video stream, panoramic video stream, teacher voice stream, environmental sound field data, and environmental perception data according to a unified classroom timeline to obtain the original multimodal classroom data. Based on the original multimodal classroom data, a sequence of teacher teaching behavior characteristics, a sequence of student learning status characteristics, and a sequence of classroom atmosphere characteristics are generated, and teaching behavior scores and learning engagement scores are generated simultaneously. The sequence of teacher teaching behavior characteristics includes teacher body movement characteristics, blackboard writing characteristics, facial expression characteristics, and speech rhythm characteristics. The sequence of student learning status characteristics includes student spatiotemporal trajectory characteristics, individual attention characteristics, group distribution characteristics, group movement characteristics, classroom noise characteristics, and student response delay characteristics. A time-series correlation result is generated based on the teacher's teaching behavior feature sequence and the student's learning state feature sequence. The time-series correlation result is then subjected to interference correction based on the classroom atmosphere feature. Candidate correlation feature pairs are selected, and a directional time-series causal analysis is performed on the candidate correlation feature pairs, pointing from the teacher's teaching behavior feature to the student's learning state feature, to generate a directed weighted causal graph and critical path. Based on a pre-trained attribution analysis model, attribution results are generated according to the critical path, the teaching behavior score, the learning engagement score, and the classroom atmosphere characteristics; based on a pre-trained suggestion generation model, teaching improvement suggestions and learning guidance suggestions are generated according to the attribution results, the critical path, the teaching behavior score, the learning engagement score, and the classroom atmosphere characteristics. Based on the changes in teaching behavior scores and learning engagement scores after feedback, the attribution analysis model and the suggestion generation model are updated.

2. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of generating the original multimodal classroom data includes: The panoramic video stream is acquired by a panoramic camera, and the teacher area video stream and the student area video stream are divided from the panoramic video stream according to the pre-calibrated classroom coordinate system. Spatial sound wave information is captured by a directional microphone array, and beamforming and blind source separation are performed on the spatial sound wave information to separate the teacher's speech stream and the environmental sound field data. The environmental sensing data is composed of light intensity data, temperature data, humidity data, and carbon dioxide concentration data collected by an environmental sensor array. Based on the video frame time, audio window start time, and sensor sampling time, the teacher area video stream, the student area video stream, the panoramic video stream, the teacher's speech stream, the environmental sound field data, and the environmental perception data are resampled and mapped to a length of [length missing]. seconds and sliding step size is Within a continuous overlapping time window of seconds, output the time-aligned original multimodal classroom data; Specifically, video frame data is mapped to the consecutive overlapping time windows according to the principle of closest time or linear interpolation, audio data is mapped to the consecutive overlapping time windows according to the audio window energy aggregation method, and environmental perception data is mapped to the consecutive overlapping time windows according to linear interpolation or window mean aggregation method; when any modal data cannot be completed, a modal validity mark is generated for the corresponding time window.

3. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of generating the teacher's teaching behavior characteristic sequence includes: Extract the teacher's skeletal key point sequence from the teacher's area video stream, and generate the teacher's limb movement features based on the teacher's skeletal key point sequence; The blackboard area, whiteboard area, or electronic screen area in the classroom coordinate system is identified, and the blackboard writing behavior characteristics are generated based on the newly added text, graphics, formula traces, erasure traces, and dwell time within the blackboard writing area. The facial expression features are generated based on the teacher's facial key points and head posture in the teacher area video stream; The speech rhythm features are generated based on the speech rate, volume, pauses, pitch changes, and question semantics in the teacher's speech stream; The teaching behavior score is generated by comparing the teacher's body language characteristics, blackboard writing characteristics, facial expression characteristics, and speech rhythm characteristics with the characteristic distribution of history classes in the same course; the characteristic distribution of history classes in the same course is a statistical distribution of teacher teaching behavior characteristics and student learning status characteristics in history classes in the same course.

4. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of generating the student learning state feature sequence includes: Perform multi-target detection and cross-frame tracking based on the student area video stream to generate the student spatiotemporal trajectory features associated with the seat mapping number; The individual attention characteristics are generated based on the deviation between the student's facial orientation, head posture, gaze direction, and the direction of their attention in class. Based on the statistical distribution of students' attention, movement amplitude, and response status in different seating areas, the group distribution characteristics and the group movement characteristics are generated. The classroom noise features are generated based on the student acoustic data after suppressing the teacher's speech in the environmental sound field data; The student response delay feature is generated based on the time difference between the end of the teacher's question and the beginning of the student's acoustic response. The learning engagement score is generated by comparing the student's spatiotemporal trajectory characteristics, individual attention characteristics, group distribution characteristics, group movement characteristics, classroom noise characteristics, and student response delay characteristics with the historical classroom characteristics distribution of the same course.

5. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The classroom atmosphere characteristics include classroom comfort index, classroom activity index, classroom interaction intensity index, and classroom concentration index; The classroom comfort index is generated by normalizing and mapping the temperature, humidity, light intensity, and carbon dioxide concentration data from the environmental perception data, and then fusing them according to the influence weights of the corresponding physical quantities. The value range of the classroom comfort index is [missing value]. to The value ranges for the weighting of temperature, humidity, light intensity, and carbon dioxide concentration are all [missing information]. to Furthermore, the sum of the weights for the effects of temperature, humidity, light intensity, and carbon dioxide concentration is... ; The classroom activity index is generated from the group movement characteristics; The classroom interaction intensity index is generated by fusing the speech energy of the teacher's speech stream and the acoustic energy of the classroom noise features; The classroom focus index is generated from the statistical distribution of the individual's attention characteristics.

6. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of correcting interference in the time-series correlation results and selecting candidate correlation feature pairs based on the classroom atmosphere characteristics includes: Calculate the initial mutual information between the teacher's teaching behavior characteristics in a specific dimension and the student's learning status characteristics in a specific dimension within each time window, and construct an initial time-series correlation matrix; Read the classroom comfort index from the classroom atmosphere characteristics, and calculate the quantitative deviation of the classroom comfort index from the standard distribution benchmark value; When the quantization deviation exceeds the preset safety baseline, the abnormal environmental condition is determined to be established, and the quantization deviation is used to perform multiplicative numerical attenuation on the corresponding element in the initial time series correlation matrix to obtain the corrected time series correlation matrix element. Threshold filtering is performed based on the elements of the modified temporal correlation matrix to select and determine the candidate correlation feature pairs as feature combinations that exceed the preset correlation threshold. The range of values ​​for the preset security baseline is as follows: to The range of values ​​for the preset association threshold is: to .

7. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 6, characterized in that, The multiplicative numerical attenuation includes: The initial value of the mutual information between the teacher's teaching behavior characteristics and the student's learning status characteristics is used as the correlation value to be corrected. The quantitative deviation of the classroom comfort index from the standard distribution baseline value is used as an environmental penalty variable. The environmental penalty variable is input into a preset exponential environmental decay function to obtain the environmental decay coefficient. The exponential environmental decay function includes an environmental sensitivity decay coefficient, the value range of which is... to ; The environmental attenuation coefficient is used to perform multiplicative attenuation on the correlation value to be corrected to obtain the elements of the corrected time-series correlation matrix.

8. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of the directional temporal causal analysis includes: The teacher teaching behavior feature sequence and the student learning state feature sequence in the candidate association feature pair are judged for stationarity. When it is determined that the teacher teaching behavior feature sequence or the student learning state feature sequence does not meet the stationarity condition, the sequence that does not meet the stationarity condition is subjected to difference processing. A baseline prediction model is constructed using the historical windows of the student learning status characteristics as input, and an extended prediction model is constructed using the historical windows of the student learning status characteristics and the historical windows of the teacher teaching behavior characteristics as input. The maximum number of historical time window spans ranges from [value missing]. to ; Within a set of consecutive validation time windows, the prediction residuals of the baseline prediction model and the extended prediction model are calculated respectively. The prediction error reduction rate is then calculated based on the sum of squared residuals of the baseline and extended prediction models. The set of consecutive validation time windows contains no fewer than [number missing]. indivual; When the prediction error reduction rate is greater than zero, and the positive and negative directions of the prediction error reduction rate remain consistent across multiple adjacent verification time windows, the corresponding candidate associated feature pairs are determined as causal relationship pairs pointing from teacher teaching behavior features to student learning status features.

9. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 8, characterized in that, A reverse verification is also performed before determining the causal relationship pair; The reverse verification uses the historical window of the student's learning status characteristics as the cause candidate input and the current window or subsequent window of the teacher's teaching behavior characteristics as the result candidate output. When the rate of reduction in prediction error from teacher teaching behavior characteristics to student learning state characteristics is greater than the rate of reduction in reverse prediction error from student learning state characteristics to teacher teaching behavior characteristics, the causal relationship is confirmed. When the reverse prediction error reduction rate is not less than the prediction error reduction rate from teacher teaching behavior characteristics to student learning state characteristics, the corresponding candidate association feature pair is marked as a relationship to be confirmed, and the corresponding candidate association feature pair is not written into the directed weighted causal graph. The relationships to be confirmed are stored in the set of samples to be confirmed, and the directional temporal causal analysis and the reverse verification are performed again in subsequent comparable classroom records; when the same candidate association feature pair appears consecutively in subsequent comparable classroom records... When the rate of reduction in prediction error from teacher teaching behavior characteristics to student learning state characteristics is greater than the rate of reduction in reverse prediction error, the relationship to be confirmed is updated to the causal relationship pair.

10. The AI-assisted classroom teaching behavior analysis and feedback method according to claim 1, characterized in that, The process of generating the directed weighted causal graph and the critical path includes: Using the teacher's teaching behavior characteristics and the student's learning status characteristics as topological network nodes, the determined causal relationship pairs as directed edges, and the prediction error reduction rate as edge weights, the directed weighted causal graph is constructed. Traverse the topological paths in the directed weighted causal graph and calculate the path contribution value corresponding to each topological path. The path contribution value is determined by a weighted fusion of the average edge weight on the path, the path length, the importance of the intermediate nodes in the topology, and the degree of abnormality in the student's learning state as the result node. The calculated path contribution values ​​are sorted from largest to smallest. The topological paths that are at the top of the sort and simultaneously connect the teacher teaching behavior feature nodes whose teaching behavior scores are lower than the preset teaching behavior score threshold and the student learning status feature nodes whose student learning status abnormality is higher than the preset student learning status abnormality threshold are selected as the key paths. Wherein, the path length is not greater than the maximum allowed path length, and the maximum allowed path length has a range of values ​​of [value missing]. to .

Images