Intelligent classroom student concentration dynamic evaluation method and system

Abstract

The application discloses a smart classroom student concentration dynamic evaluation method and system, relates to the field of classroom intelligent sensing technology, carries out multi-modal semantic understanding on teacher teaching audio stream and digital teaching resource picture stream, identifies current teaching intention in real time, and dynamically generates an expected attention area heat map representing a teaching expected gaze probability distribution according to the current teaching intention and teaching resource picture content; from the face video stream of each student, the three-dimensional gaze direction of the student is estimated in real time and mapped to a unified teaching scene coordinate system to obtain an actual gaze point trajectory sequence, while individual visual behavior features are extracted. The application finely mines the three-dimensional gaze trajectory and micro visual behavior features of the student, constructs a quantitative evaluation index in combination with the space-time matching relationship between the actual gaze trajectory and an ideal teaching gaze path, can effectively distinguish between invalid gaze, passive watching and active learning concentration states of the student, and greatly improves the fineness of visual concentration behavior recognition of the student.

Description

Technical Field

[0001] This invention relates to the field of classroom intelligent sensing technology, and in particular to a method and system for dynamically assessing student focus in smart classrooms. Background Technology

[0002] Current smart classroom attention assessment technologies mostly rely on single visual image features to determine student status, using only superficial visual information such as facial posture and eye state for static threshold judgment. This type of assessment ignores the semantic differences in dynamic classroom teaching scenarios, fails to connect the impact of teacher's teaching content, teaching interaction methods, and the difficulty of teaching knowledge points on students' attention status, and has a low degree of matching between the assessment logic and the real classroom teaching patterns.

[0003] Meanwhile, existing technologies generally use fixed criteria to identify students' focused and distracted states, failing to consider the differences in students' reasonable gaze behavior in different teaching stages, and unable to adapt to the dynamic changes in students' cognitive load caused by the different cognitive difficulties of knowledge points. This easily leads to problems such as misjudging high-focus and high-thinking states and missing attention during interactive sessions. Traditional assessment models lack detailed analysis of students' actual visual gaze trajectories and microscopic visual behaviors, and cannot distinguish between the differences between students' passive gaze and active cognitive learning states.

[0004] Furthermore, the existing classroom attention assessment system lacks a deep integration of teaching knowledge graphs and assessment models, fails to establish a dynamic correlation between teaching difficulty and students' cognitive state, lacks the ability to adapt to classroom scenarios, and generally lacks a closed-loop optimization mechanism for teaching feedback. After long-term operation, the assessment accuracy is difficult to adapt to diverse and dynamic normalized teaching scenarios, and cannot meet the actual application needs of precise teaching and dynamic monitoring of classroom learning. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and system for dynamically assessing student focus in smart classrooms. The technical solution adopted is as follows: The method for dynamically assessing student focus in smart classrooms includes the following steps: Step 1: Synchronously collect facial video streams of each student, audio streams of the teacher's lecture, and video streams of digital teaching resources in the teaching scenario, and align the timestamps of each data stream to obtain spatiotemporally synchronized multimodal data. Step 2: Perform multimodal semantic understanding on the audio stream of the teacher's lecture and the video stream of digital teaching resources, identify the current teaching intention in real time, and dynamically generate a heatmap of expected attention area representing the probability distribution of expected gaze in teaching based on the current teaching intention and the content of the teaching resource video. Step 3: Estimate the three-dimensional gaze direction of each student in real time from the facial video stream and map it to the unified teaching scene coordinate system to obtain the actual gaze point trajectory sequence. At the same time, extract individual visual behavior features, including microsaccades, blink frequency, and head shaking energy. Step 4: Calculate the spatiotemporal matching degree between the actual gaze point trajectory sequence of each student and the heatmap of the expected attention area. At least based on the probability density of the gaze point falling into the high expectation area and the similarity between the gaze scanning pattern and the ideal scanning path, generate the teaching intention matching degree index. Step 5: Construct a dynamic state-space model with focus state and cognitive load as latent variables. The state transition of cognitive load is modulated by the cognitive difficulty of the current knowledge point obtained from the teaching knowledge graph. The teaching intention matching index and individual visual behavior characteristics are used as observations. The posterior probability of each student's instantaneous focus state is inferred online through Bayesian filtering and converted into a focus score. Step 6: Generate and output a visual assessment result based on the focus score.

[0006] Optionally, in step 1, a lightweight RGB visual acquisition device is deployed to collect the facial video stream of all students, and the audio stream of the teacher's lecture is collected through the classroom audio pickup device. The digital teaching resource video stream is captured in real time by connecting to the teaching terminal API. Using NTP global network time as a unified benchmark, high-precision timestamps are applied to each frame of the three types of multimodal data streams. Frame offset errors are compensated by time-series interpolation, and the spatiotemporal alignment error of the multimodal data is strictly controlled within 50ms, forming frame-level synchronized structured multimodal time-series data.

[0007] Optionally, step 2, which involves multimodal semantic understanding and generation of the expected attention region heatmap, includes the following sub-steps: Step 21: Perform speech-to-text processing on the teacher's audio stream, input it into the classroom domain pre-trained large language model to complete semantic parsing, and identify five types of teaching intentions and corresponding dense semantic feature vectors: explaining new knowledge, asking questions and interacting, group discussion, demonstrating operation, and silent thinking. Step 22: Perform visual semantic segmentation and key region detection on the digital teaching resource video stream to locate the core teaching focus areas, including text knowledge point areas, formula demonstration areas, graphic explanation areas, and interactive operation areas. Step 23: Integrate the semantics of real-time teaching intent with the key regional features of teaching resources, and combine the standard student gaze behavior paradigms corresponding to different teaching intents. Using Gaussian kernel probability modeling, dynamically generate a pixel-level heatmap of expected attention regions. The gray value of each pixel in the heatmap corresponds to the expected probability of student classroom gaze.

[0008] Optionally, in step 3, the student's facial video stream is analyzed frame by frame based on the lightweight 3D face deformation model 3DDFA, the key points of the eyes and facial posture parameters are fitted, the student's 3D gaze direction vector and head rotation Euler angle are solved in real time, and the individual gaze direction is uniformly mapped to the global teaching scene coordinate system through the coordinate transformation matrix to generate a continuous time sequence of actual gaze point trajectories. The microsaccade amplitude, microsaccade frequency, and steady-state blink frequency features are extracted frame by frame by eye temporal features. The head shaking speed and cumulative shaking energy are calculated by the difference of head Euler angle temporal variation to quantify the student's head movement activity. Finally, a multi-dimensional individual visual behavior feature sequence containing microsaccade, blink frequency, and head shaking energy is obtained.

[0009] Optionally, step 4, which calculates the spatiotemporal matching degree and generates the teaching intention matching degree index, includes the following sub-steps: Step 41: Based on the heatmap of expected attention regions, divide the region into high-expectation fixation region, medium-expectation fixation region, and low-expectation fixation region, and calculate the cumulative duration and probability density of the student's actual fixation point falling into the high-expectation region within a fixed time window. Step 42: Based on the current teaching intention, preset the ideal scanning path for standard knowledge points, and calculate the morphological similarity and temporal matching degree between the student's actual gaze scanning trajectory and the ideal scanning path through the dynamic time warping algorithm; Step 43: Adaptively weighted fusion of region gaze probability density and scanning path similarity, and normalized to obtain the teaching intention matching index in the 0-1 interval, which accurately represents the degree of fit between students' visual gaze behavior and the current teaching rhythm and teaching focus.

[0010] Optionally, the construction of the dynamic state-space model in step 5 includes two parts: the observation equations and the state transition equations. The observation equation uses real-time acquired teaching intention matching index, micro-saccade characteristics, blink frequency, and head shaking energy as multidimensional observation variables to establish a nonlinear mapping relationship between the observation variables and the latent variables of focus state and cognitive load. The state transition equation constructs a self-iterative transition rule for the focused state. At the same time, a teaching knowledge graph is introduced to pre-label the cognitive difficulty coefficients of the memory, understanding, and application levels of each knowledge point. The temporal transition probability of cognitive load is dynamically modified with the cognitive difficulty of the knowledge point as a modulation factor, so as to realize the differentiated iterative update of the hidden state under different teaching difficulty scenarios.

[0011] Optionally, step 5, the focus state inference and score conversion based on Bayesian filtering, includes the following sub-steps: Step 51: Initialize the prior probability distribution of students' attention state and cognitive load. Combine the updated multidimensional observation data at each moment and complete the posterior iterative update of the dynamic state space model through unscented Kalman filtering to solve the posterior probability distribution of students' instantaneous attention state in real time. Step 52: Linearly normalize and calibrate the posterior probability of the focus state, and adjust the scoring weights according to the level of cognitive load. When the cognitive load is higher and the matching degree of teaching intention is met, the focus score weights are appropriately increased. Finally, a standardized real-time student focus score in the range of 0-100 is obtained.

[0012] Optionally, the model closed-loop optimization step is also included: collecting manual labeled true value samples of students' attention status by teachers after class, including three types of labeled data: overestimation, underestimation, and accurate judgment; constructing an attention assessment reward function; and using a proximal strategy optimization algorithm to dynamically correct the knowledge point cognitive difficulty modulation coefficient, spatiotemporal matching degree weighting coefficient, and Bayesian filter iteration parameters to continuously improve the accuracy of attention assessment in complex teaching scenarios.

[0013] Optionally, the visualization evaluation results in step 6 include both individual student and class-wide dimensions: the individual student dimension outputs real-time focus value, focus time-series fluctuation curve, and a comparison chart of the gaze trajectory and the expected heat map; the class-wide dimension outputs a heat map of the overall student focus distribution and a trend chart of the average focus of the class during each teaching intention period, thus achieving a panoramic visualization of the focus status of individuals and the class.

[0014] The Smart Classroom Student Attention Dynamic Assessment System is used to implement a dynamic assessment method for student attention in smart classrooms. The system includes a multimodal data synchronous acquisition module, a teaching intention semantic analysis module, a student visual behavior perception module, an indicator calculation module, a dynamic implicit state inference assessment module, and a visualization output module. The multimodal data synchronization acquisition module is used to acquire student facial video streams, teacher lecture audio streams, and digital teaching resource video streams, and to complete high-precision timestamp alignment and time sequence synchronization of multimodal data; The teaching intent semantic parsing module is used to integrate the teacher's audio semantics and the teaching resource visual semantics to identify real-time teaching intent and dynamically generate a heatmap of the expected attention area based on the probability distribution of teaching expectation gaze. The student visual behavior perception module is used to solve the student's three-dimensional gaze direction and global gaze point trajectory, and extract individual visual behavior features; The index calculation module is used to calculate the spatiotemporal matching degree between the actual gaze trajectory and the expected heatmap, integrate the region probability density and the scanning path similarity, and generate a standardized teaching intention matching degree index. The dynamic latent state inference and evaluation module is used to construct a dynamic state space model that modulates the cognitive difficulty of knowledge points, and infers the posterior probability of students' attention state online through Bayesian filtering, and outputs a real-time standardized attention score. The visualization output module is used to generate multi-dimensional classroom attention visualization results, receive manual annotation feedback from teachers, and complete closed-loop adaptive optimization of model parameters through reinforcement learning.

[0015] In summary, the present invention has at least one of the following beneficial technical effects: This invention provides a method and system for dynamic evaluation of student attention in smart classrooms. It refines the mining of students' three-dimensional gaze trajectory and microscopic visual behavior characteristics, and constructs quantitative evaluation indicators by combining the spatiotemporal matching relationship between actual gaze trajectory and ideal teaching gaze path. It can effectively distinguish between students' ineffective gaze, passive viewing and active learning attention states, and greatly improve the precision of students' visual attention behavior recognition.

[0016] By introducing a teaching knowledge graph to dynamically modulate the cognitive difficulty of knowledge points, constructing a dynamic state space model with two latent variables, and relying on Bayesian filtering to infer students' focus state and cognitive load in real time, the static scoring mode is upgraded to a dynamic probability inference mode, which can adapt to the dynamic changes in students' cognitive state under different teaching difficulties and different teaching stages, and the assessment results are more in line with students' real learning state.

[0017] By establishing a learning progress annotation and feedback mechanism and a closed-loop optimization mechanism for the model, the model parameters and assessment strategies can be continuously optimized based on feedback from front-line teaching, enabling adaptive iterative upgrades of the assessment system. Simultaneously, through multi-dimensional visualization output, a panoramic view of individual student and overall class learning progress is presented, providing reliable data support for teachers to monitor classroom learning in real time and conduct precise teaching interventions, significantly improving the intelligence and accuracy of smart classroom learning progress monitoring. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the dynamic assessment method for student focus in smart classrooms according to the present invention. Figure 2 This is a schematic diagram of a single student's real-time focus visualization according to a specific embodiment of the present invention; Figure 3 This is a visualization diagram of the gaze trajectory and the expected heatmap superimposed in a specific embodiment of the present invention; Figure 4 This is a visual schematic diagram of the focus fluctuation curve in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the seat heatmap of a specific embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings.

[0020] This invention discloses a method and system for dynamically assessing student focus in smart classrooms.

[0021] Reference Figures 1-5 Example 1, a dynamic assessment method for student focus in a smart classroom, includes the following steps: Step 1: Synchronously collect facial video streams of each student, audio streams of the teacher's lecture, and video streams of digital teaching resources in the teaching scenario, and align the timestamps of each data stream to obtain spatiotemporally synchronized multimodal data. Step 2: Perform multimodal semantic understanding on the audio stream of the teacher's lecture and the video stream of digital teaching resources, identify the current teaching intention in real time, and dynamically generate a heatmap of expected attention area representing the probability distribution of expected gaze in teaching based on the current teaching intention and the content of the teaching resource video. Step 3: Estimate the three-dimensional gaze direction of each student in real time from the facial video stream and map it to the unified teaching scene coordinate system to obtain the actual gaze point trajectory sequence. At the same time, extract individual visual behavior features, including microsaccades, blink frequency, and head shaking energy. Step 4: Calculate the spatiotemporal matching degree between the actual gaze point trajectory sequence of each student and the heatmap of the expected attention area. At least based on the probability density of the gaze point falling into the high expectation area and the similarity between the gaze scanning pattern and the ideal scanning path, generate the teaching intention matching degree index. Step 5: Construct a dynamic state-space model with focus state and cognitive load as latent variables. The state transition of cognitive load is modulated by the cognitive difficulty of the current knowledge point obtained from the teaching knowledge graph. The teaching intention matching index and individual visual behavior characteristics are used as observations. The posterior probability of each student's instantaneous focus state is inferred online through Bayesian filtering and converted into a focus score. Step 6: Generate and output a visual assessment result based on the focus score.

[0022] By adopting the above technical solution, the first step is to acquire the facial video stream of each student, the audio stream of the teacher's lecture, and the video stream of digital teaching resources from the teaching terminal during the multimodal data synchronous acquisition stage. High-precision timestamps are then applied to all data streams based on a unified NTP network time reference, achieving frame-level spatiotemporal alignment with an error of less than 50 milliseconds. Next, the teaching intent analysis stage is implemented. Speech recognition and semantic understanding using a large-scale language model in the classroom domain are performed on the teacher's audio stream to identify the current teaching intent category and generate a dense semantic feature vector. Simultaneously, visual semantic segmentation is performed on the digital teaching resource video to locate key areas such as text knowledge points and formula demonstration areas. After fusing the semantics of the teaching intent with the visual features of the resources, a pixel-level heatmap of the expected attention area is generated using Gaussian kernel probability modeling based on the standard student gaze behavior paradigm under different intents. The grayscale value of each pixel in the heatmap represents the probability that the teacher expects the student to gaze at that area.

[0023] At the perception end, a lightweight 3D facial deformation model is used to solve the 3D gaze direction vector and head rotation Euler angles frame by frame from the student's facial video stream. An individual gaze direction is uniformly mapped to the global teaching scene coordinate system through a pre-calibrated coordinate transformation matrix, forming a continuous sequence of actual gaze point trajectories. Simultaneously, the amplitude and frequency of micro-saccades are calculated from eye temporal features, and head sway energy is extracted from the head Euler angle temporal difference. These are then combined with blink frequency to construct multi-dimensional individual visual behavior characteristics. In the matching calculation stage, the actual gaze point trajectory is aligned with the expected attention area heatmap in the spatiotemporal dimension. The cumulative duration and probability density of gaze points falling into high, medium, and low expectation areas within the sliding window are statistically analyzed. Simultaneously, a preset ideal knowledge point scanning path is retrieved based on the current teaching intent. A dynamic time warping algorithm is used to measure the morphological and temporal similarity between the student's actual scanning trajectory and the ideal path. An adaptive weighted fusion of the region probability density and path similarity generates a teaching intent matching index with values ​​between 0 and 1. This indicator is incorporated into a dynamic state-space model as a core observation. The model defines two latent variables: focus state and cognitive load. The observation equation establishes a nonlinear probability mapping between the teaching intention matching index, micro-saccade features, blink frequency, head sway energy, and latent variables. The state transition equation modulates the evolution of cognitive load by the cognitive difficulty coefficient of the current knowledge point extracted from the teaching knowledge graph. The cognitive difficulty coefficient covers levels such as memory, comprehension, and application. High cognitive load has an inhibitory effect on changes in focus state that conforms to cognitive laws. Through online iterative updates using unscented Kalman filtering, the posterior probability distribution of focus state is solved at each time step. After correcting the posterior expectation with cognitive load weights, it is mapped to a standardized real-time focus score from 0 to 100 points through an sigmoid function.

[0024] The final visualization results include a comparison chart of the individual student's attention fluctuation curve, gaze trajectory, and expected heatmap, as well as a heatmap of the overall attention distribution of the class and a trend chart of the average attention change during each teaching intention period.

[0025] In Example 2, step 1 involves deploying a lightweight RGB visual acquisition device to capture video streams of all students' faces, acquiring audio streams of teachers' lectures through classroom audio pickup devices, and connecting to the teaching terminal API to capture digital teaching resource video streams in real time. Using NTP global network time as a unified benchmark, high-precision timestamps are applied to each frame of the three types of multimodal data streams. Frame offset errors are compensated through temporal interpolation, and the spatiotemporal alignment error of the multimodal data is strictly controlled within 50ms, forming frame-level synchronized structured multimodal temporal data.

[0026] By adopting the above technical solution, lightweight RGB visual acquisition devices are deployed at the front end of the classroom and at the student terminals. These devices connect to the local area network (LAN) via Gigabit Ethernet or Wi-Fi 6 links. The front-end devices capture panoramic video of the teacher and digital teaching resources at a rate of at least 25 frames per second, while the student terminal devices independently acquire facial video streams of their respective students at the same frame rate. An omnidirectional microphone array or wireless lavalier microphone is deployed in the center of the classroom to capture the teacher's audio stream, with a sampling rate set to 16kHz and encoded in AAC format to reduce transmission bandwidth. Upon startup, all acquisition devices perform a clock synchronization handshake with an edge computing node within the LAN, which acts as an NTP server. This server connects to a standard GPS or BeiDou time synchronization module to provide a global reference time, with a synchronization period of every 60 seconds to compensate for crystal oscillator drift.

[0027] After the data stream enters the acquisition node, the system immediately retrieves the current PTP synchronized system time from the operating system in each frame capture callback, injects it into the frame header as a timestamp metadata with microsecond precision, forming a timestamp data packet carrying absolute time information. Due to slight offsets between different sensor clocks, the system runs a timing alignment compensation module on the edge computing node. This module maintains an adaptive jitter buffer for each type of data stream, with the buffer depth dynamically adjusted between 0.5 seconds and 1 second. By calculating the cross-correlation peak of the timestamps of each stream, the inter-frame offset is detected. For video streams, a frame dropping or frame interpolation strategy is adopted, and for audio streams, resampling interpolation is used to strictly control the timestamp deviation between any two types of data frames within 50 milliseconds. Finally, the buffer outputs a structured multimodal synchronized timing data structure. This data structure uses a unified time index as the key, and its values ​​contain the full-class multi-channel student facial image frames, aligned teacher audio clips, and whiteboard screenshots at the corresponding time, providing a reliable time-consistent data source for all subsequent algorithm modules.

[0028] Example 3, step 2, multimodal semantic understanding and expected attention region heatmap generation, includes the following sub-steps: Step 21: Perform speech-to-text processing on the teacher's audio stream, input it into the classroom domain pre-trained large language model to complete semantic parsing, and identify five types of teaching intentions and corresponding dense semantic feature vectors: explaining new knowledge, asking questions and interacting, group discussion, demonstrating operation, and silent thinking. Step 22: Perform visual semantic segmentation and key region detection on the digital teaching resource video stream to locate the core teaching focus areas, including text knowledge point areas, formula demonstration areas, graphic explanation areas, and interactive operation areas. Step 23: Integrate the semantics of real-time teaching intent with the key regional features of teaching resources, and combine the standard student gaze behavior paradigms corresponding to different teaching intents. Using Gaussian kernel probability modeling, dynamically generate a pixel-level heatmap of expected attention regions. The gray value of each pixel in the heatmap corresponds to the expected probability of student classroom gaze.

[0029] By adopting the above technical solution, the teacher's audio stream is first converted into a text sequence by a streaming speech recognition engine. This engine integrates a classroom acoustic model and a language model of hot words in the education field, which can effectively handle formula symbols and professional terms. The text sequence is semantically segmented in 3-second increments. Each segment is fed into a classroom-domain pre-trained large language model based on the Transformer architecture. This model uses a massive amount of classroom recordings of teacher questions, explanations, and instructions on a general base model for efficient parameter fine-tuning. Its output header includes a linear classifier to predict the teaching intention category corresponding to the current segment. The category space covers five typical classroom activities: explaining new knowledge, questioning and interaction, group discussion, demonstration, and silent thinking. At the same time, the weighted average of the last hidden state of the model is extracted as the dense semantic feature vector of the intention, with a dimension typically of 768 or 1024.

[0030] Concurrently, a lightweight semantic segmentation network is used for pixel-level parsing of the digital teaching resource video stream. After training on a labeled dataset, this network can stably output masks for text knowledge point areas, formula demonstration areas, graphic explanation areas, and interactive operation button areas, and calculates the geometric center coordinates and coverage area percentage for each area. In the fusion stage, the attention heatmap generator is expected to receive the intent semantic feature vector, key area masks of the resource video, and a 2D skeleton gesture heatmap extracted from the teacher's panoramic video as joint input. The generator's main body is a UNet encoder-decoder network with skip connections. The encoder progressively compresses the spatial resolution to extract hierarchical visual features. At the bottleneck layer, the intent semantic vector is mapped through a fully connected layer and injected into the feature map in the form of channel bias, enabling the network to recognize the prior gaze behavior corresponding to the current teaching intent. For example, under the intent to explain new knowledge, the student's gaze distribution tends to densely cover the currently explained text area, while under the intent to ask questions and interact, the gaze distribution is more biased towards the teacher's facial area and the raised hand area. The decoder upsamples layer by layer to restore the original image resolution. The final layer uses Sigmoid activation to output a single-channel heatmap, where each pixel value represents the probability that the teacher expects the student to gaze at that point in the current teaching image. Gaussian kernel probability modeling is implemented during the training phase. The training objective is to minimize the Kullback-Leibler divergence between the generated heatmap and the Gaussian mixture density map fitted to the eye-tracking data of a real group of high-achieving students. This allows the network parameters to implicitly learn how the probability quality should be distributed across visually salient regions under different intentions.

[0031] Example 4: In step 3, the student's facial video stream is analyzed frame by frame based on the lightweight 3D face deformation model 3DDFA, the key points of the eyes and facial posture parameters are fitted, the student's 3D gaze direction vector and head rotation Euler angle are solved in real time, and the individual gaze direction is uniformly mapped to the global teaching scene coordinate system through the coordinate transformation matrix to generate a continuous time sequence of actual gaze point trajectories. The microsaccade amplitude, microsaccade frequency, and steady-state blink frequency features are extracted frame by frame by eye temporal features. The head shaking speed and cumulative shaking energy are calculated by the difference of head Euler angle temporal variation to quantify the student's head movement activity. Finally, a multi-dimensional individual visual behavior feature sequence containing microsaccade, blink frequency, and head shaking energy is obtained.

[0032] By adopting the above technical solution, for each student's facial video stream, the system first applies a lightweight 3D face deformation model (3DDFA) for real-time frame-by-frame processing. This model regresses the shape parameters, expression parameters, and Euler angle pose parameters of the BFM 3D face model through a cascaded convolutional network. The coordinates of 68 two-dimensional feature points and the corresponding 3D point cloud output by the model provide the geometric basis for subsequent calculations. Based on the coordinates of the left and right eye centers given by the 3D model, combined with the 2D pupil position regressed from the high-resolution feature map of the eye region, the 3D gaze direction vector is calculated in real time by solving the corneal curvature refraction equation in the eye model. This vector points from the center of the eyeball to the gaze point in space. To map the individual gaze direction to a unified teaching scene coordinate system, a standard checkerboard grid is used during the deployment phase to perform single-target calibration and hand-eye calibration on the camera and the whiteboard plane at the front of the classroom for each student, obtaining the rotation matrix and translation vector from the student camera coordinate system to the global teaching space coordinate system. The three-dimensional gaze direction ray is geometrically intersected with the whiteboard plane or the teacher's spatial volume at a known three-dimensional position to obtain the two-dimensional coordinates or spatial position of the actual gaze point with physical length units. When the gaze ray does not intersect with the teaching area, the gaze point at that moment is marked as looking around. After obtaining the gaze point trajectory sequence, individual visual behavior characteristics are calculated within a sliding window of 5 seconds. Microsaccades are extracted by detecting local extrema of the gaze point velocity signal. A microsaccade is counted when the gaze point displacement between two adjacent frames exceeds 0.5 degrees of visual angle but is less than 2 degrees of visual angle. The number and average amplitude of microsaccades within the window are counted. Blink frequency is calculated based on the eye aspect ratio (EAR) index. EAR is the ratio of the vertical distance to the horizontal distance defined by the eyelid key point coordinates. A blink is counted when the EAR value momentarily drops below 0.2 and then quickly recovers. The blink frequency is obtained by dividing the total number of blinks within the window by the window length. The head-shaking energy calculation process involves obtaining the angular velocity by taking the first-order difference of the three Euler angles of the head, then calculating the L2 norm of the angular velocities of the three channels, and finally calculating the root mean square value of this norm within the window as the head-shaking energy index. Ultimately, the individual visual behavior feature sequence generated at each time step is a four-element vector, containing a comprehensive index of microsaccade amplitude and frequency, blink frequency, head-shaking energy, and the intensity of focused or confused expressions extracted from facial motion units.

[0033] Example 5, step 4, the calculation of spatiotemporal matching degree and the generation of teaching intention matching degree index, includes the following sub-steps: Step 41: Based on the heatmap of expected attention regions, divide the region into high-expectation fixation region, medium-expectation fixation region, and low-expectation fixation region, and calculate the cumulative duration and probability density of the student's actual fixation point falling into the high-expectation region within a fixed time window. Step 42: Based on the current teaching intention, preset the ideal scanning path for standard knowledge points, and calculate the morphological similarity and temporal matching degree between the student's actual gaze scanning trajectory and the ideal scanning path through the dynamic time warping algorithm; Step 43: Adaptively weighted fusion of region gaze probability density and scanning path similarity, and normalized to obtain the teaching intention matching index in the 0-1 interval, which accurately represents the degree of fit between students' visual gaze behavior and the current teaching rhythm and teaching focus.

[0034] By adopting the above technical solution, the heatmap of the expected attention region generated in step 2 is first segmented using a three-threshold method. Regions with pixel values ​​greater than 0.7 are classified as high-expectation fixation regions, regions with pixel values ​​between 0.3 and 0.7 are classified as medium-expectation fixation regions, and regions with pixel values ​​less than 0.3 are classified as low-expectation fixation regions. For each student's actual fixation point trajectory sequence generated within the past 5-second sliding window, the pixel value of the corresponding position on the heatmap at that time is queried point by point. The ratio of the number of fixations falling into the high-expectation region to the total number of effective fixations is calculated. At the same time, the mean value of the corresponding values ​​of all effective fixations on the heatmap is calculated as the region fixation probability density score.

[0035] The path similarity assessment dimension requires retrieving the corresponding ideal knowledge point scanning path from a pre-defined path template library based on the current teaching intention category. This template is predefined by educational experts based on the optimal visual guidance logic under that teaching intention. For example, the ideal path for explaining new knowledge is a directed sequence from the title area to the core formula area and then to the example area. Both the student's actual gaze point sequence and the ideal path sequence are considered as two-dimensional time series. A dynamic time warping algorithm is used to calculate the minimum cumulative warped distance between the two sequences using Euclidean distance as the local cost metric. This warped distance is normalized to a pattern matching similarity score in the range of 0 to 1 using the formula 1 divided by 1 plus the distance value. The smaller the warped distance, the closer the score is to 1. In terms of the fusion strategy, the system dynamically adjusts the weighting factor α of the region probability density score and the pattern matching similarity score according to the current teaching intention category. When the teaching intention is a task with strong visual guidance, such as blackboard pointing or demonstration, α is set to 0.3, making the path matching weight dominant. When the teaching intention is a task with weak visual constraints, such as silent thinking or group discussion, α is set to 0.7, making the region probability density weight dominant. The final teaching intention matching index is calculated by adding 1 minus α times the region probability density score to the pattern matching similarity score. This index fully describes the degree of fit between students' actual visual attention behavior and teachers' teaching expectations from the perspectives of density dimension and order dimension.

[0036] Example 6, the construction of the dynamic state-space model in step 5 includes two parts: the observation equation and the state transition equation: The observation equation uses real-time acquired teaching intention matching index, micro-saccade characteristics, blink frequency, and head shaking energy as multidimensional observation variables to establish a nonlinear mapping relationship between the observation variables and the latent variables of focus state and cognitive load. The state transition equation constructs a self-iterative transition rule for the focused state. At the same time, a teaching knowledge graph is introduced to pre-label the cognitive difficulty coefficients of the memory, understanding, and application levels of each knowledge point. The temporal transition probability of cognitive load is dynamically modified with the cognitive difficulty of the knowledge point as a modulation factor, so as to realize the differentiated iterative update of the hidden state under different teaching difficulty scenarios.

[0037] By adopting the above technical solution, the dynamic state-space model defines two continuous latent variables, attention state Z and cognitive load L, which together constitute a two-dimensional state vector. The observation equation establishes a connection between the four observations output from step 4—the teaching intention matching index, microsaccade features, blink frequency, and head movement energy—and the latent variables through probability distributions. Specifically, the teaching intention matching index is modeled as a Gaussian distribution with the attention state Z as the mean and a fixed observation variance; the higher the attention level, the higher the expected value of the matching index. The blink frequency is modeled as a Poisson distribution with the cognitive load L as the conditional strength; increased cognitive load leads to increased blink frequency, which is consistent with the research conclusion in cognitive psychology that high-load tasks inhibit blinking. The head movement energy is modeled as an exponential distribution with the attention state Z as the conditional law; when attention decreases, the head movement energy increases exponentially. The microsaccade features are modeled as a Gaussian distribution, and their mean is simultaneously affected by the bilinear influence of attention state and cognitive load. The state transition equation describes the dynamic evolution of the latent variables from the current time to the next time. The self-transfer of the focused state is modulated by the inhibitory effect of the cognitive load of the previous moment. When L is at a higher level, the inertial decay coefficient of Z increases, simulating the physiological law that focus is more likely to decline under high load. The state transition of cognitive load introduces a driving term, which points to the cognitive difficulty coefficient of the current knowledge point extracted from the teaching knowledge graph. The knowledge graph pre-labels the knowledge points associated with each lesson with Bloom's cognitive level coefficients: memory level 0.3, comprehension level 0.6, and application level 0.9. Cognitive load is modeled as a process of stepwise regression to this difficulty coefficient, and the regression rate controls how quickly the student's cognitive load adapts to the teaching content. Both state transition processes are superimposed with Gaussian process noise, represented by the diagonal covariance matrix.

[0038] Example 7, step 5, the focus state inference and score conversion based on Bayesian filtering, includes the following sub-steps: Step 51: Initialize the prior probability distribution of students' attention state and cognitive load. Combine the updated multidimensional observation data at each moment and complete the posterior iterative update of the dynamic state space model through unscented Kalman filtering to solve the posterior probability distribution of students' instantaneous attention state in real time. Step 52: Linearly normalize and calibrate the posterior probability of the focus state, and adjust the scoring weights according to the level of cognitive load. When the cognitive load is higher and the matching degree of teaching intention is met, the focus score weights are appropriately increased. Finally, a standardized real-time student focus score in the range of 0-100 is obtained.

[0039] By adopting the above technical solution, when the system starts, a prior probability distribution is initialized for each student. The initial distribution of the focus state Z is set as a Gaussian distribution with a mean of 0.7 and a variance of 0.2, and the initial distribution of the cognitive load L is set as a Gaussian distribution with a mean of 0.5 and a variance of 0.3, which reflects the physiological normal state of students generally in a state of moderate focus and moderate load at the beginning of class.

[0040] When the observation vectors from steps 3 and 4 arrive at each time step, the inference engine executes a one-step update process using unscented Kalman filtering: First, a set of sigma sampling points is generated based on the posterior mean and covariance of the previous time step. These sampling points are then propagated through the state transition equation to obtain the predicted state distribution. Simultaneously, the predicted observation value corresponding to each sampling point is calculated based on the observation equation to obtain the predicted observation distribution. Subsequently, the cross-covariance matrix between the state and the observation, as well as the observation autocovariance matrix, are calculated. The predicted state is then corrected to the actual observation using Kalman gain to obtain the joint posterior probability distribution of the current focus state and cognitive load. This posterior distribution is compactly represented in the form of a mean vector and a covariance matrix. After obtaining the posterior distribution of the focus state, its mean is taken as the point estimate and mapped to a standardized focus score of 0 to 100 using a sigmoid function. The midpoint offset of the sigmoid function is set to 0.5, and the slope parameter is calibrated according to different grade groups of students. The cognitive load correction logic works as follows: when a student's cognitive load is detected to be higher than 0.7 and their instructional intent matching index remains above 0.6, it indicates that the student is still keeping up with the teaching pace despite the high-difficulty content. A compensatory bias of 0.05 is added to the scoring formula to reward this effort. When the cognitive load is lower than 0.3 and the instructional intent matching index is also lower than 0.4, it indicates that the student is losing focus despite the simple content. A penalty bias of 0.1 is subtracted from the scoring formula. Through this mechanism, the final standardized real-time attention score can distinguish between situations where the superficial matching scores are similar but the underlying cognitive states are drastically different.

[0041] Example 8 also includes a model closed-loop optimization step: collecting manually labeled true value samples of students' attention status by teachers after class, including three types of labeled data: overestimation, underestimation, and accurate judgment, constructing an attention assessment reward function, and using a proximal strategy optimization algorithm to dynamically correct the knowledge point cognitive difficulty modulation coefficient, spatiotemporal matching degree weighting coefficient, and Bayesian filter iteration parameters to continuously improve the accuracy of attention assessment in complex teaching scenarios.

[0042] By adopting the above technical solution, after class, teachers can use a timeline selection method to mark the attention status of specific students on the playback interface, categorizing them as overestimated, underestimated, or accurate, thus forming a preference feedback dataset. A reward function is constructed with positive reward for scoring accuracy and negative reward for deviation magnitude. A proximal strategy optimization algorithm is adopted, using the fusion weights in the expected attention heatmap generator, the modulation coefficient of the cognitive difficulty of knowledge points on state transition, the matching degree index weighting factor, and the initial prior parameters of Bayesian filtering as adjustable strategies. Gradient updates based on KL divergence constraints achieve closed-loop adaptive optimization of the model.

[0043] Example 9: In step 6, the visualized evaluation results include a single student dimension and a class-wide dimension. The single student dimension outputs real-time focus value, focus time-series fluctuation curve, and a comparison chart of the gaze trajectory and the expected heat map. The class-wide dimension outputs a heat map of the overall student focus distribution and a trend chart of the average focus of the class during each teaching intention period, realizing a panoramic visualization of the focus status of individuals and the class.

[0044] By adopting the above technical solution, the single-student dimension interface displays a real-time attention dashboard, using green, yellow, and red to indicate high, medium, and low ranges. Below, an attention fluctuation curve is attached, and semi-transparent color blocks are used to mark the teaching intention periods. The right side displays a comparison view of the gaze trajectory and the expected heat map overlaid, using a warm and cool color heat map and a dot trajectory with decreasing transparency to intuitively present the quality of attention following. At the class-wide dimension, a seating layout heat map is refreshed every 5 seconds to display the overall attention distribution, while a line graph shows the trend of the class's average attention change during various teaching intention periods such as explaining new knowledge and interactive questioning.

[0045] Example 10: A dynamic assessment system for student focus in a smart classroom, used to implement a dynamic assessment method for student focus in a smart classroom. The system includes a multimodal data synchronous acquisition module, a teaching intention semantic analysis module, a student visual behavior perception module, an indicator calculation module, a dynamic latent state inference assessment module, and a visualization output module. The multimodal data synchronization acquisition module is used to acquire student facial video streams, teacher lecture audio streams, and digital teaching resource video streams, and to complete high-precision timestamp alignment and time sequence synchronization of multimodal data; The teaching intent semantic parsing module is used to integrate the teacher's audio semantics and the teaching resource visual semantics to identify real-time teaching intent and dynamically generate a heatmap of the expected attention area based on the probability distribution of teaching expectation gaze. The student visual behavior perception module is used to solve the student's three-dimensional gaze direction and global gaze point trajectory, and extract individual visual behavior features; The index calculation module is used to calculate the spatiotemporal matching degree between the actual gaze trajectory and the expected heatmap, integrate the region probability density and the scanning path similarity, and generate a standardized teaching intention matching degree index. The dynamic latent state inference and evaluation module is used to construct a dynamic state space model that modulates the cognitive difficulty of knowledge points, and infers the posterior probability of students' attention state online through Bayesian filtering, and outputs a real-time standardized attention score. The visualization output module is used to generate multi-dimensional classroom attention visualization results, receive manual annotation feedback from teachers, and complete closed-loop adaptive optimization of model parameters through reinforcement learning.

[0046] By adopting the above technical solution, the system consists of a perception layer, an edge computing layer, and a cloud service layer. The perception layer's multimodal data synchronization acquisition module aligns student facial video, teacher audio, and teaching resource video streams with the NTP time synchronization protocol via an embedded synchronization box. The edge computing layer runs a teaching intent semantic parsing module, a student visual behavior perception module, an indicator calculation module, and a dynamic latent state inference and evaluation module on a GPU-accelerated server, achieving a single-student processing latency of less than 30 milliseconds through pipelined parallelism and memory pool reuse. The cloud service layer pushes attention scores and visualization charts to the teacher's browser via WebSocket. The adaptive optimization engine periodically extracts teacher-annotated data from the database and updates the edge model parameters using a near-end strategy optimization algorithm, forming a closed-loop system of perception, evaluation, presentation, and self-optimization.

[0047] The implementation principle of the present invention is illustrated below through specific embodiments: Taking a high school mathematics classroom as an example, the teaching content is the derivation of the vertex formula for a quadratic function, and the corresponding teaching intention is to explain new knowledge. The classroom teaching resources are teaching PPTs that include a title area, a formula derivation area, and an example problem display area. This classroom deploys the aforementioned smart classroom student attention dynamic assessment system. The following section uses student Zhang San as the assessment subject to explain in detail the complete attention assessment process for a single cycle.

[0048] Multimodal data synchronous acquisition: The wide-angle camera at the classroom front end and the student terminal camera simultaneously capture images at a frame rate of 25fps, capturing the teaching PPT resource images and student facial video streams respectively. A wireless microphone device is also used to capture the teacher's audio stream in real time. The edge server completes satellite time synchronization every 60 seconds via the NTP protocol, and all acquired data frames are equipped with microsecond-level timestamps.

[0049] The system's timing alignment compensation module monitors the timing deviation of multi-stream data in real time. It detects that the audio stream is 32ms ahead of the video stream in this acquisition. Timing compensation is completed through audio resampling interpolation algorithm. The inter-frame alignment deviation of the three types of multimodal data, namely student face video stream, teacher lecture audio stream and digital teaching resource video stream, is controlled within 50ms. Finally, the system outputs frame-level spatiotemporally synchronized structured multimodal timing data.

[0050] Teaching intent recognition and expected attention area heatmap generation: The teacher's lecture audio is converted into lecture text via streaming speech recognition: Substituting the vertex coordinates into the general formula yields the vertex formula. A classroom-domain pre-trained large language model performs multimodal joint encoding on the lecture text and real-time PPT screen, recognizing the current teaching intent as explaining new knowledge, and outputting a dense semantic feature vector with a dimension of 768.

[0051] A lightweight semantic segmentation network is used to segment the PPT screen into regions, accurately locating three core teaching areas. The center coordinates and mask area ratio parameters of each region are as follows: title region center coordinates (150, 80), mask area ratio 0.1; formula derivation region center coordinates (400, 300), mask area ratio 0.25; example region center coordinates (650, 450), mask area ratio 0.15.

[0052] The system integrates teaching semantic feature vectors, teaching area mask features, and the thermal features of the teacher's gesture pointing to the formula derivation area. It then uses a UNet network to decode and generate a pixel-level expected attention area heatmap with a resolution of 1920×1080 pixels. The heatmap's numerical distribution conforms to the matching rules of teaching focus. Specifically, the pixel value range for the formula derivation area is 0.85-0.95, the title area has a pixel value of approximately 0.6, and the non-teaching background area has a pixel value below 0.2, accurately representing the expected gaze probability distribution of students in scenarios where new knowledge is being explained.

[0053] Student 3D gaze estimation and visual behavior feature extraction: A lightweight 3D face deformation model (3DDFA) was used to analyze real-time facial video frames of student Zhang San, fitting key eye parameters and head posture parameters: left eye center coordinates (-29.8, 5.1, 12.3) mm, right eye center coordinates (29.2, 5.0, 12.4) mm; head Euler angles: yaw -2.1°, pitch 3.5°, roll 0.3°. Based on the pupil position parameters, the student's 3D gaze direction vector (0.12, -0.08, 0.99) was obtained. A coordinate system mapping was performed using a pre-set teaching scenario calibration matrix, and the intersection with the whiteboard display plane was calculated to obtain the student's real-time actual gaze point coordinates (388, 312).

[0054] The system uses a 5-second sliding feature extraction window, collecting 150 frames of gaze point data within a single window, with 148 frames showing valid gaze points. Of these, 144 frames fall within the core area of ​​the PPT formula derivation. Multiple microscopic visual behavior features are extracted simultaneously: 18 valid microsaccade events were detected, with the microsaccade judgment criterion being a visual angle displacement of 0.5°-2° between adjacent frames, and an average microsaccade amplitude of 1.2°; 12 valid blink events were detected within the window, and the blink frequency was calculated. The head shaking energy was 0.08 rad / s; the root mean square value of the angular velocity was obtained by solving the three-dimensional Euler angle temporal difference of the head; the facial motion unit features were extracted simultaneously, and the intensity of the student's focused expression was obtained as 0.72.

[0055] Spatiotemporal matching degree calculation and teaching intention matching degree index generation: The system divides the expected attention region heatmap into levels based on pixel probability values. Pixel values ​​greater than 0.7 are high-expectation fixation regions, 0.3-0.7 are medium-expectation fixation regions, and less than 0.3 are low-expectation fixation regions. All 148 effective fixation points of student Zhang San fall within the high-expectation fixation region, and the fixation time in the high-expectation region accounts for 100%.

[0056] Ten sets of effective fixation points were randomly selected, corresponding to the following heatmap pixel values: 0.88, 0.92, 0.90, 0.94, 0.87, 0.91, 0.93, 0.89, 0.92, and 0.90. The mean fixation probability density of the region was calculated. =0.906; In the context of teaching new knowledge, the system pre-sets a standardized ideal scanning path for each knowledge point. The path coordinates are normalized to 0-1, resulting in the ideal path sequence: (0.078, 0.074), (0.208, 0.278), (0.339, 0.417). Correspondingly, the actual gaze center coordinates of student Zhang San are extracted and normalized to obtain the actual trajectory sequence: (0.075, 0.080), (0.202, 0.289), (0.330, 0.420).

[0057] The cumulative Euclidean distance between the two sets of sequences was calculated using the dynamic time warping algorithm, yielding a distance value of 0.047. Further calculations were then performed to determine the gaze pattern matching similarity. ; In the context of explaining new knowledge, the weights for region matching and path matching are equally distributed, with a weighting coefficient α=0.5. The final teaching intent matching index is then calculated. ; Dynamic state-space model inference and attention score conversion: The system initializes the student's latent state prior distribution. The attention state Z follows a Gaussian distribution with a mean of 0.7 and a variance of 0.2, and the cognitive load L follows a Gaussian distribution with a mean of 0.5 and a variance of 0.3. The derivation of the quadratic function vertex formula, a key knowledge point in this lesson, corresponds to a comprehension level in the teaching knowledge graph, with a cognitive difficulty coefficient of 0.6.

[0058] State transition equation parameter settings: focus state self-transition coefficient a = 0.1, cognitive load regression rate b = 0.2, and process noise standard deviation uniformly set to 0.05. Observation equations construct multi-dimensional correlations: the teaching intention matching degree index follows a Gaussian distribution with focus state as the mean and 0.08 as the standard deviation; normalized blink frequency is observed as a Poisson distribution, with the intensity parameter linearly correlated with cognitive load; head sway energy follows an exponential distribution under focus state conditions; standardized microsaccade amplitude is used as an auxiliary Gaussian observation variable.

[0059] The state iterative update is completed by unscented Kalman filtering. The sigma point is generated using the posterior state of the previous time step. After propagation through state transition and observation equation, the cross-covariance and Kalman gain are calculated to update the posterior mean of the hidden state at the current time step: the posterior mean of the focused state Z=0.838 and the posterior mean of the cognitive load L=0.48.

[0060] An S-shaped mapping function was used to convert the attention score, with a midpoint offset of 0.5 and a slope parameter k=8, to calculate the basic attention score: ; The system's cognitive load compensation threshold is set to 0.7. The current student's posterior mean cognitive load of 0.48 is lower than the threshold, so the load compensation correction mechanism is not triggered. The basic score is rounded to the nearest integer, and the student's real-time focus score is 94.

[0061] Visualized Results Presentation: The system generates visualized assessment results from two dimensions: individual student and class as a whole. At the individual student level, the teacher's dashboard displays Zhang San's current focus score of 94 points in real time, with the status indicated as excellent. Figure 2 As shown, the interface features a 10-minute attention span fluctuation curve, marked with semi-transparent light-colored blocks to visually represent the dynamic trend of student attention steadily increasing from 85 to 94 points. The right side of the interface overlays a comparison chart of the gaze trajectory and the teaching heatmap. Using the teaching PPT as the background, the formula derivation area is highlighted in high-probability red, and the student gaze trajectory dots are orderly clustered in the highlighted core area. The trajectory decay effect is clear, visually demonstrating that student attention closely matches the teaching expectations. Figure 4 As shown.

[0062] At the overall class level, the seating arrangement heatmap visually displays the distribution of students' attention levels. Zhang San's seat corresponds to the dark green high-attention block, while also intuitively showing the differentiated attention levels of some students within the class. Figure 5As shown in the line graph, the average concentration level of the class during the new knowledge explanation session was 81 points, which is a significant improvement compared to the average concentration level of 76 points during the previous question and interaction session. This provides teachers with intuitive and quantitative data support for adjusting the teaching pace and optimizing classroom interaction strategies in real time.

[0063] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A dynamic assessment method for student focus in smart classrooms, characterized in that, Includes the following steps: Step 1: Synchronously collect facial video streams of each student, audio streams of the teacher's lecture, and video streams of digital teaching resources in the teaching scenario, and align the timestamps of each data stream to obtain spatiotemporally synchronized multimodal data. Step 2: Perform multimodal semantic understanding on the audio stream of the teacher's lecture and the video stream of digital teaching resources, identify the current teaching intention in real time, and dynamically generate a heatmap of expected attention area representing the probability distribution of expected gaze in teaching based on the current teaching intention and the content of the teaching resource video. Step 3: Estimate the three-dimensional gaze direction of each student in real time from the facial video stream and map it to the unified teaching scene coordinate system to obtain the actual gaze point trajectory sequence. At the same time, extract individual visual behavior features, including microsaccades, blink frequency, and head shaking energy. Step 4: Calculate the spatiotemporal matching degree between the actual gaze point trajectory sequence of each student and the heatmap of the expected attention area. At least based on the probability density of the gaze point falling into the high expectation area and the similarity between the gaze scanning pattern and the ideal scanning path, generate the teaching intention matching degree index. Step 5: Construct a dynamic state-space model with focus state and cognitive load as latent variables. The state transition of cognitive load is modulated by the cognitive difficulty of the current knowledge point obtained from the teaching knowledge graph. The teaching intention matching index and individual visual behavior characteristics are used as observations. The posterior probability of each student's instantaneous focus state is inferred online through Bayesian filtering and converted into a focus score. Step 6: Generate and output a visual assessment result based on the focus score.

2. The method for dynamically assessing student focus in a smart classroom according to claim 1, characterized in that, In step 1, a lightweight RGB visual acquisition device is deployed to collect facial video streams of all students, and a classroom audio acquisition device is used to collect audio streams of teachers' lectures. The teaching terminal API is connected to capture digital teaching resource video streams in real time. Using NTP global network time as a unified benchmark, high-precision timestamps are applied to each frame of the three types of multimodal data streams. Frame offset errors are compensated through time-series interpolation, and the spatiotemporal alignment error of the multimodal data is strictly controlled within 50ms, forming frame-level synchronized structured multimodal time-series data.

3. The method for dynamically assessing student focus in a smart classroom according to claim 2, characterized in that, Step 2, multimodal semantic understanding and expected attention region heatmap generation, includes the following sub-steps: Step 21: Perform speech-to-text processing on the teacher's audio stream, input it into the classroom domain pre-trained large language model to complete semantic parsing, and identify five types of teaching intentions and corresponding dense semantic feature vectors: explaining new knowledge, asking questions and interacting, group discussion, demonstrating operation, and silent thinking. Step 22: Perform visual semantic segmentation and key region detection on the digital teaching resource video stream to locate the core teaching focus areas, including text knowledge point areas, formula demonstration areas, graphic explanation areas, and interactive operation areas. Step 23: Integrate the semantics of real-time teaching intent with the key regional features of teaching resources, and combine the standard student gaze behavior paradigms corresponding to different teaching intents. Using Gaussian kernel probability modeling, dynamically generate a pixel-level heatmap of expected attention regions. The gray value of each pixel in the heatmap corresponds to the expected probability of student classroom gaze.

4. The method for dynamically assessing student focus in a smart classroom according to claim 3, characterized in that, In step 3, the student's facial video stream is analyzed frame by frame based on the lightweight 3D face deformation model 3DDFA, the key points of the eyes and facial posture parameters are fitted, the student's 3D gaze direction vector and head rotation Euler angle are solved in real time, and the individual gaze direction is uniformly mapped to the global teaching scene coordinate system through the coordinate transformation matrix to generate a continuous time sequence of actual gaze point trajectories. The micro-saccade amplitude, micro-saccade frequency, and steady-state blink frequency features were extracted by frame-by-frame detection of eye temporal features. The head shaking speed and cumulative shaking energy are calculated by the time-series variation difference of the Euler angles of the head, which quantifies the activity level of students' head movements. Finally, a multi-dimensional individual visual behavior feature sequence including micro-saccades, blink frequency, and head shaking energy is obtained.

5. The method for dynamically assessing student focus in a smart classroom according to claim 4, characterized in that, Step 4, the calculation of spatiotemporal matching degree and the generation of the teaching intention matching degree index, includes the following sub-steps: Step 41: Based on the heatmap of expected attention regions, divide the region into high-expectation fixation region, medium-expectation fixation region, and low-expectation fixation region, and calculate the cumulative duration and probability density of the student's actual fixation point falling into the high-expectation region within a fixed time window. Step 42: Based on the current teaching intention, preset the ideal scanning path for standard knowledge points, and calculate the morphological similarity and temporal matching degree between the student's actual gaze scanning trajectory and the ideal scanning path through the dynamic time warping algorithm; Step 43: Adaptively weighted fusion of region gaze probability density and scanning path similarity, and normalized to obtain the teaching intention matching index in the 0-1 interval, which accurately represents the degree of fit between students' visual gaze behavior and the current teaching rhythm and teaching focus.

6. The method for dynamically assessing student focus in a smart classroom according to claim 5, characterized in that, Step 5 involves constructing the dynamic state-space model, which includes two parts: the observation equations and the state transition equations. The observation equation uses real-time acquired teaching intention matching index, micro-saccade characteristics, blink frequency, and head shaking energy as multidimensional observation variables to establish a nonlinear mapping relationship between the observation variables and the latent variables of focus state and cognitive load. The state transition equation constructs a self-iterative transition rule for the focused state. At the same time, a teaching knowledge graph is introduced to pre-label the cognitive difficulty coefficients of the memory, understanding, and application levels of each knowledge point. The temporal transition probability of cognitive load is dynamically modified with the cognitive difficulty of the knowledge point as a modulation factor, so as to realize the differentiated iterative update of the hidden state under different teaching difficulty scenarios.

7. The method for dynamically assessing student focus in a smart classroom according to claim 6, characterized in that, Step 5, the focus state inference and score conversion based on Bayesian filtering, includes the following sub-steps: Step 51: Initialize the prior probability distribution of students' attention state and cognitive load. Combine the updated multidimensional observation data at each moment and complete the posterior iterative update of the dynamic state space model through unscented Kalman filtering to solve the posterior probability distribution of students' instantaneous attention state in real time. Step 52: Linearly normalize and calibrate the posterior probability of the focus state, and adjust the scoring weights according to the level of cognitive load. When the cognitive load is higher and the matching degree of teaching intention is met, the focus score weights are appropriately increased. Finally, a standardized real-time student focus score in the range of 0-100 is obtained.

8. The method for dynamically assessing student focus in a smart classroom according to claim 7, characterized in that, It also includes a model closed-loop optimization step: collecting manual labeled true value samples of students' attention status by teachers after class, including three types of labeled data: overestimation, underestimation, and accurate judgment, constructing an attention assessment reward function, and using a proximal strategy optimization algorithm to dynamically correct the knowledge point cognitive difficulty modulation coefficient, spatiotemporal matching degree weighting coefficient, and Bayesian filter iteration parameters, so as to continuously improve the accuracy of attention assessment in complex teaching scenarios.

9. The method for dynamically assessing student focus in a smart classroom according to claim 8, characterized in that, Step 6 provides visualization assessment results including individual student and class-wide dimensions: individual student dimension outputs real-time focus value, focus time-series fluctuation curve, and a comparison chart of gaze trajectory and expected heat map overlay; class-wide dimension outputs a heat map of student focus distribution across the entire domain and a trend chart of average focus change in class during each teaching intention period, achieving a panoramic visualization of individual and class focus status.

10. A smart classroom student attention dynamic assessment system, characterized in that, To implement the smart classroom student attention dynamic assessment method as described in claim 9, the system includes a multimodal data synchronous acquisition module, a teaching intention semantic analysis module, a student visual behavior perception module, an indicator calculation module, a dynamic hidden state inference assessment module, and a visualization output module. The multimodal data synchronization acquisition module is used to acquire student facial video streams, teacher lecture audio streams, and digital teaching resource video streams, and to complete high-precision timestamp alignment and time sequence synchronization of multimodal data; The teaching intent semantic parsing module is used to integrate the teacher's audio semantics and the teaching resource visual semantics to identify real-time teaching intent and dynamically generate a heatmap of the expected attention area based on the probability distribution of teaching expectation gaze. The student visual behavior perception module is used to solve the student's three-dimensional gaze direction and global gaze point trajectory, and extract individual visual behavior features; The index calculation module is used to calculate the spatiotemporal matching degree between the actual gaze trajectory and the expected heatmap, integrate the region probability density and the scanning path similarity, and generate a standardized teaching intention matching degree index. The dynamic latent state inference and evaluation module is used to construct a dynamic state space model that modulates the cognitive difficulty of knowledge points, and infers the posterior probability of students' attention state online through Bayesian filtering, and outputs a real-time standardized attention score. The visualization output module is used to generate multi-dimensional classroom attention visualization results, receive manual annotation feedback from teachers, and complete closed-loop adaptive optimization of model parameters through reinforcement learning.

Images