Interactive Classroom Teaching Methods and Systems Based on Edge Computing

Abstract

This invention relates to the fields of educational informatization and edge computing technology, and discloses a classroom teaching interaction method and system based on edge computing. The method involves a teacher acquisition node collecting teaching audio and screen images, extracting features to generate teaching incentive vectors and teaching modal labels; a convergence gateway generating frequency domain gating parameters based on the label mapping and distributing them; student visual nodes adjusting the passband of their local digital filters according to the parameters, performing adaptive frequency domain filtering and energy aggregation on the video stream optical flow field, generating and uploading scalar-form student response energy data; and the convergence gateway performing time-series alignment of multi-source data, calculating the cross-correlation coefficient between the two within a preset physiological effective window to obtain an interaction synchronization index, and driving the feedback device when abnormal conditions are met. This invention utilizes an edge computing architecture to achieve distributed perception, filters out environmental noise through a modality-driven frequency domain gating mechanism, reduces network bandwidth consumption, and improves the accuracy and causality of interaction feature extraction.

Description

Technical Field

[0001] This invention relates to the fields of educational informatization and edge computing technology, specifically to a classroom teaching interaction method and system based on edge computing. Background Technology

[0002] With the deepening of educational informatization, classroom teaching quality assessment has gradually shifted from the traditional manual observation model to automated analysis based on artificial intelligence. Current classroom interaction analysis solutions rely on computer vision technology, deploying multiple high-definition cameras in the classroom to collect video streams and transmitting the data in real time to a central server or cloud platform. Deep learning algorithms are then used to centrally identify students' head-up rate, facial expressions, or body movements to quantify classroom activity.

[0003] However, this technology architecture, which relies on centralized video stream analysis, has revealed several limitations in large-scale practical applications. First, this model places high demands on network bandwidth and central computing resources. The continuous uploading of multiple high-definition video streams can lead to congestion on the campus LAN, and the central server is prone to processing delays when handling concurrent requests from hundreds of classrooms, making it difficult to guarantee real-time feedback. Simultaneously, the transmission and storage of raw video streams involve sensitive biometric information such as students' facial features, posing a risk of privacy leaks and limiting its widespread application in regular teaching.

[0004] Secondly, existing technologies lack the ability to adaptively perceive the teaching context in interactive feature extraction. The classroom is a complex audio-visual environment, and students generate numerous non-interactive random physical movements during lessons (such as tidying up stationery or adjusting posture). These background motion noises overlap with real interactive responses (such as raising hands or nodding) in the frequency domain. Traditional fixed-threshold detection algorithms cannot dynamically adjust detection sensitivity based on the teacher's current teaching state, resulting in a high false alarm rate during non-interactive periods and a tendency to miss detections when capturing weak signals, making it difficult to obtain high signal-to-noise ratio interactive features.

[0005] Furthermore, current interactive assessment methods often sever the temporal causal logic between teachers' teaching behaviors and students' response behaviors. Existing systems mostly employ simple time-slice statistical methods, that is, to separately count the percentage of teacher and student states within a certain time period, while ignoring the objectively existing physiological latency period from the reception of audiovisual stimuli to the generation of neuromuscular action. Due to the lack of microsecond-level global clock synchronization and physiologically consistent time-delay window constraints, the system cannot effectively eliminate premature or delayed responses, resulting in calculated interaction indicators that reflect more the statistical correlation of data than the true causal relationship of teaching motivation. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a classroom teaching interaction method and system based on edge computing. It aims to solve the problems of high network bandwidth consumption and privacy leakage risks in existing classroom teaching solutions, the difficulty of traditional detection algorithms to adaptively filter out environmental action noise according to the teaching context, and the lack of rigorous physiological time sequence constraints leading to the loss of causality of interaction indicators.

[0007] To achieve the above objectives, the present invention provides a classroom teaching interaction method based on edge computing.

[0008] The method includes: teacher acquisition nodes acquiring teaching audio signals and screen image frames, extracting multimodal features to generate teaching stimulus vectors, and determining corresponding teaching modality labels based on the multimodal features; a convergence gateway receiving the teaching modality labels, generating frequency domain gating parameters based on a preset modality frequency domain mapping relationship, and distributing them to student visual nodes distributed in the classroom; the student visual nodes adjusting the passband of their local digital filters according to the frequency domain gating parameters, acquiring video streams in the student area, and performing adaptive frequency domain filtering and energy aggregation on the optical flow field of the video stream, uploading scalar-form student response energy data to the convergence gateway; the convergence gateway performing time-series alignment on the teaching stimulus vectors and the student response energy data, calculating the cross-correlation coefficient between the two within a preset physiological effective window to obtain an interaction synchronization index; when the interaction synchronization index meets a preset abnormal triggering condition, the convergence gateway generating a drive signal to control the feedback device to perform physical feedback.

[0009] Preferably, when generating the teaching incentive vector, the teacher acquisition node segments the teaching speech signal into frames and calculates the short-time root mean square energy through the audio acquisition module; simultaneously, the first processing module converts the screen image frames into a grayscale matrix, calculates the sum of the absolute values ​​of the pixel grayscale differences between adjacent frames to obtain the video change rate; the first processing module normalizes the short-time root mean square energy and the video change rate based on the extreme value statistics within the sliding time window, and performs weighted fusion of the two using a preset modal weighting coefficient.

[0010] Preferably, when determining the corresponding teaching modality label, the first processing module classifies the content according to a preset audio silence threshold and a high-frequency interaction threshold: when the short-time root mean square energy is lower than the audio silence threshold and the video change rate is lower than the preset video silence threshold, it is marked as a blackboard writing or silent modality; when the short-time root mean square energy is higher than the high-frequency interaction threshold or the video change rate is higher than the preset video high-frequency threshold, it is marked as a high-frequency interaction modality; and in other states, it is marked as a lecture modality.

[0011] In one specific embodiment, the modal frequency domain mapping relationship is configured as a mapping table between modes and frequency domain parameters. The mapping table stores frequency domain gating parameters that correspond one-to-one with each of the teaching modal labels. The frequency domain gating parameters include the lower cutoff frequency and upper cutoff frequency used to configure the passband of the local digital filter.

[0012] Preferably, when performing adaptive frequency domain filtering and energy aggregation, the student visual node uses a dense optical flow algorithm to convert the grayscale video stream into a dense motion vector field containing instantaneous displacement vectors. Then, based on the frequency domain gating parameters and using the bilinear transform method, the feedback coefficients and feedforward coefficients of the digital filter are calculated in real time, driving the local digital filter to filter out noise components outside the passband frequency range in the dense motion vector field.

[0013] Furthermore, the student response energy data is obtained by spatial integration and dimensionality reduction mapping. The calculation process includes: summing the squares of the horizontal and vertical components of the filtered motion vector field to obtain the instantaneous pixel power; accumulating the effective pixel power exceeding the noise filtering threshold within the field of view to obtain the original aggregated total energy; and normalizing the original aggregated total energy into scalar data by combining the depth coefficient and the total number of effective pixels.

[0014] Preferably, the timing alignment is based on a global synchronization time base established between the aggregation gateway, the teacher acquisition node, and the student visual node through a precise time protocol. The teacher acquisition node and the student visual node directly generate hardware timestamps at the time of physical signal acquisition and encapsulate them in the header of the data packet. The aggregation gateway uses a jitter buffer to reconstruct the timing relationship of the data sequence based on the timestamps.

[0015] Preferably, the calculation of the cross-correlation coefficient is limited to a time lag interval consisting of the minimum physiological response latency and the effective duration of the teaching instruction. The aggregation gateway first performs a de-DC centering operation on the teaching incentive vector and the student response energy data within the sliding time window, and then traverses the time lag within the time lag interval to solve for the maximum value of the normalized cross-correlation function.

[0016] Preferably, when determining whether the abnormal triggering condition is met, a dual-threshold state machine model with hysteresis is adopted, wherein the trigger threshold is set to be lower than the recovery threshold. The system only confirms the entry into the abnormal triggering state when the proportion of the interaction synchronization index that is continuously lower than the trigger threshold within the verification window exceeds the minimum abnormal frame proportion required for triggering, and only releases the triggering state when the interaction synchronization index rises back to above the recovery threshold.

[0017] A second aspect of the present invention provides a classroom teaching interaction system based on edge computing.

[0018] The system includes: a teacher acquisition node, deployed in the classroom podium area, for acquiring teaching audio signals and screen image frames, generating teaching incentive vectors and teaching modality labels; the teacher acquisition node includes an audio acquisition module, a video capture module, and a first processing module; student vision nodes, distributed and deployed on the classroom ceiling, for acquiring video streams and performing edge calculations based on frequency domain gating parameters, outputting student response energy data; the student vision nodes include an image sensing module and a second processing module with a built-in variable parameter digital filter; a convergence gateway, communicatively connected to both the teacher acquisition node and the student vision node, for sending the corresponding frequency domain gating parameters to the student vision node based on the teaching modality labels, and calculating the interaction synchronization index between the teaching incentive vector and the student response energy data; the convergence gateway includes a communication module and an analysis module; and a feedback device, connected to the convergence gateway, for performing physical feedback when the interaction synchronization index meets abnormal triggering conditions.

[0019] This invention provides a classroom teaching interaction method and system based on edge computing. It has the following beneficial effects:

[0020] 1. This invention can dynamically adjust the passband range of digital filters distributed in the visual nodes of students in the classroom according to the teacher's current teaching status. It can distinguish and filter out random noise caused by students' unconscious small movements at the hardware source according to the current teaching situation, or enhance the signal capture ability of large hand-raising response actions, thereby improving the signal-to-noise ratio and accuracy of interactive feature extraction in complex classroom environments.

[0021] 2. In this invention, the student visual nodes perform high-computational tasks locally, such as dense optical flow calculation, frequency gating, and spatial energy aggregation, and only upload normalized scalar student response energy data to the aggregation gateway. Compared to traditional schemes that upload raw video streams, this avoids the network transmission and storage of privacy data such as student facial images, and decouples the linear dependence between the number of sensors and the computing power load of the central gateway, making the system easy to deploy in large-scale, multi-node classroom environments.

[0022] 3. This invention, by marking hardware timestamps at the physical signal acquisition end and strictly limiting the search range of the cross-correlation coefficient to a window from the minimum physiological latency period perceived by humans to the effective duration of the teaching instruction, enables the system to physically eliminate premature responses that violate causality or simultaneously occurring random overlapping signals. This stringent time-domain constraint ensures that the calculated interaction synchronization index truly reflects the causal logic of student responses triggered by teacher incentives, rather than simply statistical correlation. Attached Figure Description

[0023] Figure 1This is a schematic diagram of the system architecture of the present invention;

[0024] Figure 2 This is a flowchart of the method of the present invention;

[0025] Figure 3 This is a schematic diagram of the time-domain waveforms and phase lag of the teacher's excitation signal and the student's response signal in an application embodiment of the present invention.

[0026] Figure 4 This is a dynamic time-domain distribution heatmap of the normalized cross-correlation function within the time delay interval, according to an application embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram showing the comparison between the interactive synchronization index calculation curve and the traditional energy method, as well as the feedback trigger state judgment, in an application embodiment of the present invention.

[0028] Among them, 100 is the teacher acquisition node; 110 is the audio acquisition module; 120 is the video capture module; 130 is the first processing module; 200 is the student visual node; 210 is the image sensing module; 220 is the second processing module; 300 is the aggregation gateway; 310 is the communication module; 320 is the analysis module; and 400 is the feedback device. Detailed Implementation

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] See attached document Figure 1 This invention provides a classroom teaching interaction system based on edge computing. The system is built on a local area network environment and includes: a teacher acquisition node 100, student visual nodes 200, a convergence gateway 300, and a feedback device 400. The teacher acquisition node 100, student visual nodes 200, and feedback device 400 are all communicatively connected to the convergence gateway 300 via communication links.

[0031] A teacher acquisition node 100 is deployed in the classroom podium area to collect and quantify teaching motivation signals. The teacher acquisition node 100 includes an audio acquisition module 110, a video capture module 120, and a first processing module 130. The audio acquisition module 110 is configured to collect teaching audio signals. The video capture module 120 is connected to a teaching display device to acquire screen image frames. The first processing module 130 is configured to extract features from the teaching audio signals and screen image frames, generate teaching motivation vectors and teaching modality labels, and send the teaching motivation vectors and teaching modality labels to the aggregation gateway 300.

[0032] Student vision nodes 200 are deployed on the ceiling of the classroom, each covering a specific area. Each student vision node 200 includes an image sensing module 210 and a second processing module 220. The image sensing module 210 is configured to acquire continuous video frames. The second processing module 220 has built-in optical flow calculation logic and a variable parameter digital filter. Each student vision node 200 is configured to perform local calculations, sending only student response energy data to the aggregation gateway 300, without uploading raw video images. The second processing module 220 receives frequency domain parameter instructions from the aggregation gateway 300, adjusts the passband frequency range of the digital filter according to the instructions, and performs optical flow feature extraction and frequency domain filtering on the video frames.

[0033] The aggregation gateway 300 is configured as the system's local control center. The aggregation gateway 300 includes a communication module 310 and an analysis module 320. The communication module 310 receives teaching stimulus vectors, teaching modality labels, and student response energy data. The analysis module 320 is configured to establish a mapping relationship between teaching modality labels and frequency domain parameter commands. The aggregation gateway 300 generates corresponding frequency domain parameter commands based on the teaching modality labels and sends them to the student visual nodes 200. The analysis module 320 is also configured to calculate the cross-correlation coefficient between the teaching stimulus vectors and student response energy data, generating an interaction synchronization index.

[0034] Feedback device 400 is connected to aggregation gateway 300. When the interactive synchronization indicator meets the abnormal triggering condition, aggregation gateway 300 controls feedback device 400 to perform physical feedback, which may include:

[0035] Haptic feedback wristband: Worn on the teacher's wrist, when the system detects abnormal interactive synchronization indicators (such as students being seriously distracted), it executes vibration feedback at different frequencies (e.g., short vibrations indicate mild abnormalities, and continuous vibrations indicate severe abnormalities), allowing the teacher to perceive the classroom status without interrupting the teaching rhythm.

[0036] Visual cue lights: Installed next to the clock on the back wall of the classroom, they provide teachers with intuitive cues through LED color changes (a solid green light indicates good synchronization, while a flashing red light indicates a lack of interaction);

[0037] Teaching terminal pop-up: A silent prompt box appears at the edge of the electronic lesson plan screen on the podium.

[0038] See attached document Figure 2 , Figure 2 This is a flowchart of a classroom teaching interaction method based on edge computing according to an embodiment of the present invention. The present invention provides a classroom teaching interaction method based on edge computing, comprising the following steps:

[0039] S100, the teacher acquisition node 100, collects teaching voice signals and screen image frames in real time, generates teaching incentive vectors based on teaching voice signals and screen image frames, and determines the corresponding teaching modality labels;

[0040] S200 and aggregation gateway 300 receive teaching modal tags, determine frequency domain gating parameters according to the preset modal frequency domain mapping relationship, and send the frequency domain gating parameters to student vision node 200;

[0041] S300 and student vision node 200 collect video streams of the student area, perform adaptive frequency domain filtering on the optical flow field of the video stream according to the received frequency domain gating parameters, calculate the student response energy data after noise removal, and upload the student response energy data to the aggregation gateway 300.

[0042] S400 and aggregation gateway 300 perform time-series alignment of teaching incentive vectors and student response energy data, and calculate the cross-correlation number within the sliding time window as an interactive synchronization indicator.

[0043] S500 and aggregation gateway 300 determine whether the interactive synchronization index is lower than the preset threshold. If so, they drive feedback device 400 to issue a prompt signal.

[0044] The following describes each step in more detail with reference to specific embodiments and accompanying drawings.

[0045] See attached document Figure 2 Step S100 involves the acquisition of teaching incentive signals and the quantization of multimodal features. In this embodiment, the teacher acquisition node 100, as the perception front end of the system, is responsible for converting unstructured physical field signals into computable mathematical features, providing a timing reference for subsequent edge-end collaborative control.

[0046] S110 and audio acquisition module 110 perform digitization and energy feature extraction of the teaching speech signal. Considering the non-stationary nature of the teaching speech signal, but its potential to be considered stationary within a short time range (e.g., 10ms to 30ms), the system employs short-time analysis technology to quantify speech intensity. Audio acquisition module 110 discretizes the analog signal using an analog-to-digital converter at a frequency satisfying the sampling theorem (e.g., 44.1kHz). First processing module 130 pre-emphasizes the discrete audio sequence to improve high-frequency resolution and applies a Hanning or Hamming window for frame division to suppress spectral leakage and smooth discontinuities between frames. Based on this, the short-time root-mean-square energy of each frame of audio signal is calculated using the root-mean-square formula. This physical quantity directly reflects the loudness and energy density of the teacher's voice at the current moment, and its expression is:

[0047] ;

[0048] in, Indicates the first The short-time root-mean-square energy of a frame; This represents the frame length, typically the number of sampling points corresponding to a duration of 20ms to 30ms. Represents the discrete sampling point number (or local index) within the current analysis frame; This represents the frame shift step size. To ensure the smoothness of the characteristic curve, it is usually taken as... 50%; These are the amplitude values ​​of the discrete sampling points after preprocessing. For window function coefficients; The short-time average power of the current frame represents the average work intensity of the audio signal within this time window. To eliminate the basis interference of constant background noise generated by air conditioners or equipment fans in the classroom on the energy calculation, the system will perform a pre-calculation analysis. Perform spectral subtraction noise reduction processing.

[0049] S120 and the video capture module 120 process the screen image frame sequence in parallel to quantify the visual information throughput. During teaching, the update rate of screen content typically corresponds to the switching of knowledge points or the generation of blackboard notes. To reduce the dimensionality of data processing and eliminate the interference of color information on content change judgment, the first processing module 130 converts the captured RGB image frames into a grayscale matrix. Based on the principle of pixel domain difference, the system uses the inter-frame difference method to calculate the sum of the absolute values ​​of the pixel grayscale differences between adjacent grayscale images, thereby characterizing the excitation intensity of the visual channel. The video change rate is calculated using the following change rate calculation formula:

[0050] ;

[0051] in, For the first Video change rate at any given time; and These are the width and height pixel counts of the image resolution, respectively. This represents the horizontal coordinate index of the current pixel in the image, with a value ranging from 1 to... ; This represents the vertical coordinate index of the current pixel in the image, with a value ranging from 1 to... ; The coordinates of the current frame The pixel grayscale value at that location; Represents the previous frame (the first frame) (Frame) Images at the same coordinates The pixel grayscale value at that location; a single term in the formula. Defined as "single-pixel temporal differential intensity", it characterizes the degree of brightness jump of a specific pixel on the screen between two adjacent frames; the algorithm ignores static background and can effectively respond to obvious visual changes such as PPT page turning, large mouse movement or electronic whiteboard writing, while being insensitive to weak noise caused by wire transmission.

[0052] S130, the first processing module 130 constructs a multimodal fusion teaching stimulus vector. Due to the short-time root-mean-square energy of the audio signal... (Sound pressure level related) and video change rate (Related to pixel throughput) There is a fundamental difference in physical dimensions, and direct fusion would lead to the larger value dominating the feature. Therefore, the system adopts a dynamic extremum normalization strategy based on a sliding time window to map the two to a unified dimensionless interval. The synthesis logic of the teaching incentive vector is as follows:

[0053] ;

[0054] in, The normalized teaching incentive vector has a numerical range of [value missing]. ; and These are the maximum audio energy and the maximum video change rate within a preset time window (e.g., 300 seconds) preceding the current moment, respectively. By dynamically updating the denominator, the system can adapt to the volume reference of different teachers and the brightness differences of different display devices. It is a very small positive number (e.g., 1×10). -6 This is used to prevent calculation errors caused by prolonged silence or static screen, where the denominator approaches zero. These are the modal weighting coefficients, and their value range is set to... This coefficient can be preset according to the course type, for example, in rewritten humanities courses. Take the larger value, and in the re-derived science blackboard course Take the smaller value.

[0055] S140, the first processing module 130 determines the teaching modality label based on a logical threshold using bimodal features. The system divides classroom states into three basic modalities according to the psychological characteristics of teaching behavior, and determines the teaching modality label by comparing feature values ​​with thresholds in real time. The specific classification logic is as follows:

[0056] when and When this occurs, it is determined to be either blackboard writing or silent mode. This state corresponds to the teacher stopping lecturing to write, or leaving blank space for students to think; both audio and video signals are at a low energy level.

[0057] when or When, it is determined to be a high-frequency interaction mode ( This state corresponds to the teacher speaking enthusiastically, rapidly switching courseware, or asking questions; at least one of the audio-visual signals is in a high-energy state.

[0058] Intermediate states other than the two cases mentioned above are classified as teaching modes. At this time, the teacher spoke at a steady pace, and the content on the screen updated at a regular rhythm.

[0059] In the above judgment logic, The threshold for determining audio silence is defined as follows: ,in and These represent the mean and standard deviation of the ambient background noise energy, respectively. The confidence coefficient is used to ensure effective filtering of ambient noise such as air conditioner sounds; in this embodiment, it is preferably set to 3 ( ),in accordance with The criteria are designed to ensure that the interference of steady-state environmental noise such as air conditioning fan noise and electrical noise on the silence determination can be effectively filtered out.

[0060] The threshold for determining high-frequency interaction is defined as follows: ,in and These represent the mean and standard deviation of the vocal energy of a teacher's normal lecture, respectively. The intensity factor is preferably set to 2.0 to 2.5 in this embodiment to accommodate the volume habits of different teachers.

[0061] Meanwhile, for video modalities, the system uses similar logic to set the video silence determination threshold ( ) and video high-frequency determination threshold ( ). This corresponds to the slight back and hand movements a teacher makes while writing on the blackboard, and This corresponds to the upper limit of the range of motion of a teacher when their body language is rich.

[0062] Based on the aforementioned adaptive threshold, the first processing module 130 executes the following classification logic:

[0063] When the short-time root mean square energy is calculated in real time And video change rate When the system determines that the teacher is currently in a writing or silent mode, it assumes that the teacher is writing or thinking and has not issued any effective interactive commands.

[0064] when or When the system determines that the current state is a high-frequency interaction mode, it assumes that the teacher is conducting high-intensity questioning or physical demonstration.

[0065] If neither of the above two conditions is met (i.e., it is in an intermediate state), it is determined to be a teaching mode, representing a conventional knowledge transmission process.

[0066] Generated teaching modality tags Accompanying teaching incentive vector It is sent to the aggregation gateway 300 as input parameters for subsequent control loops.

[0067] See attached document Figure 2 Step S200 involves feedforward control of frequency domain gating parameters based on mode mapping. In this embodiment, the aggregation gateway 300 acts as the collaborative control center of the system. Its core logic lies in using the teaching scenario identified in the preceding steps as prior knowledge to reconstruct the physical parameters of the sensing nodes in the downlink, including the following sub-steps:

[0068] S210, the aggregation gateway 300 establishes a logical mapping between discrete teaching modalities and continuous frequency domain parameters. The aggregation gateway 300 internally stores a pre-set mapping table between modalities and frequency domain parameters. This mapping relationship is constructed based on the biomechanical characteristics of human movement and is used to define the spectral distribution range of effective student behaviors in different teaching scenarios. The system defines a mapping operator based on the currently received teaching modal label. The corresponding bandpass filter parameters are indexed, and their mathematical expression is as follows:

[0069] ;

[0070] in, For the first The target frequency domain gating interval at any given time; Defined as a frequency domain parameter tuple (frequency domain gating parameter), it contains a pair of frequency boundary values ​​that determine the filter bandwidth; and These are the lower and upper cutoff frequencies of the filter, respectively, in Hertz (Hz). This represents a preset mapping operator between modes and parameters, used to establish the correspondence between discrete labels and continuous frequency parameters.

[0071] S220, for whiteboard writing or silent mode ( The system will focus on the high-frequency micro-motion band, with specific parameters configured as follows: The physical significance of setting this frequency band is that, in this scenario, effective learning behaviors manifest as high-frequency handwriting movements and rapid head scanning caused by the eyeballs switching between the blackboard and the textbook. The motion frequency energy of these movements is mainly concentrated in... In addition, the high-pass characteristics of this frequency band can effectively filter out unconscious low-frequency shaking of the student's torso, ensuring that the system only responds to micro-movements of "cognitive activity".

[0072] S230, for the teaching modality ( The system switches to the low-frequency attitude stabilization band, with specific parameter configurations as follows: .

[0073] This setup is designed to monitor students' concentration in a quasi-static state. According to human posture control theory, the human body produces low-frequency postural swaying coupled with breathing frequency when maintaining seated balance; this swaying is typically below 0.5 Hz. Within this frequency range, if the detected energy contains only regular breathing fundamental waves, it indicates that the student is in a focused, seated state; if there is a sudden increase in energy, it corresponds to restless body twisting or large-amplitude adjustments in posture.

[0074] S240, targeting high-frequency interaction modes ( The system enables the intermediate frequency limb interaction band, with specific parameters configured as follows: This frequency band covers the main energy distribution of macroscopic body language. According to statistics, the frequency range corresponding to the natural cycle of interactive actions such as students raising their hands, nodding in confirmation, or turning their bodies is between 0.5Hz and 2.0Hz. By selecting this frequency band, the system can maximize the gain of the interactive signal while suppressing high-frequency noise from the image sensor and extremely low-frequency drift caused by slow changes in ambient light.

[0075] The S250 aggregation gateway 300 performs protocol encapsulation and timing alignment of control commands. To overcome the uncertainty of wireless network transmission, the system adopts a time-triggered synchronization control mechanism. The communication module 310 encapsulates the parsed frequency parameters into a custom UDP packet (a connectionless, message-oriented transport layer protocol) and inserts a command sequence number and an effective trigger timestamp into the packet header. The calculation logic for this timestamp is as follows:

[0076] ;

[0077] in, The current system absolute time of the gateway; To buffer network security latency, its value is set based on the maximum one-way transmission latency within the local area network, typically ranging from 50ms to 100ms, to ensure that the command has stably reached all node buffers before it takes effect.

[0078] S260, the student vision node 200 receives instructions and executes a time-locked parameter switch. To ensure that sensors distributed in different locations in the classroom switch perception modes at the same physical instant, the system employs a time-triggered mechanism with a delayed effect.

[0079] Specifically, the communication module 310 transmits the frequency domain gating parameters via a UDP multicast channel. and target effective timestamp The data packets are broadcast to the entire network. After receiving and verifying the data packets, each student vision node 200 does not immediately perform configuration updates. Instead, it adds the task to the waiting queue and continuously compares it with its local high-precision system clock (which has been aligned with the gateway via the PTP protocol). It is worth mentioning that this local clock has been pre-synchronized with the aggregation gateway at the sub-microsecond level via IEEE 1588 PTP (Precision Time Protocol).

[0080] When local time arrives At a specific time (e.g., 200ms after the current time), all nodes synchronously trigger an interrupt. At this time, the second processing module 220, based on the received... and The value is immediately determined based on the received data. and The numerical values ​​are calculated in real time using the bilinear transform method to determine the coefficients of the digital filter.

[0081] The specific process of coefficient calculation is as follows:

[0082] First, to overcome the frequency winding effect caused by the bilinear transform, the module performs pre-distortion processing on the target cutoff frequency. Taking the calculation of the cutoff angular frequency of a low-pass filter as an example, let the sampling frequency be... The target simulation cutoff frequency is (Right now The simulated angular frequency after pre-distortion... The calculation is as follows:

[0083] ;

[0084] Subsequently, based on the preset normalized simulated low-pass prototype transfer function (e.g., a second-order Butterworth filter), using the bilinear transform formula Will Domain mapping to The domain is used to derive the discrete-time transfer function. :

[0085] ;

[0086] Through the above calculations, the second processing module 220 updates the feedback coefficients of the IIR filter in real time. With feedforward coefficients This global time-based synchronous switching mechanism ensures that all visual nodes in the classroom complete the characteristic transition from silent mode filter to interactive mode filter at the same microsecond level, effectively avoiding spatiotemporal data alignment deviations caused by differences in processing speed of each node or network jitter, thereby ensuring the authenticity of the spatial distribution of the subsequently generated energy heatmap.

[0087] Step S300 involves adaptive optical flow filtering and energy aggregation processes at the edge. In this embodiment, the student visual node 200 utilizes local embedded computing power to perform high-density visual flow analysis. This process aims to transform video frame sequences that originally contained privacy information into desensitized physical fields that reflect microscopic motion energy, thereby providing a clean raw data foundation for subsequent frequency domain gating.

[0088] S310, the second processing module 220 performs dense optical flow calculations based on polynomial expansion.

[0089] First, to eliminate the interference of color information on motion estimation and reduce computational bandwidth, the second processing module 220 converts the acquired RGB image into a single-channel grayscale matrix and applies Gaussian smoothing to suppress high-frequency thermal noise generated by the image sensor. Based on this, the system employs the Farneback dense optical flow algorithm to perform polynomial modeling of the local pixel neighborhood. For each pixel coordinate in the image... The system uses a quadratic polynomial to fit its brightness distribution surface, and this local signal model is expressed as:

[0090] ;

[0091] in, This represents a local coordinate vector with the pixel center as the origin. This represents the fitted pixel brightness value; It is a 2×2 symmetric matrix that represents the second-order curvature features of the local image texture; It is a 2×1 vector that represents the direction of the brightness gradient in a local image; It is a local average brightness scalar; The transpose operator for a matrix or vector; Defined as the "quadratic curvature term," it describes the second-order nonlinear variation characteristics of image brightness in a local region; Defined as a "linear gradient term," it describes the first-order linear variation of image brightness within a local region. The second processing module 220 performs weighted least-squares fitting on the image, which is then compared to the previous frame image. and the current frame image Solve for the corresponding set of coefficients for each pixel in the dataset. and This transforms the spatial variation of pixel brightness into a mathematical feature description.

[0092] Based on the assumption of constant brightness—that is, after an object in an image undergoes displacement within a very short time, its brightness structure remains unchanged—a relation can be established. By comparing the polynomial coefficients of the two frames, the second processing module 220 calculates the instantaneous displacement vector of each pixel. The displacement vector is calculated based on the following linear equation:

[0093] ;

[0094] in, Let be the displacement vector to be determined. ; This is the average value of the curvature matrix at corresponding positions in the two frames of images; and These represent the local brightness gradient direction vectors at corresponding positions in the previous frame and the current frame, respectively.

[0095] To improve computational robustness and prevent damage caused by local texture loss (i.e.) To address computational divergence caused by singularities, the system introduces a neighborhood-weighted averaging mechanism during the solution process and adds a Tikhonov regularization term during matrix inversion. ,in For small regularization parameters; Represents a 2×2 identity matrix; Defined as a "regularized mean curvature matrix", it ensures that displacement calculations maintain numerical stability even in flat regions (regions with indistinct textures).

[0096] Finally, the second processing module 220 outputs a dense motion vector field with the same resolution as the original image. Each element of this vector field contains a horizontal component. With vertical component It accurately describes the motion state of each pixel in the field of view at the current moment.

[0097] S320 and the second processing module 220 construct a variable parameter digital filter and perform dynamic updates.

[0098] For the dense motion vector field output by the previous step The system is divided into its horizontal components. and vertical components Instantiate two structurally identical but logically independent bandpass filters. The core mathematical model of the bandpass filter adopts a second-order direct type II or double second-order belated structure, which has good numerical stability under fixed-point operations. For each pixel coordinate... Its time-domain difference equation is described as follows:

[0099] ;

[0100] in, Indicates the current number The original optical flow component of the frame (i.e. or ); Indicates the filtered output components; To input historical status; To output feedback status; , , Forward coefficients; , The feedback coefficient is used to weight and correct the current observation value by using the motion state of the past moment. The feedback loop retains the oscillation signal of a specific frequency (such as hand waving) while suppressing DC drift (such as gradual changes in illumination) and high-frequency noise (such as thermal noise of image sensor).

[0101] When a frequency domain command is received from the aggregation gateway 300 At that time, the system does not rely on table lookups, but instead generates the corresponding set of numerator and denominator coefficients for the digital filter through real-time calculation. This supports continuous, stepless adjustment of frequency parameters. The calculation process follows a mapping logic from the analog prototype to the digital domain:

[0102] First, based on the sampling frequency of image acquisition The target simulated cutoff frequency is pre-distorted to obtain the digital domain angular frequency:

[0103] ;

[0104] ;

[0105] in, and These represent the lower cutoff angular frequency and the upper cutoff angular frequency in the digital domain after pre-distortion processing, respectively. and These are the lower cutoff frequency and the upper cutoff frequency of the filter, respectively.

[0106] Subsequently, based on the Butterworth bandpass prototype design and combined with the bilinear transformation formula, the coefficient expression was derived. The coefficients in the denominator were then used as the basis for the expression. Taking the normalized coefficient as an example, the intermediate variables for its calculation are as follows:

[0107] ;

[0108] in, The first term coefficient (normalization factor) of the denominator polynomial of the filter transfer function is represented; the remaining coefficients are based on... Normalization is performed. By running this mathematical mapping algorithm in real time at the edge, the system eliminates the need to store massive preset coefficient tables and enables the filter passband to accurately match arbitrarily set teaching modal boundaries.

[0109] The command's preset trigger timestamp is reached at the local system time. At that time, the second processing module 220 triggers a high-priority interrupt, transferring the newly calculated coefficients. The coefficients are written into the coefficient register of the DSP or FPGA. To prevent sudden changes in coefficients from causing the filter output to diverge or produce significant ringing effects, the system performs a decay or reset operation on the historical state buffer at the moment of update. This step ensures that the filter transitions smoothly between different teaching modes (such as switching from 2-5Hz for writing focus to 0.5-2Hz for interactive focus).

[0110] In this embodiment, the motion vector field after frequency domain filtering still contains directional and spatial distribution information. In order to facilitate the aggregation gateway 300 to perform group collaborative analysis under low bandwidth, the second processing module 220 performs a dimensionality reduction operation to compress the high-dimensional matrix data into a single physical quantity—student response energy.

[0111] S330 and the second processing module 220 perform energy aggregation, spatial masking, and normalized dimensionality reduction.

[0112] To compress high-dimensional matrices into scalars that facilitate collaborative analysis, the system first bases its approach on the kinetic energy theorem. The system focuses on the intensity of motion rather than its direction. Therefore, the second processing module 220 processes the horizontal component output after the IIR filter. and vertical components Perform a summation of squares to obtain the scalarized motion intensity of the pixel at the current moment. The calculation process is as follows:

[0113] ;

[0114] in, Representing coordinates Instantaneous pixel power at the location; and These are the horizontal and vertical components after bandpass filtering in the aforementioned steps.

[0115] Subsequently, to obtain macroscopic indicators reflecting the overall student status, the system needs to spatially integrate the power of all pixels within the field of view. During this process, to eliminate inherent thermal noise from the sensor and interference from non-human areas (such as desktop glare), the system introduces an energy threshold mechanism. The aggregation calculation logic is as follows:

[0116] ;

[0117] in, Indicates the first The original aggregated total energy of the frame; and These are the width and height in pixels, respectively, representing the image resolution. and These represent the horizontal and vertical coordinate indices of the pixel, respectively. The noise floor filtering threshold is determined by the dark current statistics when the system is idle, and is used to zero out weak random noise. This is an indicator function, which takes the value 1 when the condition within the parentheses is met, and 0 otherwise. Through this spatial integration process, the microscopic motion components dispersed in different parts of the student's body, such as the hands and head, are superimposed into a macroscopic physical quantity that can characterize the overall limb activity.

[0118] Finally, considering that the target's proportion in the image will vary depending on the installation distance of different student nodes (i.e., the near-large and far-small effect), we directly use... This can lead to spatial inequity in data weights. Therefore, the system incorporates pre-calibrated depth coefficients. With the total number of effective pixels Normalization is performed. Therefore, the system incorporates pre-calibrated depth coefficients. With the total number of effective pixels Normalization is performed. This embodiment uses a pixel-level depth-weighted matrix as the implementation of the depth coefficients. This matrix is ​​adjusted during the device installation phase based on the camera's mounting height. and pitch angle Generation, for an image coordinate system with a ordinate of... The pixel row, and its corresponding depth weight The geometric projection compensation relationship is approximately satisfied:

[0119] ;

[0120] in Image height, This is the perspective correction factor (typical values ​​can be 0.5 to 1.5).

[0121] Final student response energy data The following weighted normalization formula is used to solve for the problem:

[0122] ;

[0123] in, That is, the first Each student node at time The output is the student response energy, a scalar data. For the above steps to satisfy The total number of pixels represents the area of ​​the moving region; To prevent division by zero of small constants. This scalar data. Redundant information such as direction, texture, and ambient lighting has been removed, and only the motion intensity characteristics under a specific frequency band have been retained. Its data volume is only 4 bytes, which is the size of a floating-point number, thus reducing the network load for transmitting data back to the aggregation gateway 300.

[0124] S340 is a privacy protection and transmission mechanism based on unidirectional data flow.

[0125] In this embodiment, the student visual node 200 is designed at the physical level to allow only high-dimensional image data to flow into the local volatile memory, while only allowing low-dimensional abstract data to flow out to the network layer, thereby cutting off the risk of privacy leakage at the source.

[0126] Generate scalar form of student response energy Subsequently, the communication module 310 extracts only the floating-point value and combines it with the current device's unique identifier and timestamp sequence to construct a telemetry data packet. The payload of this data packet does not contain any form of pixel matrix, texture features, or JPEG / H.264 encoded stream. To prevent data eavesdropping within the local area network, the communication module 310 encrypts the payload using a lightweight encryption protocol and sends the encrypted scalar data to the aggregation gateway 300 via UDP. Because the amount of transmitted data is compressed to the byte level (e.g., each node generates only about 1KB of traffic per second), this mechanism reduces the bandwidth consumption of the campus network.

[0127] The system allocates a fixed-length circular buffer in RAM to temporarily store image frame sequences. Once the current frame completes optical flow calculation and generates the corresponding energy value, or the frame moves out of the time window range, the second processing module 220 unlocks the memory block. When the next frame image acquisition is triggered, the DMA controller directly overwrites the new sensor data to this physical memory address. For raw pixel arrays involving facial or body features, their lifespan is strictly limited to a millisecond-level calculation window, and no read / write operations are involved in non-volatile storage media (such as Flash or SD cards) throughout the process, ensuring that the data is completely lost and unrecoverable after power failure.

[0128] From an information theory perspective, the aforementioned energy aggregation process constitutes an irreversible mapping function with an extremely high compression ratio. Due to the energy of student response Reverse engineering the original image matrix is ​​a severely underdetermined problem, with its solution space approaching infinity. Even if an attacker intercepts the uploaded energy data, they cannot mathematically reconstruct any visual image with semantic information, ensuring that the system meets the quantitative requirements for interactive teaching while strictly complying with data privacy protection requirements.

[0129] Step S400 involves the temporal cross-correlation coupling analysis of stimulus and response. In this embodiment, the core challenge of the system lies in how to accurately quantify the causal time delay between the teacher's teaching behavior (as the stimulus signal) and the responses of student groups distributed in different spatial locations (as the response signal). Due to the unpredictable jitter in traditional network transmission, this solution abandons the traditional processing method based on the data arrival time, and instead constructs a global synchronization time base based on the IEEE 1588 Precise Time Protocol (PTP), and restores the true temporal relationship of multi-source data streams at the moment of generation at the physical level, thereby extracting causal coupling features within the limited physiological time domain.

[0130] S410 constructs a hardware-assisted high-precision network-wide synchronization time base and data alignment mechanism.

[0131] First, the aggregation gateway 300, acting as a highly stable master clock, establishes a synchronization domain with the student visual node 200, which acts as a slave clock. The system utilizes the hardware tagging mechanism unique to the PTP protocol to periodically perform message exchanges to eliminate the uncertainty of protocol stack dwell time. During this process, based on the prior assumption of symmetrical physical transmission delays between the uplink and downlink on the network, the system calculates the deviation of the local clock relative to the global time base. :

[0132] ;

[0133] in, to These are the physical layer transmission and reception times of synchronization messages and delay request messages, respectively. All of the above timestamps are latched by the network interface controller the instant the start-of-frame delimiter is detected at the physical medium layer, thus ensuring microsecond-level time measurement accuracy.

[0134] get Subsequently, considering that optical flow calculations are highly dependent on the continuity of the time differential term, this embodiment strictly prohibits direct write operations to the system time register (i.e., time jumps are prohibited) to avoid computational divergence. The system instead employs a frequency fine-tuning strategy, dynamically adjusting the counting step size of the hardware timer to smoothly eliminate errors. This process is implemented through a discrete proportional-integral (PI) controller.

[0135] ;

[0136] in, Indicates the first Frequency adjustment amount within each adjustment cycle; Indicates the current index of the discrete control cycle; Indicates the first The deviation of the current clock relative to the global time base obtained from this measurement; This represents the cumulative sum of historical deviations; Indicates the first The deviation of the current clock relative to the global time base obtained from this measurement; and These are the proportional gain coefficient and the integral gain coefficient, respectively. This closed-loop control ensures that, while maintaining a monotonically increasing clock speed, the synchronization error between each node's clock and the master clock converges to a certain value. Within.

[0137] With the guarantee of network-wide time base alignment, the system executes the "acquisition and labeling" data encapsulation logic. For the audio and video streams from the teacher's end and the energy scalar data from the student nodes, the acquisition driver reads the tamed local PTP time within a microsecond delay triggered by a hardware interrupt (such as the end of sensor exposure Frame_End or the completion of ADC conversion) and encapsulates it into a 64-bit timestamp. Write this timestamp into the packet header. This timestamp represents the actual objective moment the physical event occurred, independent of the time it took to reach the gateway via network transmission, thus completely decoupling measurement time from network status at the data source. A jitter buffer based on a minimum heap structure is established on the aggregation gateway 300 side, and jitter is only applied within a certain time window of the buffer. Only when the data frame sequence is complete is the aligned time slice sent to subsequent calculations. This mechanism effectively reconstructs asynchronously arriving network data packets into a strictly synchronous time-series signal stream, laying the time-domain foundation for subsequent high-precision causal inference.

[0138] S420, Signal preprocessing and normalized cross-correlation calculation based on asymmetric sliding window.

[0139] To capture non-stationary teaching interactions, the system is set to a length of [length value missing]. A sliding window with a time interval of 5 seconds (e.g., 5 seconds) and increments of 5 seconds. (For example, 1 second) Proceeding forward along the time axis. It is worth noting that, considering the time-shifting operation required for cross-correlation calculations, this embodiment employs an asymmetric data reading strategy: for each analysis window, the system extracts a length of... Teacher incentive sequence Simultaneously extract length of Student response sequence ,in The maximum physiological response delay is preset. This oversampling mechanism ensures that sufficient raw data is always available for computation when calculating arbitrary time delays, avoiding the edge effect attenuation caused by zero padding in traditional algorithms.

[0140] Subsequently, the system performs centering processing on the sequences within the window, i.e., subtracting their respective arithmetic mean. The physical significance of this step is to remove the DC component from the signal, eliminate the constant deviation introduced by sensor static operating point drift or different ambient lighting references, and ensure that the correlation calculation is entirely based on the AC fluctuation characteristics of the signal. The processed sequence is represented as follows:

[0141] ;

[0142] ;

[0143] in, and These represent the arithmetic mean of the original sequence; and These are the original sampled values; and For the processed DC-neutralized values

[0144] Furthermore, the system operates within a time delay range that conforms to the laws of human cognitive response. Within this process, a traversal search is performed on the lag time τ. For each candidate... Value, normalized cross-correlation function The specific calculation logic is as follows:

[0145] ;

[0146] in, This represents the total number of samples within the calculation window (i.e., the sequence length). The loop index variable represents the discrete summation operation; Indicates the time lag of the attempt, in frames or time steps; This indicates that in the student response energy sequence, relative to Lag The corresponding sampling point values ​​for each time step; This represents the numerical stability constant.

[0147] The denominator term in the formula not only standardizes the result to The role of intervals makes teaching activities of different intensities comparable; more importantly, it introduces a numerical stability constant. (Recommended value: 1) This parameter is used to prevent division overflow or NaN errors when the classroom is completely still (signal variance approaches 0), thus ensuring the numerical stability of the embedded algorithm under all operating conditions.

[0148] S430, Extreme value search and index time-domain smoothing based on physiological time-sensitivity constraints.

[0149] To ensure that the extracted synchronization features conform to the objective causal logic of "teacher incentives triggering student responses", the system abandons the full-time domain search strategy and instead strictly limits the optimization space of the optimal time delay point to a preset physiological effective window (time delay interval). Inside. The boundary parameter settings for this window have a clear physiological basis:

[0150] The Lower World (for example, set to) It is determined based on the minimum physiological latency period from human perception of visual and auditory stimuli to neuromuscular action, and is used to eliminate premature responses or simultaneous random coincidences that violate causality at the physical level.

[0151] Upper Realm (for example, set to) The duration of instruction is determined based on the effective retention time of the instruction in the student's short-term working memory, and is used to exclude outdated responses that are irrelevant to the current instruction.

[0152] Aggregation Gateway 300 is only available here. Time lag during traversal within the interval We find the maximum value of the normalized cross-correlation function to obtain a synchronization index that can truly reflect the quality of teacher-student interaction.

[0153] Under these physical constraints, the system solves for the global maximum value of the normalized cross-correlation coefficient and its corresponding time-shift index:

[0154] ;

[0155] ;

[0156] in, Indicates the optimal time-shift index; This represents an operator for optimizing the independent variable, used to return the result that optimizes the objective function. The variable that reaches its maximum value The value of ; Indicates the preset time delay interval; This represents the peak cross-correlation coefficient.

[0157] This constrained optimization mechanism effectively shields against spurious correlations with long time delays caused by random coincidences outside the window, thus ensuring the authenticity and uniqueness of the measurement results.

[0158] Considering the presence of numerous non-instructional random movements in a real classroom environment, the system introduces a validity judgment threshold based on the signal-to-noise ratio. This threshold is obtained by performing a background noise calibration process during non-teaching periods (such as breaks between classes), and is typically taken as the mean of the cross-correlation coefficient distribution plus three standard deviations. Based on this threshold, the system calculates the instantaneous synchronization state. Only when The value is retained if the condition is met, otherwise it is set to zero. The technical effect of this decision logic is to achieve "denoising gating" of the signal, preventing low-quality data from polluting the evaluation model.

[0159] Finally, in order to eliminate transient motion jitter, the system performs instantaneous synchronization. Perform an exponentially weighted moving average (EWMA) filter to generate the final interactive synchronization indicator. :

[0160] ;

[0161] in, Indicates the first The smooth interactive synchronization index output at each moment; Represents a discrete time series index; Indicates the first The instantaneous synchronization state after the threshold decision at a given moment; Indicates the first Smooth interactive synchronization metrics in real time; This represents the smoothing factor (typically ranging from 0.1 to 0.25). This embodiment uses this time-domain integration mechanism to transform discrete, high-frequency fluctuating correlation coefficients into continuous, stable indicators of teaching quality.

[0162] S440, Multi-dimensional Interactive State Mapping and Physical Meaning Analysis.

[0163] After obtaining the quantitative indicators, the system combines them with the average energy level of the student population during the same period. Perform multi-dimensional state space mapping.

[0164] when In the high range ( When the classroom system enters a stable master-slave control mode, the teaching instructions issued by the teacher are transformed into consistent behavioral outputs by the student group with extremely low information loss and a certain time delay, corresponding to a highly efficient knowledge transfer process.

[0165] when In the low range ( When this happens, the system further subdivides based on the energy dimension: if At the same time, it is below the silent threshold. Passive silence is judged as a sign of a dull classroom atmosphere or cognitive overload; if If the behavior is chaotic and disorderly, it indicates that although the students are active, their behavior is unrelated to the teacher's instructions, corresponding to a scenario of loss of control.

[0166] In addition, the system uses the optimal time-shift index The distribution range distinguishes the depth of interaction: when When it is determined to be "reflexive synchronization", it corresponds to mechanical training or conditioned reflex; when The time was determined to be "cognitive synchronization", which corresponds to deep interaction that has undergone logical processing in the prefrontal cortex.

[0167] In summary, this embodiment uses... , and The constructed three-dimensional feature vectors visualize the abstract waveform data into an interpretable teaching state map.

[0168] Step S500, as the physical execution link of the system, aims to implement the interactive synchronization indicators output in step S400. This constructs a closed-loop mapping from the data space to the physical sensory space. In this embodiment, the main challenge facing the system is how to ensure effective information transmission while avoiding abrupt physical stimuli that could disrupt the normal flow of teaching. To this end, the aggregation gateway 300 adopts a state decision model based on hysteresis characteristics and a pulse width modulation driving mechanism to provide implicit state-aware feedback to the teacher.

[0169] S510. Construct a feedback triggering strategy based on time-domain hysteresis and persistence verification.

[0170] Considering that normal brief intervals in classroom teaching (such as erasing blackboard writing or changing teaching aids) can cause instantaneous fluctuations in indicators, using only a single threshold for instantaneous comparison can easily lead to frequent and meaningless false alarms. Therefore, this embodiment abandons simple comparator logic and instead adopts a dual-threshold state machine model with hysteresis characteristics.

[0171] The aggregation gateway 300 internally maintains a length of A state FIFO queue is used to store a sequence of interaction synchronization indicators from the most recent period (e.g., 30 seconds). The system defines two key thresholds: a trigger threshold and a trigger threshold. (e.g., 0.3, representing the upper limit of weak interaction states) and recovery threshold (e.g., 0.6, representing the lower limit of normal regression interaction), and must satisfy... To form a stable hysteresis interval.

[0172] The system executes the following persistent verification logic to determine the current feedback trigger state. :

[0173] ;

[0174] in, This is an indicator function that takes the value 1 when the condition inside the parentheses is met, and 0 otherwise; The number of frames corresponding to the verification window; The minimum percentage of abnormal frames required to trigger (e.g., set to 0.8, meaning 80% of the time is spent in low interaction).

[0175] The physical significance of this judgment logic lies in establishing an asymmetric entry and exit mechanism: the system only confirms the entry into an inefficient teaching state and sets the trigger flag when the intensity of classroom interaction remains statistically significant and persistently low over a long time window; once the indicator rises back to a high level, the system immediately resets. This filtering characteristic of being difficult to enter and easy to exit effectively suppresses system jitter caused by occasional noise, ensuring the seriousness of feedback intervention.

[0176] S520: Generate an adaptive drive coding signal based on severity grading.

[0177] When feedback is triggered When enabled, the aggregation gateway 300 does not send a single switch signal, but instead sends a signal based on the current... The degree of deviation from the threshold is used to calculate the feedback strength in order to achieve the simulation quantization of physical perception.

[0178] Drive strength coefficient The calculation of (corresponding to PWM duty cycle) follows the inverse proportional mapping principle:

[0179] ;

[0180] in, This represents a numerical limiting function (or saturation function) used to forcibly restrict the input value to a range. Inside; This represents the current interactive synchronization indicator, i.e., the real-time state quantity input by the system; Indicates the trigger threshold; This is the intensity gain factor (typical value 1.0). Set a lower limit for the tactile perception threshold (e.g., 20%) to ensure that the feedback signal can overcome the damping of clothing and be perceived by the skin; To limit the maximum intensity (e.g., 80%), prevent excessive stimulation from causing a startle or overheating of the equipment; As a non-zero denominator term, it ensures the stability of the calculation.

[0181] Subsequently, the gateway constructs a data frame containing control commands via Bluetooth Low Energy (BLE) or a proprietary wireless protocol. This data frame uses a compact TLV (Tag-Length-Value) format, and the payload includes:

[0182] Action type (vibration / light effect), frequency parameters and the calculated duty cycle This encoding method transforms physical feedback from simple binary reminders into a sophisticated tactile / visual language that carries information about the current level of classroom silence.

[0183] S530 performs non-intrusive tactile or visual physical feedback.

[0184] The feedback device 400 (specifically, a smart bracelet worn on the teacher's wrist or a prompt light placed on the edge of the podium) receives a drive signal and executes a microscopic physical action, aiming to transmit information in the form of subconscious perception. In tactile feedback mode:

[0185] The linear resonant actuator (LRA) inside the device operates according to the received signal. and The vibration is initiated based on the parameters. To achieve a "non-intrusive" technical effect, this embodiment preferably sets the vibration frequency to... Set at to The frequency band is selected based on biophysical principles and falls precisely within the sensitive response range of Meissner bodies in human skin. This allows the wearer to experience tactile alertness with minimal energy consumption without eliciting auditory attention (i.e., without mechanical buzzing). Simultaneously, the vibration waveform is modulated into an intermittent pulse sequence, rather than continuous vibration, to avoid sensory adaptation and subsequent desensitization in the human nervous system.

[0186] In visual feedback mode:

[0187] The LED array at the edge of the device performs brightness variations based on sinusoidal modulation. Instantaneous brightness. The governing equation that varies with time is:

[0188] ;

[0189] in, Maximum brightness is the reference. Indicates the driving strength coefficient; Indicates the current time Breathing modulation frequency (usually set to) to This is used to set the pace of the brightness transition. Experiments show that the human eye has a keen perception of such low-frequency brightness changes at the edge of the visual field, but it does not produce the forced visual deprivation or cause photosensitivity discomfort like high-frequency flicker. Teachers only need to use their peripheral vision to perceive the status cues, thus adjusting the teaching pace independently without interrupting the flow of instruction or distracting students, achieving the best balance between technological intervention and teaching fluency.

[0190] To better illustrate the technical solution of the present invention and verify its technical effects, a specific application embodiment based on a real teaching scenario is provided below, along with a comparison of experimental verification data based on this embodiment.

[0191] Specific application example: Interactive monitoring in K-12 STEM experimental classrooms

[0192] 1. Scene Construction and Hardware Deployment

[0193] In this application example, the test environment is selected as a standard 80m room. 2 A middle school STEM physics lab. The classroom contains 30 students (divided into 6 lab groups) and 1 teacher.

[0194] Data acquisition end:

[0195] Visual acquisition equipment: Four GigEVision industrial cameras supporting the IEEE 1588 PTP protocol are deployed in a grid pattern on the classroom ceiling. These cameras are equipped with global exposure CMOS sensors to eliminate the rolling shutter effect when students are moving quickly, and are connected to the network via a PoE interface.

[0196] Audio / feedback equipment: Teachers wear a high-fidelity wireless lavalier microphone and a programmable smart bracelet with a built-in linear resonant actuator on their wrist.

[0197] Computing end: An embedded edge computing gateway is connected to the aforementioned industrial cameras via a gigabit Ethernet switch, and is responsible for the PTP master clock synchronization and real-time algorithm processing of the entire network.

[0198] 2. Interactive event handling process drill

[0199] Phase 1: Intensive Synchronized Teaching (Time: 10:05-10:15)

[0200] The teacher is demonstrating a simple pendulum motion experiment, using hand gestures to give instructions: "Look, what happens to the period when the pendulum length shortens?" (Excitation signal) (A clear envelope appears).

[0201] Response analysis: The industrial camera captured the students raising their heads and turning towards the demonstration table almost simultaneously, and there was subsequent data recording of these actions.

[0202] Algorithm output: Calculates the value at... Peak cross-correlation coefficient at (visual and auditory conduction delay) .

[0203] State Mapping: Interactive Synchronization Metrics After smoothing, the value stabilizes above 0.8. The system determines this as "strong synchronization" and does not trigger feedback.

[0204] Phase Two: Implicit Classroom Dispersion (Time: 10:25-10:35)

[0205] The teacher begins writing formula derivations on the blackboard for an extended period, explaining with their back to the students. At this time, some students start looking down at their phones or chatting in their groups (non-instructional actions).

[0206] Response analysis: Although there is still motion energy in the classroom (students are moving), the optical flow energy captured by the camera is randomly and chaotically distributed, completely decoupled from the rhythm of the teacher's speech.

[0207] Algorithm output: Cross-correlation scans are displayed in the time delay interval. There is no obvious peak value. It dropped to around 0.15. At the same time, the background energy exceeded the silence threshold. .

[0208] State mapping: The system determines the state to be "chaotic and disordered".

[0209] Feedback execution: After 30 seconds of continuous verification, the flag bit is triggered to flip. The drive duty cycle is calculated. Teacher's wristband The frequency emits a "breathing" vibration. Upon sensing this, the teacher immediately turns around and asks a question, regaining the students' attention. The temperature then rises, and the feedback automatically stops.

[0210] Experimental verification and effect comparison

[0211] To verify the superiority of the present invention over traditional technologies, we conducted data collection and comparative testing in the above-mentioned experimental environment for two weeks (a total of 20 class hours).

[0212] 1. Experimental setup

[0213] Benchmark values: Three senior teaching supervision experts in the monitoring room annotated the classroom video frame by frame, and manually marked the classroom status into three categories: "effective interaction", "ineffective / discrete" and "silent".

[0214] Comparison group settings:

[0215] Solution A (this invention): Enables complete PTP clock synchronization, cross-correlation analysis, and feedback logic.

[0216] Option B (Traditional Energy Threshold Method): Only the average movement amplitude and sound decibels of the student are detected. When the values ​​are lower than a fixed threshold, it is determined that there is no interaction.

[0217] Option C (Cross-correlation method without clock synchronization): Remove the PTP protocol and directly use the network arrival time (NTP / system time) for data alignment and correlation calculation.

[0218] 2. Visualization of experimental results

[0219] Using a typical classroom segment (time window) to Taking the algorithm running data as an example.

[0220] like Figure 3 The solid black line represents the teacher's verbal encouragement signal, and the dashed gray line represents the student group response signal synchronized via PTP.

[0221] As can be seen, in Scheme A of the present invention, the two exhibit obvious phase-following characteristics (the gray dashed line always lags the black solid line by approximately...). In scheme C, due to network jitter, the gray dashed line exhibits unstable leading or excessive lagging relative to the black solid line, resulting in the destruction of causal characteristics.

[0222] like Figure 4 This demonstrates the normalized cross-correlation coefficients. Over time and lag The distribution of changes. A bright "synchronization ridge" is clearly visible in the figure, stabilizing at... The effectiveness of the physiological time-dependent constraint was verified in the vicinity. However, the corresponding heatmap generated by scheme C was a chaotic discrete region (without obvious peaks), which could not focus on the optimal time delay.

[0223] like Figure 5 In the middle, the black solid line represents the calculations of this invention. The indicators, marked with hollow dots, represent expert human scoring. to During the chaotic period, Scheme B (gray dotted line, representing the traditional energy method) incorrectly maintained a high score (false alarm) because it detected students making large movements (although disruptive movements); while Scheme A of this invention... It accurately dropped below 0.2, highly consistent with expert ratings, and in The low-level active feedback signal was correctly triggered (gray shaded area).

[0224] Key Performance Indicators (KPIs) Statistics Table

[0225] The table below shows the detection performance of 150 abnormal interaction events over 20 class hours:

[0226] Detection accuracy 94.6% 62.3% 78.5% Option A can distinguish between "chaos" and "interaction". False alarm rate 3.2% 28.7% 15.4% Option B cannot distinguish between normal stillness during blackboard writing. Average detection delay 4.8s 12.5s 8.2s Response speed after EWMA filter parameter optimization Timing alignment error 120μs N / A 158ms Hardware advantages brought by the PTP protocol

[0227] 4. Experimental Conclusions

[0228] Necessity of physical layer synchronization: Compared with schemes A and C, the lack of PTP synchronization will lead to a significant decrease in the signal-to-noise ratio of correlation calculation, especially in short-term interaction scenarios (such as rapid question and answer). Scheme C is very prone to missed detection due to timing misalignment.

[0229] Anti-interference performance of causal correlation: Compared with schemes A and B, this invention effectively solves the logical defect of traditional schemes that do not pay attention to relationships by introducing causal logic of incentives and responses. The accuracy rate is improved by more than 30% in identifying negative states in chaotic classrooms.

[0230] Effectiveness of the feedback mechanism: Statistics show that within 60 seconds of triggering feedback, classroom interaction indicators... The average rebound rate reached 0.45, proving that non-intrusive feedback can effectively assist teachers in conducting teaching interventions.

Claims

1. A classroom teaching interaction method based on edge computing, characterized in that, Includes the following steps: The teacher acquisition node (100) acquires teaching voice signals and screen image frames, extracts multimodal features to generate teaching incentive vectors, and determines the corresponding teaching modality labels based on the multimodal features; The aggregation gateway (300) receives the teaching modality tag, generates frequency domain gating parameters according to the preset modality frequency domain mapping relationship, and sends them to the student visual nodes (200) distributed in the classroom. The student visual node (200) adjusts the passband of the local digital filter according to the frequency domain gating parameters and collects the video stream of the student area. It performs adaptive frequency domain filtering and energy aggregation on the dense motion vector field of the video stream and uploads scalar student response energy data to the aggregation gateway (300). The aggregation gateway (300) performs time-series alignment of the teaching incentive vector and the student response energy data, and calculates the cross-correlation coefficient of the teaching incentive vector and the student response energy data within a preset physiological effective window, thereby obtaining the interaction synchronization index. The aggregation gateway (300) compares the interactive synchronization index with a preset threshold. When the preset abnormal triggering condition is met, it generates a drive signal to control the feedback device (400) to perform physical feedback. 2.The edge computing based classroom teaching interaction method according to claim 1, characterized in that, The teacher acquisition node (100) includes an audio acquisition module (110), a video capture module (120), and a first processing module (130). In the step of generating teaching incentive vectors, the first processing module (130) divides the teaching speech signal collected by the audio acquisition module (110) into frames and calculates the short-time root mean square energy using the root mean square formula. The first processing module (130) converts the screen image frames obtained by the video capture module (120) into a grayscale matrix according to the video change rate calculation formula, and solves the video change rate by calculating the sum of the absolute values ​​of the pixel grayscale differences between adjacent frames. The first processing module (130) further normalizes the short-time root mean square energy and the video change rate based on the extreme value statistics within the sliding time window, and uses a preset modal weighting coefficient to perform weighted fusion of the two. 3.The edge computing based classroom teaching interaction method according to claim 2, characterized in that, In the step of determining the corresponding teaching modality label, the first processing module (130) classifies the data according to the preset audio silence determination threshold and high-frequency interaction determination threshold: When the short-time root mean square energy is lower than the audio silence determination threshold and the video change rate is lower than the preset video silence determination threshold, it is marked as a blackboard writing or silent mode. When the short-time root mean square energy is higher than the high-frequency interaction determination threshold or the video change rate is higher than the preset video high-frequency determination threshold, it is marked as a high-frequency interaction mode; in other states, it is marked as a teaching mode. 4.The edge computing based classroom teaching interaction method according to claim 3, characterized in that, The modal frequency domain mapping relationship is configured as a mapping table between modes and frequency domain parameters. The mapping table stores frequency domain gating parameters that correspond one-to-one with each of the teaching modal labels. The frequency domain gating parameters include the lower cutoff frequency and upper cutoff frequency used to configure the passband of the local digital filter. 5.The edge computing based classroom teaching interaction method according to claim 1, characterized in that, The student visual node (200) includes a second processing module (220). When performing the adaptive frequency domain filtering and energy aggregation, the second processing module (220) first uses the dense optical flow algorithm to convert the grayscale video stream into a dense motion vector field containing instantaneous displacement vectors. Then, according to the frequency domain gating parameters, it calculates the feedback coefficients and feedforward coefficients of the filter in real time using the bilinear transform method, and drives the local digital filter to filter out noise components outside the passband frequency range in the dense motion vector field. 6.The edge computing based classroom teaching interaction method according to claim 5, characterized in that, The student response energy data is obtained by the second processing module (220) through spatial integration and dimensionality reduction. The instantaneous pixel power is obtained by summing the squares of the horizontal and vertical components of the filtered motion vector field. The original aggregated total energy is obtained by accumulating the effective pixel power in the field of view that exceeds the noise filtering threshold. The original aggregated total energy is normalized into scalar data by combining the depth coefficient and the total number of effective pixels. 7.The edge computing based classroom teaching interaction method according to claim 1, characterized in that, The timing alignment is based on a global synchronization time base established between the aggregation gateway (300), the teacher acquisition node (100), and the student vision node (200) through a precise time protocol. The teacher acquisition node (100) and the student vision node (200) directly generate a 64-bit timestamp at the time of physical signal acquisition and encapsulate it in the header of the data packet, so that the aggregation gateway (300) can use the jitter buffer to reconstruct the timing relationship of the data sequence based on the timestamp. 8.The edge computing based classroom teaching interaction method according to claim 1, characterized in that, In the step of calculating the cross-correlation coefficient, the search is limited to the physiological effective window composed of the minimum physiological response latency and the effective duration of the teaching instruction. The aggregation gateway (300) first performs a de-DC centering operation on the teaching incentive vector and student response energy data in the sliding time window, and then traverses the time lag in the physiological effective window to solve for the maximum value of the normalized cross-correlation function. 9.The edge computing based classroom teaching interaction method according to claim 1, characterized in that, When determining whether the abnormal triggering conditions are met, a dual-threshold state machine model with hysteresis is adopted, wherein the trigger threshold is set to be lower than the recovery threshold. The system only confirms the entry into the abnormal triggering state when the proportion of the interactive synchronization index that is continuously lower than the trigger threshold within the verification window exceeds the minimum proportion of abnormal frames required for triggering, and only releases the triggering state when the interactive synchronization index rises back to above the recovery threshold.

10. A classroom teaching interaction system based on edge computing, characterized in that, The classroom teaching interaction method based on edge computing, as described in any one of claims 1-9, includes: The teacher acquisition node (100) is deployed in the classroom podium area to acquire teaching audio signals and screen image frames, and generate teaching incentive vectors and teaching modality labels; the teacher acquisition node (100) includes an audio acquisition module (110), a video capture module (120) and a first processing module (130). The student visual node (200) is distributed on the top of the classroom and is used to collect video streams and perform edge calculations according to frequency domain gating parameters, and output student response energy data; the student visual node (200) includes an image sensing module (210) and a second processing module (220), the second processing module (220) having a built-in variable parameter digital filter; The aggregation gateway (300) is communicatively connected to the teacher acquisition node (100) and the student visual node (200), respectively, and is used to send the corresponding frequency domain gating parameters to the student visual node (200) according to the teaching modality label, and calculate the interaction synchronization index between the teaching incentive vector and the student response energy data; the aggregation gateway (300) includes a communication module (310) and an analysis module (320). A feedback device (400), connected to the aggregation gateway (300), is used to perform physical feedback when the interactive synchronization indicator meets the abnormal triggering conditions.

Images