A teacher-student attention cooperative tracking and feedback adjustment device for a smart classroom

Abstract

The application discloses a teacher-student attention collaborative tracking and feedback adjusting device for a smart classroom and relates to the technical field of smart education. The device comprises a multi-modal collaborative perception mechanism, a data synchronization and edge computing mechanism and a hierarchical feedback execution mechanism. The multi-modal collaborative perception mechanism comprises a student perception sub-mechanism and a teacher perception sub-mechanism, which respectively collect behavior data of students and teachers. The data synchronization and edge computing mechanism is internally provided with a hardware clock synchronization board and a heterogeneous edge computing module, realizes multi-source sensor hardware level synchronization and data fusion processing, and executes teacher end, student end and environment regulation feedback according to the fusion result. The device realizes classroom attention collaborative tracking of teacher-student double subjects, has the characteristics of high synchronization precision and flexible deployment.

Description

Technical Field

[0001] This invention relates to the field of smart education technology, specifically a teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms. Background Technology

[0002] Classroom attention tracking is a research hotspot in the field of smart education. Existing vision-based attention monitoring solutions infer attention status by collecting external behavioral characteristics such as students' head posture and facial orientation. These solutions are entirely student-oriented in their hardware deployment, relying solely on student behavior for attention assessment. However, the classroom is a dynamic interactive system comprised of both teachers and students. Factors such as the teacher's teaching position, gaze area, and facing direction significantly influence students' attention distribution. Excluding teacher factors from the monitoring system is one of the important reasons for the bias in judgments made by existing solutions in real classroom environments. Meanwhile, existing devices have the following shortcomings at the hardware architecture level: to expand to dual-subject tracking, it is necessary to collect multi-source data such as video, audio, and positioning simultaneously. However, the clock sources of each sensor are independent, and the software layer timestamp alignment error can reach hundreds of milliseconds to seconds, causing the causal correlation analysis between teachers and students to lose its temporal basis. In addition, most existing cameras are fixed wall-mounted, and after the classroom layout is adjusted, rewiring and drilling are required. Furthermore, physical limitations such as front row obstruction and backlighting of projection cannot be fundamentally solved by algorithms. Uploading the collected video and image data to the cloud for processing also poses risks of transmission delay and leakage of biometric data. Summary of the Invention

[0003] The purpose of this invention is to provide a teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms.

[0004] The objective of this invention can be achieved through the following technical solutions: This application provides a teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms, including a multimodal collaborative sensing mechanism, a data synchronization and edge computing mechanism, and a hierarchical feedback execution mechanism; The multimodal collaborative sensing mechanism includes a student sensing sub-mechanism and a teacher sensing sub-mechanism. The student sensing sub-mechanism is installed on the classroom ceiling and includes a reconfigurable slide rail bracket fixed to the ceiling, an electric sliding seat mounted on the slide rail bracket and sliding along it, and a distributed camera array mounted on the electric sliding seat via a magnetic quick-release base. The reconfigurable slide rail bracket is modularly assembled from several slide rail units, and the connection angle and relative position between each slide rail unit are adjustable. The field of view of the distributed camera array is oriented towards the student area. The teacher sensing sub-mechanism includes a spatial positioning component worn by the teacher and a teacher tracking camera component installed at the front of the classroom. The data synchronization and edge computing mechanism is installed in a rack at the front of the classroom, and includes a hardware clock synchronization board and a heterogeneous edge computing module. The hardware clock synchronization board has a synchronization pulse output interface, which is connected via signal lines to the trigger signal input terminals of each camera in the distributed camera array and the teacher tracking camera component. The synchronization pulse output interface is also connected via signal lines to the spatial positioning base station, which has a built-in wireless synchronization pulse transmission module for sending wireless synchronization trigger signals to the spatial positioning tag. The data input terminal of the heterogeneous edge computing module is connected to the data output terminals of the distributed camera array, the teacher tracking camera component, and the spatial positioning component. The hierarchical feedback actuator includes a teacher-side feedback actuator installed at the front edge of the podium, a student-side feedback actuator installed on the student's desk, and an environmental control output interface installed on the side wall of the classroom; the control signal output terminal of the heterogeneous edge computing module is respectively connected to the control signal input terminal of the teacher-side feedback actuator, the student-side feedback actuator, and the environmental control output interface.

[0005] The beneficial effects of this invention are as follows: This invention enables collaborative tracking of classroom attention by having both students and teachers work together, overcoming the shortcomings of traditional solutions that rely solely on student behavior as the basis for attention assessment and neglect the influence of teachers' teaching positions and visual focus areas on student attention, thus improving the accuracy of attention state assessment. By using a hardware clock synchronization board to perform hardware-level synchronization triggering of the distributed camera array, spatial positioning component, and teacher tracking camera component, the problem of large software layer timestamp alignment error caused by the independent clock sources of each sensor is solved, providing an accurate timing basis for causal correlation analysis between teacher and student behaviors. This invention utilizes a reconfigurable sliding rail bracket, which is modularly assembled from several sliding rail units, and the connection angle and relative position between each sliding rail unit are adjustable. This allows for flexible reconfiguration of the camera layout without the need for rewiring or drilling, effectively avoiding physical limitations such as difficulty in adjusting classroom layout, front row obstruction, and projection backlighting under fixed wall-mounted installation. This invention uses a heterogeneous edge computing module to perform multi-source data fusion processing and attention state inference locally in the classroom, avoiding the transmission delay and biometric leakage risks caused by uploading video and image data to the cloud, thus ensuring the real-time performance and data security of classroom attention tracking. Attached Figure Description

[0006] To better understand and implement this application, the technical solution is described in detail below with reference to the accompanying drawings.

[0007] Figure 1 A top view of the overall layout of the device provided in an embodiment of the present invention in a classroom; Figure 2 This is a side view showing the installation relationship between the reconfigurable slide rail bracket and the electric sliding seat provided in an embodiment of the present invention. Figure 3 A partial schematic diagram of a tracking close-up camera provided in an embodiment of the present invention; Figure 4 This is a partial schematic diagram of the teacher-side feedback actuator provided in an embodiment of the present invention; Figure 5 A hierarchical feedback adjustment flowchart provided for embodiments of the present invention; Figure 6 This is a flowchart of attention collaborative tracking and fusion reasoning provided in an embodiment of the present invention; In the diagram: 1-Classroom; 2-Reconfigurable slide rail bracket; 21-Slide rail unit; 22-Hinge; 23-Fixed seat; 3-Electric sliding seat; 4-Magnetic quick-release base; 5-Distributed camera; 6-Panoramic positioning camera; 7-Tracking close-up camera; 8-Two-degree-of-freedom electric pan-tilt head; 81-Horizontal rotation mechanism; 82-Pitch rotation mechanism; 83-ARM servo controller; 9-Spatial positioning base station; 10-Spatial positioning tag; 11-First illuminance sensor; 12-Second illuminance sensor; 13-Temperature and humidity sensor; 14-CO2 concentration sensor; 15-Teacher feedback actuator; 16-Touch display screen; 17-RGB indicator light; 18-Environmental control output interface; 19-Hardware clock synchronization board; 20-Heterogeneous edge computing module. Detailed Implementation

[0008] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0009] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0010] The following detailed description of the specific implementation methods, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided in detail.

[0011] Example 1 Please see Figures 1-6 This embodiment provides a teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms, including a multimodal collaborative sensing mechanism, a data synchronization and edge computing mechanism, and a hierarchical feedback execution mechanism. The functional structures work together to realize the synchronous acquisition of teacher and student dual-subject behavioral data during classroom attention tracking, hardware-level synchronous triggering of multi-source sensors and real-time fusion inference at the edge, and differentiated hierarchical feedback control based on the fusion results, effectively improving the accuracy and real-time performance of attention state judgment.

[0012] In this embodiment, the classroom is a rectangular space with a blackboard and podium at the front and student desks at the back. The multimodal collaborative sensing mechanism includes a student sensing sub-mechanism and a teacher sensing sub-mechanism.

[0013] The student sensing sub-mechanism is installed on the classroom ceiling and includes a reconfigurable slide rail bracket 2 fixed to the ceiling, an electric sliding seat 3 mounted on and sliding along the slide rail bracket 2, and a distributed camera array 5 mounted on the electric sliding seat 3 via a magnetic quick-release base 4. The reconfigurable slide rail bracket 2 is modularly assembled from several slide rail units 21, and the connection angle and relative position between each slide rail unit 21 can be adjusted according to the actual layout of the classroom.

[0014] Specifically, the reconfigurable slide rail bracket 2 is arranged in a W-shape on the ceiling directly above the student desk area. It is composed of four slide rail units 21. The left half is formed by the first and second slide rail units forming a V-shape, and the right half is formed by the third and fourth slide rail units forming a V-shape. The left and right sides meet and connect in the middle of the classroom through a central hinge. The included angle between each slide rail unit 21 can be adjusted between 120° and 150° according to the width of the classroom and the arrangement of the student desks. The left and right ends of the reconfigurable slide rail bracket 2 are fixed to the side walls of the classroom by expansion bolts through fixing seats 23. Each hinge 22 is fixed to the ceiling by a hanger rod through expansion bolts, providing vertical load-bearing support for the middle of the slide rail bracket. The hinge component 22 includes a first hinge plate, a second hinge plate, and a central rotating shaft; the first hinge plate is fixedly connected to the end of one of two adjacent slide rail units 21, and the second hinge plate is fixedly connected to the end of the other, with the first and second hinge plates hinged to each other via the central rotating shaft; the mating surface of the first hinge plate is provided with an annular toothed groove arranged around the central rotating shaft, and the mating surface of the second hinge plate is provided with an annular toothed ring adapted to the annular toothed groove; the second hinge plate is provided with a guide hole extending axially along the central rotating shaft, through which the central rotating shaft passes. The guide hole is provided, and a clamping handle is provided at one end of the central rotating shaft. A return spring is also sleeved on the central rotating shaft. One end of the return spring abuts against the second hinge plate, and the other end abuts against the clamping handle. The return spring drives the annular gear ring and the annular gear groove to maintain a meshing and locking state. When it is necessary to adjust the included angle between adjacent slide rail units 21, the clamping handle is pulled to disengage the annular gear ring from the annular gear groove. After rotating the slide rail unit 21 to the required angle, the clamping handle is released, and the return spring drives the annular gear ring and the annular gear groove to re-mesh and lock. Multiple electric sliding seats 3 are provided, each riding on a slide rail unit 21. Each electric sliding seat 3 has a built-in drive motor and rollers, and can slide back and forth along the length of the slide rail unit 21. A distributed camera 5 is installed under each electric sliding seat 3 via a magnetic quick-release base 4. The magnetic quick-release base 4 uses a permanent magnet to achieve quick disassembly, assembly, and position adjustment of the camera 5. The magnetic quick-release base 4 also includes a mechanical anti-detachment buckle, which is manually locked after being magnetically attached to prevent the camera 5 from accidentally falling off when the electric sliding seat 3 accelerates or decelerates. The distributed camera array 5 consists of multiple wide-angle high-definition cameras. The field of view of each camera 5 is directed towards the student desk area below. The field of view of adjacent cameras 5 has a set overlap range at the edge area to achieve no blind spots in the student area.

[0015] The teacher sensing sub-mechanism includes a spatial positioning component worn by the teacher and a teacher tracking camera component installed at the front of the classroom. The spatial positioning component includes a spatial positioning tag 10 worn by the teacher and spatial positioning base stations 9 fixed at the four corners of the classroom. The spatial positioning tag 10 is a UWB positioning tag worn by the teacher and continuously transmits positioning signals. Four spatial positioning base stations 9 are fixedly installed on the walls at the four corners of the classroom to receive the positioning signals transmitted by the spatial positioning tags 10 and calculate the teacher's three-dimensional spatial coordinates in the classroom in real time using a time difference of arrival algorithm. Each spatial positioning base station 9 has a built-in wireless synchronization pulse transmission module for sending wireless synchronization trigger signals to the spatial positioning tags 10.

[0016] The teacher tracking camera assembly includes a fixedly installed panoramic positioning camera 6 and a tracking close-up camera 7 mounted on a two-degree-of-freedom motorized pan-tilt head 8. The panoramic positioning camera 6 is fixedly installed on the ceiling at the front of the classroom, and its field of view covers the entire teacher's activity area (including the blackboard area and the podium area), used for global monitoring of the teacher's teaching position, facial orientation, and direction of gaze. The two-degree-of-freedom motorized pan-tilt head 8 is fixedly installed on the ceiling at the front of the classroom and located next to the panoramic positioning camera 6.

[0017] The two-degree-of-freedom motorized gimbal 8 includes a horizontal rotation mechanism 81 and a pitch rotation mechanism 82. The horizontal rotation mechanism 81 can rotate horizontally from 0° to 355°, and the pitch rotation mechanism 82 can adjust the pitch from -15° to +45°. The two-degree-of-freedom motorized gimbal 8 has a built-in ARM servo controller 83, which is connected to the panoramic positioning camera 6. Based on the real-time position of the teacher in the classroom identified by the panoramic positioning camera 6, the horizontal rotation mechanism 81 and the pitch rotation mechanism 82 are driven to keep the tracking close-up camera 7 always pointing at the teacher, thus achieving high-definition close-up tracking and shooting of the teacher's behavior. The ARM servo controller 83 also receives the images captured by the tracking close-up camera 7 and performs fine-tuning closed-loop control by comparing the center offset of the teacher target in the image of the tracking close-up camera 7 to achieve precise locking of the teacher target.

[0018] The panoramic positioning camera 6 and the tracking close-up camera 7 work together. The panoramic positioning camera 6 is responsible for large-area positioning, while the tracking close-up camera 7, driven by the ARM servo controller 83, follows the teacher's movement to capture close-up shots. Together, they form a complete teacher behavior data acquisition channel. At the same time, the spatial positioning component (spatial positioning base station 9 and spatial positioning tag 10) and the teacher tracking camera component (panoramic positioning camera 6 and tracking close-up camera 7) cooperate to form a redundant positioning mechanism: when the teacher is within the field of view of the panoramic positioning camera 6, the panoramic positioning camera 6 completes the visual recognition of the teacher's position; when the teacher moves to the back or side of the classroom, or to areas where the visual recognition accuracy of the panoramic positioning camera 6 decreases, the UWB spatial positioning data provided by the spatial positioning tag 10 and the spatial positioning base station 9 continues to complete the real-time tracking of the teacher's position, ensuring that the teacher's position data is continuous and uninterrupted throughout the entire classroom. By employing the above technical solution, the collaborative work of the student perception sub-mechanism and the teacher perception sub-mechanism enables collaborative tracking of classroom attention by both teachers and students. This overcomes the shortcomings of traditional solutions that rely solely on student behavior for attention assessment and neglect the influence of teachers' teaching positions and line-of-sight areas on student attention, thus improving the accuracy of attention state assessment. The reconfigurable slide rail bracket 2, modularly assembled from several slide rail units 21 with adjustable connection angles and relative positions, allows for flexible reconfiguration of the camera 5 layout without the need for rewiring or drilling. This effectively avoids physical limitations such as difficulties in adjusting classroom layout, front-row obstruction, and backlighting from projectors associated with fixed wall-mounted installations. Furthermore, the electric sliding base 3 reciprocates along the slide rail unit 21, enabling the distributed cameras 5 to continuously move and scan within the range of a single slide rail unit 21, achieving comprehensive monitoring of the entire classroom with a limited number of cameras. The system provides multi-angle and all-round dynamic coverage of the student area. A W-shaped sliding rail bracket 2 covers the student desk area, ensuring that multiple electric sliding seats 3 and distributed cameras 5 on the four sliding rail units 21 can simultaneously collect image data of the student area from different angles. The locking mechanism of the hinge 22 between each sliding rail unit 21 ensures structural stability after adjustment, and the suspension rod at the hinge 22 ensures the load-bearing reliability of the middle section of the sliding rail bracket. The coordinated operation of the panoramic positioning camera 6 and the tracking close-up camera 7 enables simultaneous acquisition of global monitoring and local close-up shots of teacher behavior. The redundant positioning mechanism of the spatial positioning base station 9 and the panoramic positioning camera 6 ensures the continuity and uninterrupted nature of teacher location data throughout the classroom, avoiding the positioning loss problem caused by obstructed views or excessive distance in the rear or side areas of the classroom, which is a problem with single-vision positioning methods.

[0019] The data synchronization and edge computing mechanism is installed in a rack at the front of the classroom, and includes a hardware clock synchronization board 19 and a heterogeneous edge computing module 20. The hardware clock synchronization board 19 is equipped with an FPGA synchronization controller and a PPS second pulse generator electrically connected to the FPGA synchronization controller. The FPGA synchronization controller has a time signal receiver for receiving time reference signals from external time sources. The PPS second pulse generator has a synchronization pulse output interface. The synchronization pulse output interface is connected via signal lines to the trigger signal input terminals of each camera 5 in the distributed camera array 5 and the teacher tracking camera component (i.e., the trigger signal input terminals of the panoramic positioning camera 6 and the tracking close-up camera 7). The synchronization pulse output interface is also connected via signal lines to the trigger signal input terminals of each spatial positioning base station 9. Specifically, the synchronization pulse signal generated by the hardware clock synchronization board 19 is directly transmitted via wired signal lines to each fixedly installed camera (each camera 5 in the distributed camera array 5, the panoramic positioning camera 6, and the tracking close-up camera 7), triggering each camera to synchronously expose and acquire images. The synchronization pulse signal is also transmitted via wired signal lines to each spatial positioning base station 9. After receiving the synchronization pulse, the spatial positioning base station 9 sends a wireless synchronization trigger signal to the spatial positioning tag 10 through its built-in wireless synchronization pulse transmission module, ensuring that the spatial positioning tag 10 is synchronized with each camera in time. Using the above technical solution, a high-precision synchronization pulse signal is generated by an FPGA synchronization controller in conjunction with a PPS second pulse generator. The synchronization pulse output interface is then connected via signal lines to the trigger signal input terminals of each camera 5 in the distributed camera array 5, the panoramic positioning camera 6, and the tracking close-up camera 7, respectively, to achieve wired hardware-level synchronization triggering for fixed-installation cameras. By transmitting the synchronization pulse signal to the spatial positioning base station 9 and sending a wireless synchronization trigger signal to the spatial positioning tag 10 via its built-in wireless synchronization pulse transmission module, wireless synchronization triggering of the mobile spatial positioning tag 10 is achieved. By combining wired and wireless methods, the problem of large software layer timestamp alignment error (hundreds of milliseconds to seconds) caused by the independent clock sources of each sensor is solved, providing an accurate timing basis for the causal correlation analysis between teachers and students (synchronization accuracy better than 1 millisecond).

[0020] The heterogeneous edge computing module 20 includes an NPU neural network processing unit, a DSP digital signal processor, and a CPU central processing unit interconnected via a high-speed bus. The heterogeneous edge computing module 20 is connected to the hardware clock synchronization board 19 via a PCIe bus and is used to receive multi-source sensing data synchronized by the hardware clock and perform fusion inference.

[0021] The data input terminals of the heterogeneous edge computing module 20 are respectively connected to the data output terminals of the distributed camera array 5, the teacher tracking camera component (panoramic positioning camera 6 and tracking close-up camera 7), and the spatial positioning component (spatial positioning base station 9 and spatial positioning tag 10); the heterogeneous edge computing module 20 has a built-in dedicated hardware acceleration circuit for causal correlation analysis of teacher and student behavior. The dedicated hardware acceleration circuit is an ASIC acceleration unit or FPGA acceleration unit integrated inside the heterogeneous edge computing module 20, which is interconnected with the NPU neural network processing unit, DSP digital signal processor, and CPU central processing unit through a high-speed bus; the dedicated hardware acceleration circuit includes: a data buffer queue for buffering data transmitted via a hardware clock synchronization board. The system utilizes multi-source sensor data synchronized with the SIM19 chip; a pipelined parallel computing unit, employing a multi-stage pipeline architecture, performs parallel preprocessing of student individual attention inference results and teacher behavioral characteristics output by the NPU; a feature alignment engine, using the unified hardware timestamp provided by the SIM19 hardware clock synchronization board as an index, aligns teacher behavioral characteristics at the same moment with student behavioral characteristics in the time dimension; and an association matching operation unit, with a built-in parallel comparator array, is used to calculate the spatiotemporal matching degree between each student behavioral characteristic and the teacher behavioral characteristic in parallel. The pipelined parallel computing unit, feature alignment engine, and association matching operation unit are connected via an on-chip interconnect bus, enabling pipelined parallel acceleration of teacher-student behavior association analysis through hardware logic. The processing can complete the low-level calculation of spatiotemporal correlation matching without the intervention of the CPU central processing unit; it is used to perform fusion inference on multi-source perception data synchronized by the hardware clock synchronization board 19, specifically including: receiving student head posture and facial orientation image data, teacher panoramic positioning and tracking close-up video data, and teacher three-dimensional spatial coordinate data provided by spatial positioning tag 10, and running a pre-trained attention recognition model for parallel inference through the NPU neural network processing unit. The attention recognition model is a deep learning model based on a hybrid architecture of convolutional neural network and Transformer. Its network structure includes: an input layer that receives RGB images collected by the distributed camera array 5; The feature extraction backbone network uses ResNet-50 as the backbone to extract features of the student's head and face regions in the image. The head pose estimation branch uses a fully connected layer to regress the student's pitch, yaw, and roll angles in 3D space as head pose angle features. The facial orientation and gaze direction estimation branch calculates the gaze direction vector by detecting 68 key feature points on the student's face and combining them with 12 feature points on the eyes, outputting facial orientation angle and gaze direction features. The attention state classification layer uses a Softmax classifier to fuse the above head pose angle, facial orientation, and gaze direction features, outputting the probability distribution of three categories: focused, slightly distracted, and severely distracted. The training method of the attention recognition model is as follows: Student video data from real classroom scenarios is collected in advance. The head posture angles (pitch, yaw, roll), facial orientation, gaze direction, and corresponding attention state labels of students in each frame are manually labeled to construct a training dataset. A transfer learning strategy is adopted, using network weights pre-trained on a large public face dataset as initialization parameters. The model is then subjected to supervised fine-tuning training using the labeled classroom dataset. The loss function is a weighted sum of the cross-entropy loss function and the angle regression loss function. The optimizer uses the Adam optimizer with an initial learning rate set to 1e-4. A cosine annealing learning rate scheduling strategy is employed, and training continues until the validation set loss function converges. Multi-source data is fused in the temporal domain using a DSP digital signal processor, and decision logic is executed by a CPU central processing unit, ultimately outputting the attention state label for each student and the teacher's behavioral feature label. The specific implementation methods for causal correlation analysis of teacher and student behavior are as follows: Based on the unified hardware timestamp provided by the hardware clock synchronization board 19, the DSP digital signal processor performs spatiotemporal correlation matching between the teacher's behavioral characteristics at the same moment and the behavioral characteristics of each student; specifically: Spatial correlation matching: Based on the teacher's three-dimensional spatial coordinates provided by the spatial positioning tag 10, combined with the teacher's facing direction vector identified by the panoramic positioning camera 6, the projection range of the teacher's current gaze area on the classroom plane is calculated; the head posture angle and gaze direction of each student are spatially overlapped with the teacher's gaze area. If the angle between the student's gaze direction and the teacher's position is less than the preset angle threshold, and the student's face is facing the same direction as the teacher's, then the student is determined to be within the teacher's gaze range. Temporal correlation analysis: Using a series of hardware timestamps from multiple consecutive frames as the analysis window, the percentage of frames in which each student is within the teacher's line of sight is statistically analyzed, as well as the temporal trend of the angle between the student's line of sight and the teacher's position. If the student follows the teacher's head posture and line of sight within a preset time window after the teacher turns or moves, it is determined that there is a temporal following relationship between the teacher and student's behavior. Attention Coordination Coefficient Calculation: Combining the spatial association matching results and temporal association analysis results above, the spatial association matching results and temporal following degree of each student are weighted and fused to calculate the attention coordination coefficient of each student. Integrated Decision Making: The CPU integrates the individual student attention inference results output by the NPU with the attention coordination coefficients calculated by the DSP. When the individual attention inference results are inconsistent with the attention coordination coefficients, the individual attention inference results are corrected using the attention coordination coefficients, and the final attention state of each student is output. For example, the preset angle threshold can be set to 15°, the preset time window can be set to 2 seconds, the analysis window can be set to 30 frames (corresponding to 1 second duration), the weights of spatial association matching results and temporal follow-up degree can be set to 0.6 and 0.4 respectively, and the attention coordination coefficient is determined to be effective influence of teacher behavior when it is higher than 0.7; By adopting the above technical solution, the heterogeneous edge computing module 20 completes the fusion processing of multi-source data and attention state inference locally in the classroom, avoiding the transmission delay and biometric leakage risks caused by uploading video and image data to the cloud, and ensuring the real-time performance and data security of classroom attention tracking. Through the heterogeneous division of labor among the NPU neural network processing unit, DSP digital signal processor and CPU central processing unit, efficient parallel processing of multi-source data is achieved to meet the real-time requirements of the classroom. The hardware clock synchronization board 19 is connected to the heterogeneous edge computing module 20 through the PCIe bus, ensuring that the synchronized data can be transmitted to the computing module for processing with low latency.

[0022] The hierarchical feedback actuator includes a teacher-side feedback actuator 15 installed at the front edge of the podium, a student-side feedback actuator installed on the student's desk, and an environmental control output interface 18 installed on the side wall of the classroom.

[0023] The teacher-side feedback actuator 15 includes a breathing light strip installed at the front edge of the podium and a touch display screen 16 embedded in the podium desktop; the breathing light strip includes at least one set of RGB full-color LED beads and a light strip driving circuit, the RGB full-color LED beads are arranged along the length of the front edge of the podium, and the light strip driving circuit is connected to the control signal output terminal of the heterogeneous edge computing module 20; the touch display screen 16 is used to display the teacher in real time a heat map of the attention distribution of all students in the class, an attention trend curve, and abnormal warning information; The student-side feedback actuator includes an RGB indicator light 17 embedded in the desktop of each student's desk. The RGB indicator light 17 is connected to the control signal output terminal of the heterogeneous edge computing module 20 through a wireless communication module. It is used to present different color indicators according to the real-time attention status of the corresponding student (such as green for focus, yellow for mild distraction, and red for severe distraction) to provide immediate reminders to the student. The environmental control output interface 18 includes an RS485 bus interface and a Zigbee wireless interface installed on the side wall of the classroom. The RS485 bus interface and the Zigbee wireless interface are respectively connected to the classroom IoT gateway and are used to output environmental control signals to IoT devices such as lights and air conditioners in the classroom according to the attention fusion result. By adopting the above technical solution, a three-layer feedback architecture consisting of teacher-side feedback actuator 15, student-side feedback actuator, and environmental control output interface 18 is used to realize a comprehensive closed-loop feedback regulation from teacher reminders and student self-reflection to environmental optimization, so that attention tracking results can be promptly transformed into effective intervention measures.

[0024] The multimodal collaborative sensing mechanism also includes an environmental sensing sub-mechanism, which includes a light intensity sensor, a temperature and humidity sensor 13, and a CO2 concentration sensor 14 distributed in the classroom. The illuminance sensor includes a first illuminance sensor 11 and a second illuminance sensor 12 installed on the ceiling of the classroom. The first illuminance sensor 11 is installed on the ceiling near the blackboard and its acquisition end faces the student desk area, and is used to collect the illuminance of the student desk area. The second illuminance sensor 12 is installed on the ceiling near the student area and its acquisition end faces the podium area, and is used to collect the illuminance of the podium area. Temperature and humidity sensor 13 and CO2 concentration sensor 14 are installed on the side and rear walls of the classroom, respectively, at a height of 1.2 meters to 1.8 meters above the ground, to collect temperature, humidity, and carbon dioxide concentration data in the activity areas for students and teachers. The data output terminals of each sensor in the environmental sensing sub-mechanism are connected to the data input terminals of the heterogeneous edge computing module 20. Using the above technical solution, the first illuminance sensor 11 and the second illuminance sensor 12 monitor the light intensity of the student desk area and the podium area respectively, providing a basis for light compensation and control; the temperature and humidity sensor 13 and the CO2 concentration sensor 14 monitor the temperature, humidity and carbon dioxide concentration of the student and teacher activity areas. When the environmental parameters deviate from the comfort threshold, the heterogeneous edge computing module 20 sends control commands to the classroom air conditioner, fresh air system and other equipment through the environmental control output interface 18 via the Internet of Things gateway, realizing intelligent adjustment of the classroom environment and optimizing the students' learning status.

[0025] The device described in this embodiment achieves synchronous collection of behavioral data of both teachers and students during classroom attention tracking, hardware-level synchronous triggering and real-time fusion inference at the edge, and differentiated hierarchical feedback control based on the fusion results through the coordinated operation of the above-mentioned components.

[0026] After the device is started, each camera 5 of the distributed camera array 5, the panoramic positioning camera 6, the tracking close-up camera 7, and the spatial positioning component are synchronously triggered by the hardware clock synchronization board 19, and at the same time, they respectively collect student head posture images, teacher panoramic images, teacher close-up images, and teacher three-dimensional spatial coordinate data. All data have a unified hardware timestamp. Data collected by each sensor is transmitted in real time to the heterogeneous edge computing module 20. The NPU neural network processing unit runs a pre-trained attention recognition model for parallel inference, identifying each student's head posture angle, facial orientation, and gaze direction, as well as the teacher's teaching position area, facing direction, and gaze direction. The DSP digital signal processor uses the unified hardware timestamp provided by the hardware clock synchronization board 19 as a reference to perform spatiotemporal correlation matching between the teacher's behavioral characteristics at the same moment and the behavioral characteristics of each student. Specifically, this includes: calculating the projection range of the teacher's current gaze area on the classroom plane based on the teacher's three-dimensional spatial coordinates provided by the spatial positioning tag 10 and the teacher's facing direction vector identified by the panoramic positioning camera 6; spatially overlapping each student's head posture angle and gaze direction with the teacher's gaze area to determine whether the student is within the teacher's gaze influence range; and using a continuous multi-frame hardware timestamp sequence as an analysis window, statistically analyzing the percentage of frames in which each student is within the teacher's gaze influence range to determine whether there is a gap between the two student's gaze and the teacher's gaze influence range. The study investigates the temporal following relationship between teacher and student behaviors. It integrates spatial correlation matching and temporal correlation analysis results, weighting and fusing the spatial correlation matching results and temporal following degree for each student to calculate their attention coordination coefficient. The CPU integrates the individual student attention inference results output by the NPU with the attention coordination coefficient calculated by the DSP. When the individual attention inference results and the attention coordination coefficient are inconsistent, the attention coordination coefficient is used to correct the individual attention inference results, outputting each student's final attention state label and corresponding feedback strategy. For example, the preset angle threshold for determining whether a student is within the teacher's line of sight in spatial correlation matching can be set to 15°, the analysis window in temporal correlation analysis can be set to 30 frames (corresponding to 1 second), and the preset time window can be set to 2 seconds. The weights of spatial correlation matching results and temporal following degree in the attention coordination coefficient calculation can be set to 0.6 and 0.4 respectively. An attention coordination coefficient higher than 0.7 is considered an effective influence of the teacher's behavior. Feedback strategies include: displaying a heat map of the whole class's attention distribution and abnormal warning information to the teacher through the touch screen 16; providing the teacher with intuitive prompts on the overall attention status of the classroom through changes in the color and flashing frequency of the breathing light strip; sending immediate reminders to individual distracted students through the RGB indicator lights 17 on the student desks; and sending environmental control commands to the classroom's lighting, air conditioning, and other equipment via the IoT gateway through the environmental control output interface 18. For example, adjusting the classroom lighting brightness when the first illuminance sensor 11 detects that the illuminance in the student desk area is lower than a set threshold, and activating the fresh air system when the CO2 concentration sensor 14 detects that the classroom CO2 concentration exceeds a set threshold. Throughout the process, the teacher's perception sub-mechanism and the student's perception data are synchronized at the sub-millisecond level through the hardware clock synchronization board 19, ensuring the precise alignment of the time sequence of teacher and student behaviors; the heterogeneous edge computing module 20 completes all data processing and inference tasks locally in the classroom without uploading to the cloud, effectively protecting the biometric privacy of students and teachers. If the classroom layout changes (such as adding or removing desks, adjusting the arrangement of desks and chairs, etc.), the operator can loosen the locking mechanism at the hinge 22 and readjust the connection angle and relative position between each slide rail unit 21 so that the overall coverage of the reconfigurable slide rail bracket 2 adapts to the newly laid-out student desk area without the need for rewiring or drilling; if it is necessary to add or remove the number of cameras, the operator can quickly install or remove the cameras 5 using the magnetic quick-release base 4, which has good on-site adaptability; Example 2: This example provides an operation method for a teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms, based on the structure described in Example 1; The device includes a multimodal collaborative sensing mechanism, a data synchronization and edge computing mechanism, and a hierarchical feedback execution mechanism; The multimodal collaborative sensing mechanism includes a student sensing sub-mechanism and a teacher sensing sub-mechanism. The student sensing sub-mechanism is installed on the classroom ceiling and includes a reconfigurable slide rail bracket 2 fixed to the ceiling, an electric sliding seat 3 installed on the slide rail bracket 2 and sliding along it, and a distributed camera array 5 installed on the electric sliding seat 3 via a magnetic quick-release base 4. The reconfigurable slide rail bracket 2 is modularly assembled from several slide rail units 21, and the connection angle and relative position between each slide rail unit 21 are adjustable. The field of view of the distributed camera array 5 is oriented towards the student area. The teacher sensing sub-mechanism includes a spatial positioning component worn by the teacher and a teacher sensor installed at the front of the classroom. The tracking camera assembly includes a spatial positioning tag 10 worn by the teacher and spatial positioning base stations 9 fixed at the four corners of the classroom. The teacher tracking camera assembly includes a panoramic positioning camera 6 fixedly installed on the ceiling in front of the classroom and a tracking close-up camera 7 installed on a two-degree-of-freedom electric pan-tilt head 8. The two-degree-of-freedom electric pan-tilt head 8 includes a horizontal rotation mechanism 81 and a pitch rotation mechanism 82. The two-degree-of-freedom electric pan-tilt head 8 has a built-in ARM servo controller 83. The ARM servo controller 83 is signal-connected to the panoramic positioning camera 6 and drives the horizontal rotation mechanism 81 and the pitch rotation mechanism 82 according to the teacher's position identified by the panoramic positioning camera 6, so that the tracking close-up camera 7 is aimed at the teacher. The data synchronization and edge computing mechanism is installed in a rack at the front of the classroom, and includes a hardware clock synchronization board 19 and a heterogeneous edge computing module 20. The hardware clock synchronization board 19 is equipped with an FPGA synchronization controller and a PPS second pulse generator. The FPGA synchronization controller has a timing signal receiving end, and the PPS second pulse generator has a synchronization pulse output interface. The synchronization pulse output interface is connected to the trigger signal input end of each camera 5 of the distributed camera array 5 and the teacher tracking camera component through signal lines, and is also connected to the trigger signal input end of the spatial positioning component through wireless communication. The heterogeneous edge computing module 20 is connected to the hardware clock synchronization board 19 through a PCIe bus, and its data input end is connected to the data output end of the distributed camera array 5, the teacher tracking camera component, and the spatial positioning component. The hierarchical feedback actuator includes a teacher-side feedback actuator 15 installed at the front edge of the podium, a student-side feedback actuator installed on the student's desk, and an environmental control output interface 18 installed on the side wall of the classroom. The teacher-side feedback actuator 15 includes a breathing light strip and a touch display screen 16. The student-side feedback actuator includes an RGB indicator light 17 and is connected to the heterogeneous edge computing module 20 via a wireless communication module. The environmental control output interface 18 includes an RS485 bus interface and a Zigbee wireless interface, which are respectively connected to the classroom IoT gateway. The control signal output terminal of the heterogeneous edge computing module 20 is respectively connected to the control signal input terminals of the teacher-side feedback actuator 15, the student-side feedback actuator, and the environmental control output interface 18. The multimodal collaborative sensing mechanism also includes an environmental sensing sub-mechanism, comprising a light intensity sensor, a temperature and humidity sensor 13, and a CO2 concentration sensor 14 distributed within the classroom. The light intensity sensor includes a first light intensity sensor 11 and a second light intensity sensor 12 installed on the classroom ceiling. The first light intensity sensor 11 is installed on the ceiling near the blackboard with its acquisition end facing the student desk area, and the second light intensity sensor 12 is installed on the ceiling near the student area with its acquisition end facing the podium area. The temperature and humidity sensor 13 and the CO2 concentration sensor 14 are respectively installed on the side wall and rear wall of the classroom, and the data output terminals of each sensor are respectively connected to the data input terminals of the heterogeneous edge computing module 20. The relative positional relationship between the student perception submechanism and the reconfigurable slide rail bracket 2 is as follows: The reconfigurable slide rail bracket 2 is fixedly installed on the ceiling directly above the student desk area. It is arranged in a W shape and is composed of four slide rail units 21. The left half is formed by the first and second slide rail units forming a V shape, and the right half is formed by the third and fourth slide rail units forming a V shape. The left and right sides meet and connect in the middle of the classroom through a middle hinge. The included angle between each slide rail unit 21 can be adjusted between 120° and 150° according to the width of the classroom and the arrangement of the student desks. The left and right ends of the reconfigurable slide rail bracket 2 are fixed to the walls on both sides of the classroom by expansion bolts through the fixing seat 23. The distributed camera array 5 is installed below the reconfigurable slide rail bracket 2 through the electric sliding seat 3 and the magnetic quick-installation base 4. Each camera 5 is set downwards and the field of view covers the student desk area. The relative positions of the teacher tracking camera assembly and the ceiling in front of the classroom are as follows: the panoramic positioning camera 6 is fixedly installed on the ceiling in front of the classroom, and its field of view covers the entire teacher activity area; the two-degree-of-freedom motorized pan-tilt head 8 is fixedly installed on the ceiling in front of the classroom and located next to the panoramic positioning camera 6; the tracking close-up camera 7 is installed on the two-degree-of-freedom motorized pan-tilt head 8. The relative positional relationship between the spatial positioning component and the classroom is as follows: the spatial positioning tag 10 is worn by the teacher, and the spatial positioning base station 9 is fixed on the walls at the four corners of the classroom. The signal connection relationship between the hardware clock synchronization board 19 and each sensor is as follows: the synchronization pulse signal generated by the FPGA synchronization controller is transmitted through the synchronization pulse output interface of the PPS second pulse generator to the trigger signal input terminal of each fixedly installed camera (each camera 5 of the distributed camera array 5, the panoramic positioning camera 6, and the tracking close-up camera 7) via a wired signal line, and then transmitted to the trigger signal input terminal of the space positioning base station 9 via a wired signal line. The space positioning base station 9 sends a wireless synchronization trigger signal to the space positioning tag 10 through its built-in wireless synchronization pulse transmission module, thereby realizing the wireless synchronization triggering of the mobile terminal space positioning tag 10. The data connection relationship between the heterogeneous edge computing module 20 and each sensor is as follows: the data input terminal of the heterogeneous edge computing module 20 is connected to the data output terminal of each camera 5 in the distributed camera array 5, the data output terminal of the panoramic positioning camera 6, the data output terminal of the tracking close-up camera 7, the data output terminal of the spatial positioning base station 9 and the spatial positioning tag 10, and the data output terminal of each sensor (first illuminance sensor 11, second illuminance sensor 12, temperature and humidity sensor 13, CO2 concentration sensor 14) of the environmental perception sub-mechanism; The control connection relationship between the hierarchical feedback actuator and the heterogeneous edge computing module 20 is as follows: the control signal output terminal of the heterogeneous edge computing module 20 is connected to the control terminal of the light strip drive circuit of the breathing light strip, the control terminal of the touch display screen 16, the control terminal of the wireless communication module of each RGB indicator 17, the RS485 bus interface of the environmental control output interface 18, and the control terminal of the Zigbee wireless interface. By adopting the above technical solution, the time synchronization accuracy of each sensor is better than 1 millisecond through a combination of wired and wireless synchronous signal transmission. The heterogeneous edge computing module 20 completes the fusion processing of multi-source data and attention state inference locally in the classroom, avoiding the transmission delay and biometric leakage risks caused by uploading video and image data to the cloud.

[0027] This embodiment details the specific operation process of the aforementioned teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms. Before starting the equipment, the operator adjusts each slide rail unit 21 of the reconfigurable slide rail bracket 2 to a suitable angle (usually 120° to 150°) according to the actual width of the classroom and the arrangement of the student desks, using the hinge 22. During adjustment, the operator pulls the clamping handle to disengage the annular toothed ring from the annular toothed groove. After rotating the slide rail unit 21 to the required angle, the operator releases the clamping handle, and the return spring drives the annular toothed ring to re-engage and lock with the annular toothed groove. The reconfigurable slide rail bracket 2 is then fixed to the walls on both sides of the classroom using expansion bolts via the fixing seat 23, thus completing the overall installation of the reconfigurable slide rail bracket 2. Each camera 5 of the distributed camera array 5 is installed on each electric sliding seat 3 via a magnetic quick-release base 4, and the orientation and spacing of each camera 5 are adjusted according to the actual coverage requirements of the student desk area; the panoramic positioning camera 6 and the two-degree-of-freedom electric pan-tilt head 8 with a tracking close-up camera 7 are fixedly installed on the ceiling at the front of the classroom, and the four spatial positioning base stations 9 are fixed on the walls at the four corners of the classroom respectively, and the spatial positioning tag 10 is worn by the teacher; the hardware clock synchronization board 19 and the heterogeneous edge computing module 20 are installed in the rack at the front of the classroom, and the signal lines between each sensor and the hardware clock synchronization board 19 and between each sensor and the heterogeneous edge computing module 20 are connected. After the device is started, the FPGA synchronization controller of the hardware clock synchronization board 19 continuously generates high-precision synchronization pulse signals, which are output through the synchronization pulse output interface of the PPS second pulse generator. The synchronization pulse signals are transmitted through wired signal lines to the trigger signal input terminals of each camera 5, panoramic positioning camera 6, and tracking close-up camera 7 in the distributed camera array 5, triggering each camera to synchronously expose and acquire images. The synchronization pulse signals are also transmitted through wired signal lines to each spatial positioning base station 9. After receiving the synchronization pulses, the spatial positioning base station 9 sends wireless synchronization trigger signals to the spatial positioning tag 10 through its built-in wireless synchronization pulse transmission module, so that the spatial positioning tag 10 is synchronized with each camera in time. Under the unified trigger of the hardware clock synchronization board 19, all sensors simultaneously acquire student head posture images, teacher panoramic images, teacher close-up images, and teacher three-dimensional spatial coordinate data. All data have a unified hardware timestamp, and the synchronization accuracy is better than 1 millisecond. Data collected by each sensor is transmitted in real time to the heterogeneous edge computing module 20. The heterogeneous edge computing module 20 performs fusion inference on the synchronously collected multi-source data: the NPU neural network processing unit runs a pre-trained attention recognition model to perform parallel inference on the student images collected by the distributed camera array 5, identify the head posture angle, facial orientation, and gaze direction of each student, and output the attention state label of each student; at the same time, the NPU neural network processing unit performs inference on the teacher images collected by the panoramic positioning camera 6 and the tracking close-up camera 7, and identifies the teacher's teaching position area, facing direction, and gaze direction; the DSP digital signal processor performs temporal fusion on the student attention state data, teacher behavior feature data, and teacher three-dimensional spatial coordinate data provided by the spatial positioning label 10, and establishes the temporal correlation between teacher and student behavior based on the unified hardware timestamp provided by the hardware clock synchronization board 19; the CPU central processing unit executes decision logic according to the fusion result to determine the final attention state of each student and the corresponding feedback strategy; The control signal output terminal of the heterogeneous edge computing module 20 outputs the feedback strategy to the hierarchical feedback actuator: based on the attention status label, the RGB indicator light 17 provides real-time reminders to students (green indicates focus, yellow indicates mild distraction, and red indicates severe distraction), the touch screen 16 displays a heat map of the whole class's attention distribution and abnormal warning information to the teacher, and the color and flashing frequency of the breathing light strip provides the teacher with prompts on the overall attention status of the classroom; based on environmental perception data, the classroom lighting, air conditioning and fresh air system are automatically controlled through the environmental control output interface 18 via the Internet of Things gateway, adjusting the lighting brightness when the illuminance is lower than the set threshold, and starting the fresh air system or adjusting the air conditioning when the CO2 concentration or temperature and humidity deviate from the comfort threshold; During device operation, the ARM servo controller 83 of the two-degree-of-freedom electric pan-tilt head 8 continuously receives the teacher's position information in the classroom as identified in real time by the panoramic positioning camera 6, and drives the horizontal rotation mechanism 81 and the pitch rotation mechanism 82 to keep the tracking close-up camera 7 always pointing at the teacher; at the same time, the ARM servo controller 83 also receives the images captured by the tracking close-up camera 7, and performs fine-tuning compensation by comparing the center offset of the teacher target in the image of the tracking close-up camera 7, forming a dual-loop closed-loop control that combines coarse positioning and fine positioning, so as to realize stable automatic tracking close-up shooting of the teacher's behavior; If the classroom layout changes (such as adding or removing desks, adjusting the arrangement of desks and chairs, etc.), the operator can pull the clamping handle on the hinge 22 to disengage the annular toothed ring from the annular toothed groove, rotate each slide rail unit 21 to the required angle, and then release the clamping handle. The return spring will drive the annular toothed ring to re-engage and lock with the annular toothed groove, completing the readjustment of the reconfigurable slide rail bracket 2 so that the overall coverage of the reconfigurable slide rail bracket 2 is adapted to the newly arranged student desk area. If it is necessary to add or remove the number of cameras, the operator can quickly install or remove the cameras 5 using the magnetic quick-release base 4. After the adjustment is completed, the device can be put back into operation according to the above operating process. Throughout the entire operation, all data processing and inference tasks are completed on the heterogeneous edge computing module 20 located in the classroom, with no video or image data uploaded to the cloud, effectively protecting the biometric privacy of students and teachers; the behavioral data of teachers and students are synchronized at the sub-millisecond level through the hardware clock synchronization board 19, ensuring the precise alignment of the temporal relationship between teachers and students' behaviors, and providing a reliable data foundation for the accurate judgment of classroom attention status.

[0028] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A teacher-student attention collaborative tracking and feedback adjustment device for smart classrooms (1), characterized in that, This includes a multimodal collaborative sensing mechanism, a data synchronization and edge computing mechanism, and a hierarchical feedback execution mechanism; The multimodal collaborative sensing mechanism includes a student sensing sub-mechanism and a teacher sensing sub-mechanism. The student sensing sub-mechanism is installed on the ceiling of the classroom (1) and includes a reconfigurable slide rail bracket (2) fixed to the ceiling, an electric sliding seat (3) installed on the slide rail bracket and sliding along it, and a distributed camera (5) array installed on the electric sliding seat (3) via a magnetic quick-release base (4). The reconfigurable slide rail bracket (2) is modularly spliced ​​from several slide rail units (21), and the connection angle and relative position between each slide rail unit (21) are adjustable. The field of view of the distributed camera (5) array is oriented towards the student area. The teacher sensing sub-mechanism includes a spatial positioning component worn on the teacher and a teacher tracking camera component installed in front of the classroom (1). The data synchronization and edge computing mechanism is installed in a rack in front of the classroom (1), and has a built-in hardware clock synchronization board (19) and a heterogeneous edge computing module (20). The hardware clock synchronization board (19) is provided with a synchronization pulse output interface. The synchronization pulse output interface is connected to the trigger signal input terminal of each camera of the distributed camera array (5) and the teacher tracking camera component through signal lines. The synchronization pulse output interface is also connected to the spatial positioning base station (9) through signal lines. The spatial positioning base station (9) has a built-in wireless synchronization pulse transmission module for sending wireless synchronization trigger signals to the spatial positioning tag (10). The data input terminal of the heterogeneous edge computing module (20) is connected to the data output terminal of the distributed camera array (5), the teacher tracking camera component and the spatial positioning component. The hierarchical feedback actuator includes a teacher-side feedback actuator (15) installed at the front edge of the podium, a student-side feedback actuator installed on the student's desk, and an environmental control output interface (18) installed on the side wall of the classroom (1). The control signal output terminal of the heterogeneous edge computing module (20) is connected to the control signal input terminals of the teacher-side feedback actuator (15), the student-side feedback actuator, and the environmental control output interface (18), respectively.

2. The apparatus according to claim 1, characterized in that, The spatial positioning component includes a spatial positioning tag (10) worn by the teacher and a spatial positioning base station (9) fixed at the four corners of the classroom (1). The spatial positioning base station (9) has a built-in wireless synchronization pulse transmission module. The hardware clock synchronization board (19) transmits the synchronization pulse to each spatial positioning base station (9) through a wired signal line. The spatial positioning base station (9) sends a wireless synchronization trigger signal to the spatial positioning tag (10).

3. The apparatus according to claim 1, characterized in that, The teacher tracking camera assembly includes a fixed panoramic positioning camera (6) and a tracking close-up camera (7) mounted on a two-degree-of-freedom electric pan-tilt head (8). The panoramic positioning camera (6) is fixedly mounted on the ceiling in front of the classroom (1) and its field of view covers the entire teacher's activity area. The two-degree-of-freedom electric pan-tilt head (8) is fixedly mounted on the ceiling in front of the classroom (1) and is located next to the panoramic positioning camera (6). The two-degree-of-freedom electric pan-tilt head (8) includes a horizontal rotation mechanism (81) and a pitch rotation mechanism (82). The two-degree-of-freedom electric pan-tilt head (8) has a built-in ARM servo controller (83). The ARM servo controller (83) is connected to the panoramic positioning camera (6) and drives the horizontal rotation mechanism (81) and the pitch rotation mechanism (82) according to the teacher's position identified by the panoramic positioning camera (6) so that the tracking close-up camera (7) is aimed at the teacher.

4. The apparatus according to claim 1, characterized in that, The multimodal collaborative sensing mechanism also includes an environmental sensing sub-mechanism, which includes a light intensity sensor, a temperature and humidity sensor (13), and a CO2 concentration sensor (14) distributed in the classroom (1). The light intensity sensor includes a first light intensity sensor (11) and a second light intensity sensor (12) installed on the ceiling of the classroom (1). The first light intensity sensor (11) is installed on the ceiling near the blackboard and its acquisition end faces the student desk area. The second light intensity sensor (12) is installed on the ceiling near the student area and its acquisition end faces the podium area. The temperature and humidity sensor (13) and the CO2 concentration sensor (14) are respectively installed on the side wall and the back wall of the classroom (1).

5. The apparatus according to claim 1, characterized in that, The hardware clock synchronization board (19) is equipped with an FPGA synchronization controller and a PPS second pulse generator electrically connected to the FPGA synchronization controller; the FPGA synchronization controller is equipped with a timing signal receiving end, and the PPS second pulse generator is equipped with the synchronization pulse output interface.

6. The apparatus according to claim 1, characterized in that, The heterogeneous edge computing module (20) includes an NPU neural network processing unit, a DSP digital signal processor and a CPU central processing unit interconnected via a high-speed bus. The heterogeneous edge computing module (20) is connected to the hardware clock synchronization board (19) via a PCIe bus.

7. The apparatus according to claim 1, characterized in that, The teacher-side feedback actuator (15) includes a breathing light strip installed at the front edge of the podium and a touch display screen (16) embedded in the podium desktop; the breathing light strip includes at least one set of RGB full-color LED beads and a light strip driving circuit, and the light strip driving circuit is connected to the control signal output terminal of the heterogeneous edge computing module (20).

8. The apparatus according to claim 1, characterized in that, The student-side feedback actuator includes an RGB indicator light (17) embedded in the desktop of each student's desk. The RGB indicator light (17) is connected to the control signal output terminal of the heterogeneous edge computing module (20) via a wireless communication module.

9. The apparatus according to claim 1, characterized in that, The environmental control output interface (18) includes an RS485 bus interface and a Zigbee wireless interface installed on the side wall of the classroom (1). The RS485 bus interface and the Zigbee wireless interface are respectively connected to the Internet of Things gateway of the classroom (1).

Images