Classroom environment intelligent supervision method and system based on data analysis

Abstract

The invention discloses a classroom environment intelligent supervision method and system based on data analysis, and the method comprises the steps: obtaining multi-source heterogeneous data in a classroom environment, eliminating the sensor noise of the multi-source heterogeneous data through employing an improved Kalman filtering algorithm, carrying out the time-space alignment of the data after noise elimination, and obtaining the aligned multi-source heterogeneous data; establishing a CRNN three-layer convolutional recurrent neural network model, and inputting the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition to obtain a classroom teaching environment state; and carrying out classroom risk early warning and self-adaptive classroom environment regulation and control according to the classroom teaching environment state. The classroom emergency risk can be actively prevented, automatic adjustment can be achieved through an intelligent air conditioner and an illumination system, a safe and comfortable teaching environment is created, and classroom efficiency improvement and teacher and student health guarantee are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of big data analysis, and in particular to a method and system for intelligently supervising a classroom environment based on data analysis. Background Art

[0002] With the development of big data analytics, data analysis is increasingly being used in classroom supervision. However, traditional video surveillance systems currently record only the visuals and lack the ability to parse behavioral semantics. Furthermore, environmental control relies on fixed thresholds, ignoring the dynamic demands of the teaching environment, preventing teachers from obtaining real-time quantitative feedback on the overall learning status of the class. While existing technologies can detect student posture, they suffer from single-modal data analysis, resulting in high rates of misjudgment. Data analysis also lacks correlation analysis of the teaching process and linearization of control strategies. Therefore, improving the efficiency and accuracy of traditional classroom supervision remains a pressing technical challenge. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and to design a classroom environment intelligent supervision method and system based on data analysis.

[0004] To achieve the above-mentioned purpose, the technical solution of the present invention is as follows: further, in the above-mentioned classroom environment intelligent supervision method based on data analysis, the classroom environment intelligent supervision method includes the following steps: Acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; Eliminating sensor noise from the multi-source heterogeneous data using an improved Kalman filtering algorithm, and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; Establishing a CRNN three-layer convolutional recurrent neural network model, inputting the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtaining the classroom teaching environment state; Classroom risk warning and adaptive classroom environment regulation are carried out according to the classroom teaching environment status.

[0005] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the multi-source heterogeneous data in the classroom environment is obtained, and the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data, including: Acquiring environmental image data in a classroom environment through an image sensor, wherein the environmental image data includes at least a color image, depth information, and a backlight scene; A channel ring microphone array is used to acquire individual student voice data, teacher lecture sounds, student discussion sounds, ambient noise, and writing friction sounds in the classroom environment to obtain environmental sound data. The temperature and humidity data in the classroom environment are obtained through the temperature and humidity sensors to obtain the ambient temperature and humidity data.

[0006] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the improved Kalman filtering algorithm is used to eliminate the sensor noise of the multi-source heterogeneous data, and the noise-eliminated data is subjected to spatiotemporal alignment to obtain aligned multi-source heterogeneous data, including: Using an improved Kalman filtering algorithm to eliminate sensor noise from the multi-source heterogeneous data, establishing an adaptive noise covariance matrix for different sensor characteristics, and dynamically adjusting filtering parameters; The timestamps of image, sound, temperature and humidity data are aligned, cross-modal data association is achieved by extending the state vector, and audio energy fluctuations are synchronously analyzed with student movements in the video to obtain aligned multi-source heterogeneous data.

[0007] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the method of eliminating sensor noise of the multi-source heterogeneous data by using an improved Kalman filtering algorithm, and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data further includes: Based on the PTP precise time protocol, devices in the noise-reduced multi-source heterogeneous data are always synchronized, and network delay differences are eliminated through hardware timestamps to obtain time-aligned data; The physical positions of the sensor nodes in the time-aligned data are mapped to the camera coordinate system, and the correspondence between the desks and the image pixels is established through the QR code calibration plate to obtain the environmental mapping data. The tracking path of the PTZ camera is dynamically corrected using the sound source localization results to obtain the visual association data; The combined actions of standing discussion and dozing off at the desk are identified through skeleton key point tracking, and the duration is determined by combining time series analysis; a comprehensive scoring model is constructed by analyzing facial orientation, gaze focus, and micro-expressions.

[0008] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the establishment of the CRNN three-layer convolutional recursive neural network model includes: The first layer of the CRNN three-layer convolutional recurrent neural network model is used to extract image edge features, the second layer is used to identify dynamic behaviors, and the third layer is used to fuse time series data; A bidirectional LSTM network is introduced into the recurrent layer of the model to process the temporal dependencies of sound signals and capture the changing patterns of environmental states. The CRNN three-layer convolutional recurrent neural network model is trained using the historical classroom environment dataset in the database. The importance of different modal data is weighted through the attention mechanism to obtain a trained CRNN three-layer convolutional recurrent neural network model.

[0009] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the aligned multi-source heterogeneous data is input into the CRNN three-layer convolutional recurrent neural network model for identification to obtain the classroom teaching environment state, including: The classroom teaching environment status includes at least the teaching behavior recognition status and the environment anomaly detection status; The quality of teacher-student interaction is judged based on the activity of group discussions, and the student concentration is analyzed through facial orientation and eye focus to obtain the teaching behavior recognition status; Identify environmental anomaly detection status through equipment failures, safety hazards and emergencies in the classroom environment.

[0010] Furthermore, in the above-mentioned intelligent classroom environment supervision method based on data analysis, the classroom risk warning and adaptive classroom environment regulation based on the classroom teaching environment state include: Trigger audible and visual alarms for fires and sudden loud noise events, and push information to the administrator terminal; Based on the duration of temperature and humidity deviation from the comfort zone and the proportion of student distraction indicators, a yellow / orange / red warning is generated to prompt teachers to adjust their teaching strategies; The air conditioning and lighting systems are controlled through the PID algorithm. When it is detected that the student's concentration is declining, interactive courseware is pushed or the teaching scene is switched.

[0011] Furthermore, in the classroom environment intelligent supervision system based on data analysis, the classroom environment intelligent supervision system includes the following modules: An environmental data acquisition module is used to acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; a data alignment processing module, configured to eliminate sensor noise of the multi-source heterogeneous data by using an improved Kalman filtering algorithm, and perform spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; An environment state recognition module is used to establish a CRNN three-layer convolutional recurrent neural network model, input the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtain the classroom teaching environment state; The classroom environment supervision module is used to provide classroom risk warning and adaptive classroom environment regulation according to the status of the classroom teaching environment.

[0012] Furthermore, in the classroom environment intelligent supervision system based on data analysis, the data encryption module includes the following submodules: An image data acquisition submodule, configured to acquire environmental image data in a classroom environment through an image sensor, wherein the environmental image data includes at least a color image, depth information, and a backlight scene; The sound data acquisition submodule is used to use the channel ring microphone array to acquire individual student voice data, teacher lecture sound, student discussion sound, environmental noise and writing friction sound in the classroom environment to obtain environmental sound data; The temperature and humidity data acquisition submodule is used to obtain temperature data and humidity data in the classroom environment through temperature and humidity sensors to obtain ambient temperature and humidity data.

[0013] Furthermore, in the classroom environment intelligent supervision system based on data analysis, the data processing module includes the following submodules: The time alignment submodule is used to synchronize devices in noise-reduced multi-source heterogeneous data based on the PTP precise time protocol, eliminate network delay differences through hardware timestamps, and obtain time-aligned data; A spatial alignment submodule is used to map the physical positions of the sensor nodes in the time-aligned data to the camera coordinate system, establish the correspondence between the desks and the image pixels through the QR code calibration plate, obtain environmental mapping data, and dynamically correct the tracking path of the PTZ camera using the sound source localization results to obtain visual association data; The data processing and analysis submodule is used to identify the combined actions of standing discussion and dozing off on the table through skeleton key point tracking, determine the duration through time series analysis, and analyze facial orientation, gaze focus, and micro-expressions to construct a comprehensive scoring model.

[0014] Its beneficial effect lies in the establishment of a three-dimensional monitoring system through a multi-source, heterogeneous data collection mechanism. This solution integrates multimodal data, including environmental images, sound, temperature, and humidity, overcoming the limitations of single-sensor monitoring and comprehensively capturing classroom dynamics. It not only captures students' attention through image recognition, but also identifies unusual sounds through sound analysis. Combined with temperature and humidity data, it provides real-time assessment of environmental comfort, providing comprehensive data support for classroom monitoring. Secondly, an improved Kalman filter algorithm and spatiotemporal alignment technology ensure data quality. To address sensor noise, an optimized filter model is used to improve the signal-to-noise ratio and avoid misjudgments. Spatiotemporal alignment eliminates temporal and spatial differences in multi-source data, ensuring the integration of image, sound, and environmental parameters in a unified spatiotemporal coordinate system, laying a solid foundation for subsequent intelligent analysis. Finally, the CRNN model and adaptive control form a closed-loop management system. Leveraging the spatiotemporal feature extraction capabilities of convolutional recurrent neural networks, this approach accurately identifies classroom environmental conditions and triggers risk warnings and environmental control in real time. This mechanism not only proactively prevents sudden risks in the classroom, but also automatically adjusts through intelligent air conditioning and lighting systems to create a safe and comfortable teaching environment, which is conducive to improving classroom efficiency and protecting the health of teachers and students. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0016] Figure 1 Schematic diagram of a first embodiment of a method for intelligently supervising a classroom environment based on data analysis in an embodiment of the present invention; Figure 2 Schematic diagram of a second embodiment of a method for intelligently supervising a classroom environment based on data analysis in an embodiment of the present invention; Figure 3 A schematic diagram of a first embodiment of a classroom environment intelligent monitoring system based on data analysis in an embodiment of the present invention; Figure 4 This is a schematic diagram of a CRNN three-layer recursive convolutional neural network model of a classroom environment intelligent supervision method based on data analysis in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the classroom environment intelligent supervision method based on data analysis includes the following steps: Step 101: Acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; Specifically, in this embodiment, environmental image data in a classroom environment is acquired through an image sensor, and the environmental image data includes at least a color image, depth information, and a backlight scene; A channel ring microphone array is used to acquire individual student voice data, teacher lecture sounds, student discussion sounds, ambient noise, and writing friction sounds in the classroom environment to obtain environmental sound data. The temperature and humidity data in the classroom environment are obtained through the temperature and humidity sensors to obtain the ambient temperature and humidity data.

[0020] 1.1 The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the classroom environment intelligent supervision method based on data analysis includes the following steps: Step 101: Acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; Specifically, in this embodiment, environmental image data in a classroom environment is acquired through an image sensor, and the environmental image data includes at least a color image, depth information, and a backlight scene; A channel ring microphone array is used to acquire individual student voice data, teacher lecture sounds, student discussion sounds, ambient noise, and writing friction sounds in the classroom environment to obtain environmental sound data. The temperature and humidity data in the classroom environment are obtained through the temperature and humidity sensors to obtain the ambient temperature and humidity data.

[0021] 1.2 Distributed Acoustic Perception Layer Equipment Deployment: Microphone Array: An 8-channel circular microphone array, arranged in a double-layer "4 inside, 4 outside" layout (1.2-meter inner diameter, 3-meter outer diameter), is suspended in the center of the classroom, 2.5 meters above the ground. Voiceprint Collector: Four directional microphones are embedded in the edges of the desks (2 per column) to collect individual voices (for answering questions) and the friction sound of writing. Functional Design: Sound Source Separation: Beamforming technology is used to distinguish between teacher lectures, student discussions, and ambient noise (the sound of flipping pages), and voiceprint ID binding is supported (to identify specific student speech). Anomaly Detection: Sudden high-decibel events (such as falls and collisions) are identified, triggering emergency steering of the PTZ camera.

[0022] 1.3 Environmental Sensing Network Layer Equipment Deployment: IoT Nodes: One wireless sensor node is deployed for every 3 square meters (approximately 15-20 per standard classroom), fixed to the side of a desk or wall at a height of 1.2 meters (the human breathing zone). Sensor Types: Air Quality: Non-Dispersive Infrared (NDIR) CO2 sensor (range 0-5000 ppm, accuracy ±50 ppm), PM2.5 laser sensor. Physical Environment: Digital temperature and humidity sensor (accuracy ±0.3°C), illuminance sensor (range 0-1000 Lux). Energy Monitoring: Smart sockets collect power consumption of devices such as projectors and air conditioners. Functional Design: Dynamic Sampling: Data is uploaded every 10 seconds under normal circumstances. When the CO2 concentration exceeds 800 ppm or the temperature and humidity change rate exceeds 5% / minute, it switches to a high-frequency sampling rate of 1 second. Self-Calibration Mechanism: Baseline calibration is initiated at 2:00 AM daily, synchronizing all node data using the sensor in the podium area as the benchmark.

[0023] Step 102: using an improved Kalman filtering algorithm to eliminate sensor noise from the multi-source heterogeneous data, and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; Specifically, in this embodiment, an improved Kalman filtering algorithm is used to eliminate the sensor noise of the multi-source heterogeneous data, an adaptive noise covariance matrix is ​​established for different sensor characteristics, and filtering parameters are dynamically adjusted; The timestamps of image, sound, temperature and humidity data are aligned, cross-modal data association is achieved by extending the state vector, and audio energy fluctuations are synchronously analyzed with student movements in the video to obtain aligned multi-source heterogeneous data.

[0024] Based on the PTP precise time protocol, devices in the noise-reduced multi-source heterogeneous data are always synchronized, and network delay differences are eliminated through hardware timestamps to obtain time-aligned data; The physical positions of the sensor nodes in the time-aligned data are mapped to the camera coordinate system, and the correspondence between the desks and the image pixels is established through the QR code calibration plate to obtain the environmental mapping data. The tracking path of the PTZ camera is dynamically corrected using the sound source localization results to obtain the visual association data; The combined actions of standing discussion and dozing off at the desk are identified through skeleton key point tracking, and the duration is determined by combining time series analysis; a comprehensive scoring model is constructed by analyzing facial orientation, gaze focus, and micro-expressions.

[0025] 2.1 Multi-source Data Spatiotemporal Alignment Time Synchronization: PTP (Precision Time Protocol) is used to synchronize all device clocks to the microsecond level, and hardware timestamping eliminates network latency differences. Spatial Calibration: Visual-Environmental Data Mapping: The physical location of sensor nodes is mapped to the camera coordinate system. For example, a QR code calibration plate is used to establish the correspondence between desks and image pixels. Acoustic-Visual Correlation: Sound source localization results (teacher movement trajectory) are used to dynamically correct the PTZ camera's tracking path.

[0026] 2.2 Data Cleaning and Feature Extraction for Noise Suppression: Visual Data: Motion optical flow analysis is used to distinguish valid actions (raising a hand) from environmental interference (shaking curtains). Acoustic Data: Spectral subtraction is used to eliminate steady-state noise (humming of the air conditioner) while retaining transient features (coughing). Behavioral Semantic Analysis: Group Behavior Recognition: Skeleton keypoint tracking (17 joints) is used to identify complex actions such as "standing discussion" and "dozing off at the desk." Time series analysis is used to determine duration (continuously looking down for more than 2 minutes is marked as distracted). Individual Focus Calculation: Facial orientation (angle deviation from the blackboard), gaze focus (coordinates of the intersection with the projection screen), and micro-expressions (frowning frequency) are analyzed to construct a comprehensive scoring model.

[0027] 2.3 Teaching Context Inference Engine Scenario Classification Model: Input: Course schedule information associated with the current timestamp ("Physics Experiment Class"), equipment status (laboratory equipment powered on), and group behavior patterns (multiple people gathered around the workbench). Output: Dynamically label teaching phases as "Theoretical Explanation," "Group Experiment," "Summary and Q&A," etc., and load corresponding supervision policy templates. Knowledge Graph Query: 87 predefined teaching events ("Teacher Writing Key Points on the Blackboard," "Student Group Argument") are used. When relevant features are detected, event chain reasoning is triggered (e.g., detecting three students leaving their seats simultaneously → querying the schedule to see if it is an active class → marking it as an anomaly if it is not).

[0028] Step 103: Establish a CRNN three-layer convolutional recurrent neural network model, input the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtain the classroom teaching environment state; Specifically, the first layer of the CRNN three-layer convolutional recurrent neural network model in this embodiment is used to extract image edge features, the second layer is used to identify dynamic behaviors, and the third layer is used to fuse time series data; A bidirectional LSTM network is introduced into the recurrent layer of the model to process the temporal dependencies of sound signals and capture the changing patterns of environmental states. The CRNN three-layer convolutional recurrent neural network model is trained using the historical classroom environment dataset in the database. The importance of different modal data is weighted through the attention mechanism to obtain a trained CRNN three-layer convolutional recurrent neural network model.

[0029] The classroom teaching environment status includes at least the teaching behavior recognition status and the environment anomaly detection status; The quality of teacher-student interaction is judged based on the activity of group discussions, and the student concentration is analyzed through facial orientation and eye focus to obtain the teaching behavior recognition status; Identify environmental anomaly detection status through equipment failures, safety hazards and emergencies in the classroom environment.

[0030] 1. Model Architecture and Core Design: The CRNN is a hybrid model that integrates convolutional neural networks (CNNs) and recurrent neural networks (RNNs, LSTMs / GRUs). It is suitable for processing spatiotemporal sequence data. In this technical solution, the "three layers" generally refer to the following: The first layer: the convolutional layer (CNN): extracts spatial and local features from multi-source heterogeneous data. For environmental image data: convolutional kernels (3×3, 5×5) are used to extract visual features (such as student posture and classroom object placement). For environmental sound data: the audio is first converted to Mel-spectrograms (Mel-spectrograms) or MFCC features, and then convolution is used to extract acoustic features (such as speech and noise intensity). For temperature and humidity data: although scalar data, it can be integrated with image and sound features through dimensionality expansion (upgrading to 1×1×2), or a separate lightweight convolutional layer can be designed to process temporal trends. The second layer: the recurrent layer (RNN / LSTM / GRU): captures the temporal dependencies of the data (i.e., "temporal features"). Input: the feature sequence output by the convolutional layer (image frame sequence, sound clip sequence, temperature and humidity time series). Function: Models the dynamic changes of classroom environment data over time through hidden state transfer (for example, analyzing fluctuations in sound decibels over a period of time to determine classroom activity, or predicting comfort risks through changing trends in temperature and humidity). Third layer: Fully connected layer (FC) + output layer: Fusion of multimodal features and classification / regression. Fully connected layer: Concatenates or weightedly fuses the features (spatial features, temporal features) of the convolutional and recurrent layers to form a global representation. Output layer: Based on the task design, outputs the classroom status category ("normal teaching", "abnormally noisy", "equipment failure") through softmax, or outputs the risk score of the environmental parameters through regression.

[0031] 2. The technical solution for multimodal data fusion involves three types of heterogeneous data: image, sound, and temperature and humidity. Cross-modal fusion must be implemented within the model: Early Fusion: In the input layer or early convolutional layer, the different modal data are converted into feature vectors of uniform dimension and then concatenated (image features, sound spectrum features, and temperature and humidity scalars are combined into a single feature vector). Late Fusion: Each modal data is processed separately through independent CNN+RNN branches, and features are merged in the fully connected layer (weighted summation or concatenation of the outputs of the image, sound, and temperature and humidity branches). Hybrid Fusion: Combining the two aforementioned approaches, for example, using early fusion for image and sound (both sensor data) and late fusion for temperature and humidity data (environmental parameters are processed independently). II. Model Building Steps 1. Data Preprocessing and Spatiotemporal Alignment (Preliminary Step) Noise Removal: Utilizing improved Kalman filter algorithms (extended Kalman filter, unscented Kalman filter) to denoise raw sensor data (especially time-series data such as temperature, humidity, and sound decibels) and reduce measurement errors. Spatiotemporal alignment: Temporal alignment: Unify the timestamps of multi-source data (based on the video frame rate, interpolate or downsample the sound and temperature and humidity data to ensure that each frame corresponds to the sound / temperature and humidity data at the same time). Spatial alignment: Scale the image data to a uniform size (224×224), convert the sound data into fixed-length segments (1 second per segment), and aggregate the temperature and humidity data into statistics (mean, variance) by time window (every 10 seconds).

[0032] 3. Network Architecture Design: Convolutional Layer Configuration: Image Branch: Uses the classic CNN architecture (a simplified version of ResNet and VGG), consisting of 2-3 layers of convolution + pooling to extract visual features (student attention status, teacher position). Sound Branch: 1D or 2D convolution (for spectrograms) to capture frequency-time domain features (keywords in speech, noise peaks). Temperature and Humidity Branch: 1D convolution layer to extract temporal fluctuation features (sudden temperature rise, abnormal humidity fluctuations). Recursive Layer Selection: If the data has strong temporal dependencies (continuous classroom sounds, temperature and humidity fluctuations), use LSTM / GRU (to avoid the RNN gradient vanishing problem); if computational efficiency is a priority, use a simple RNN or a temporal convolutional network (TCN). Number of Recursive Layers: Typically 1-2 layers, with the number of hidden units in each layer set based on data complexity (128, 256). Fusion Layer Design: After the recursive layer, a fully connected layer is used to concatenate multimodal features (concat operation), and a dropout layer is added to prevent overfitting.

[0033] 4. Training configuration loss function: Select according to the output task, use cross entropy loss for classification tasks, and mean squared error (MSE) for regression tasks. Optimizer: Adam, SGD, etc. The learning rate can be adjusted through cosine annealing or step decay. Data enhancement: Flip, crop, and adjust the brightness of image data; add Gaussian noise and time stretch to sound data; enhance the generalization of temperature and humidity data by generating simulated sequences. Training process: Use a labeled classroom environment dataset (normal / abnormal state labels), input the model in batches, update the parameters through backpropagation, and monitor overfitting on the validation set. III. Model utilization and application process 1. Input and inference Input data: Preprocess the multi-source data (image frames, sound clips, temperature and humidity values) that have been aligned in time and space into a format accepted by the model (normalization, dimension adaptation). Forward propagation: The convolutional layer extracts spatial features from each modality (e.g., students raising their hands in an image, applause in an audio file); the recursive layer processes feature sequences to capture temporal dependencies (e.g., temperature and humidity exceeding a threshold for 5 consecutive minutes, or a continuous increase in the decibel level of sound within 10 seconds); and the fully connected layer fuses features and outputs classroom status (e.g., "high student distraction rate," "abnormally hot equipment," "stuffy environment").

[0034] 5. Classroom state recognition and output state classification: Output discrete categories ("normal teaching," "abnormal noise," "unsuitable environment") or continuous values ​​(classroom activity score, risk index). Multimodal collaborative decision-making: For example, if an image shows frequent standing of students (visual features), noisy sounds (acoustic features), and normal temperature and humidity (environmental parameters), the combined judgment is "active classroom interaction." If the temperature and humidity are also above standard, the judgment is "high load risk."

[0035] 6. Risk Warning and Environmental Control: Risk warnings trigger alerts based on output status (using thresholds to determine when the probability of an "abnormal state" exceeds 80%), sending a notification to the teacher or administrator. Adaptive control: If a "stifling environment" is detected (temperature and humidity data combined with image features of a languid student posture), the air conditioner automatically adjusts the temperature. If "abnormal equipment noise" is detected (abnormal sound spectrum combined with camera footage of smoke coming out of the equipment), the device is powered off and an alarm is triggered.

[0036] Step 104: Perform classroom risk warning and adaptive classroom environment regulation according to the classroom teaching environment status.

[0037] Specifically, in this embodiment, an audible and visual alarm is triggered for fire and sudden loud noise events, and the information is pushed to the administrator terminal; Based on the duration of temperature and humidity deviation from the comfort zone and the proportion of student distraction indicators, a yellow / orange / red warning is generated to prompt teachers to adjust their teaching strategies; The air conditioning and lighting systems are controlled through the PID algorithm. When it is detected that the student's concentration is declining, interactive courseware is pushed or the teaching scene is switched.

[0038] 3.1 Anomaly Detection and Graded Response Multi-threshold Trigger Mechanism: Level 1 Warning (Low Risk): When environmental parameters deviate from the comfort zone (illuminance <300 Lux) but teaching behavior is normal, only logs are recorded. Level 2 Intervention (Medium Risk): If the local CO2 concentration is detected to be >1200 ppm and the student yawning frequency increases, the fresh air system is automatically activated and a notification is sent to the teacher's tablet. Level 3 Emergency (High Risk): If fire smoke or violent physical conflict is detected, an emergency broadcast is directly triggered and the campus security system is activated.

[0039] 3.2 Adaptive Environmental Control Multi-Objective Optimization Strategy: Priority Rule: Control of teaching equipment (projectors) takes precedence over environmental equipment (air conditioning) to prevent shaking of the projection screen due to air conditioning speed adjustments. Conflict Resolution: When simultaneously lowering the temperature (due to increased student activity) and increasing illumination (due to darkening on cloudy days), a sequential decision-making process of "adjusting lighting first, then temperature" is adopted to prevent students from experiencing discomfort due to frequent environmental changes. Implicit Adjustment Technology: Lighting adjustment utilizes a gradual change algorithm (<50 Lux per minute), while temperature adjustment wind speed is limited to <3 m / s to ensure that adjustments are not perceived by the human body.

[0040] 3.3 Teaching Assistance Feedback: Teacher-side AR Interface: Visual Presentation: Lightweight AR glasses overlay information layers, displaying real-time data (class average concentration 72%, good air quality index) in the lower right corner of the teacher's field of view. Interactive Design: Supports iris gaze triggering for detailed analysis (gaze at the "Interaction Balance" indicator for 3 seconds to display a distribution chart of speaking time for each group). Invisible Feedback: By adjusting the LED light color temperature (from 6500K cool white to 4000K warm white), student focus is subtly improved, avoiding the psychological pressure caused by direct reminders.

[0041] Its beneficial effect lies in the establishment of a three-dimensional monitoring system through a multi-source, heterogeneous data collection mechanism. This solution integrates multimodal data, including environmental images, sound, temperature, and humidity, overcoming the limitations of single-sensor monitoring and comprehensively capturing classroom dynamics. It not only captures students' attention through image recognition, but also identifies unusual sounds through sound analysis. Combined with temperature and humidity data, it provides real-time assessment of environmental comfort, providing comprehensive data support for classroom monitoring. Secondly, an improved Kalman filter algorithm and spatiotemporal alignment technology ensure data quality. To address sensor noise, the signal-to-noise ratio is improved by optimizing the filter model. Spatiotemporal alignment eliminates temporal and spatial deviations in multi-source data, ensuring the integration of image, sound, and environmental parameters in a unified spatiotemporal coordinate system, laying a solid foundation for subsequent intelligent analysis. The CRNN model and adaptive control form a closed-loop management system. Leveraging the spatiotemporal feature extraction capabilities of convolutional recurrent neural networks, the solution accurately identifies classroom environmental conditions and triggers risk warnings and environmental control in real time. This mechanism not only proactively prevents unexpected classroom risks, but also automatically adjusts intelligent air conditioning and lighting systems to create a safe and comfortable teaching environment, improving classroom efficiency and protecting the health of teachers and students.

[0042] See also Figure 2 In a method for intelligent supervision of a classroom environment based on data analysis, obtaining multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data, comprises the following steps: Step 201: Acquire environmental image data in a classroom environment through an image sensor, wherein the environmental image data at least includes a color image, depth information, and a backlight scene; Step 202: Using a channel ring microphone array, the channel ring microphone array captures individual student voice data, teacher lecture sounds, student discussion sounds, ambient noise, and writing friction sounds in a classroom environment to obtain ambient sound data. Step 203: Obtain temperature data and humidity data in the classroom environment through the temperature and humidity sensor to obtain ambient temperature and humidity data.

[0043] The above is an introduction to the embodiment of the classroom environment intelligent supervision method based on data analysis of the present invention. Figure 3 ,In the classroom environment intelligent supervision system based on data analysis, the classroom environment intelligent supervision system includes the following modules: An environmental data acquisition module is used to acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; a data alignment processing module, configured to eliminate sensor noise of the multi-source heterogeneous data by using an improved Kalman filtering algorithm, and perform spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; An environment state recognition module is used to establish a CRNN three-layer convolutional recurrent neural network model, input the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtain the classroom teaching environment state; The classroom environment supervision module is used to provide classroom risk warning and adaptive classroom environment regulation according to the status of the classroom teaching environment.

[0044] See also Figure 4 , which is a schematic diagram of the CRNN three-layer recursive convolutional neural network model in the intelligent supervision method of the classroom environment based on data analysis.

[0045] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. The intelligent supervision method of classroom environment based on data analysis is characterized by: The method for intelligently supervising a classroom environment comprises the following steps: Acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; Eliminating sensor noise from the multi-source heterogeneous data using an improved Kalman filtering algorithm, and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; Establishing a CRNN three-layer convolutional recurrent neural network model, inputting the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtaining the classroom teaching environment state; Classroom risk warning and adaptive classroom environment regulation are carried out according to the classroom teaching environment status.

2. The method for intelligent classroom environment supervision based on data analysis according to claim 1, characterized in that: The acquiring of multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data, includes: Acquiring environmental image data in a classroom environment through an image sensor, wherein the environmental image data includes at least a color image, depth information, and a backlight scene; A channel ring microphone array is used to acquire individual student voice data, teacher lecture sounds, student discussion sounds, ambient noise, and writing friction sounds in the classroom environment to obtain environmental sound data. The temperature and humidity data in the classroom environment are obtained through the temperature and humidity sensors to obtain the ambient temperature and humidity data.

3. The method for intelligent classroom environment supervision based on data analysis according to claim 1, characterized in that: The method of using an improved Kalman filtering algorithm to eliminate sensor noise of the multi-source heterogeneous data and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data includes: Using an improved Kalman filtering algorithm to eliminate sensor noise from the multi-source heterogeneous data, establishing an adaptive noise covariance matrix for different sensor characteristics, and dynamically adjusting filtering parameters; The timestamps of image, sound, temperature and humidity data are aligned, cross-modal data association is achieved by extending the state vector, and audio energy fluctuations are synchronously analyzed with student movements in the video to obtain aligned multi-source heterogeneous data.

4. The method for intelligently supervising a classroom environment based on data analysis according to claim 1, wherein: The method of using an improved Kalman filtering algorithm to eliminate sensor noise of the multi-source heterogeneous data and performing spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data further includes: Based on the PTP precise time protocol, devices in the noise-reduced multi-source heterogeneous data are always synchronized, and network delay differences are eliminated through hardware timestamps to obtain time-aligned data; The physical positions of the sensor nodes in the time-aligned data are mapped to the camera coordinate system, and the correspondence between the desks and the image pixels is established through the QR code calibration plate to obtain the environmental mapping data. The tracking path of the PTZ camera is dynamically corrected using the sound source localization results to obtain the visual association data; The combined actions of standing discussion and dozing off at the desk are identified through skeleton key point tracking, and the duration is determined by combining time series analysis; a comprehensive scoring model is constructed by analyzing facial orientation, gaze focus, and micro-expressions.

5. The method for intelligent classroom environment supervision based on data analysis according to claim 1, characterized in that: The CRNN three-layer convolutional recursive neural network model is established, including: The first layer of the CRNN three-layer convolutional recurrent neural network model is used to extract image edge features, the second layer is used to identify dynamic behaviors, and the third layer is used to fuse time series data; A bidirectional LSTM network is introduced into the recurrent layer of the model to process the temporal dependencies of sound signals and capture the changing patterns of environmental states. The CRNN three-layer convolutional recurrent neural network model is trained using the historical classroom environment dataset in the database. The importance of different modal data is weighted through the attention mechanism to obtain a trained CRNN three-layer convolutional recurrent neural network model.

6. The method for intelligently supervising a classroom environment based on data analysis according to claim 1, wherein: Inputting the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recursive neural network model for recognition to obtain the classroom teaching environment state includes: The classroom teaching environment status includes at least the teaching behavior recognition status and the environment anomaly detection status; The quality of teacher-student interaction is judged based on the activity of group discussions, and the student concentration is analyzed through facial orientation and eye focus to obtain the teaching behavior recognition status; Identify environmental anomaly detection status through equipment failures, safety hazards and emergencies in the classroom environment.

7. The method for intelligently supervising a classroom environment based on data analysis according to claim 1, wherein: The classroom risk warning and adaptive classroom environment regulation according to the classroom teaching environment state include: Trigger audible and visual alarms for fires and sudden loud noise events, and push information to the administrator terminal; Based on the duration of temperature and humidity deviation from the comfort zone and the proportion of student distraction indicators, a yellow / orange / red warning is generated to prompt teachers to adjust their teaching strategies; The air conditioning and lighting systems are controlled through the PID algorithm. When it is detected that the student's concentration is declining, interactive courseware is pushed or the teaching scene is switched.

8. The intelligent classroom environment supervision system based on data analysis is characterized by: The classroom environment intelligent supervision system includes the following modules: An environmental data acquisition module is used to acquire multi-source heterogeneous data in a classroom environment, wherein the multi-source heterogeneous data includes at least environmental image data, environmental sound data, and environmental temperature and humidity data; a data alignment processing module, configured to eliminate sensor noise of the multi-source heterogeneous data by using an improved Kalman filtering algorithm, and perform spatiotemporal alignment on the noise-eliminated data to obtain aligned multi-source heterogeneous data; An environment state recognition module is used to establish a CRNN three-layer convolutional recurrent neural network model, input the aligned multi-source heterogeneous data into the CRNN three-layer convolutional recurrent neural network model for recognition, and obtain the classroom teaching environment state; The classroom environment supervision module is used to provide classroom risk warning and adaptive classroom environment regulation according to the status of the classroom teaching environment.

9. The classroom environment intelligent supervision system based on data analysis according to claim 8, characterized in that: The environmental data acquisition module includes the following submodules: An image data acquisition submodule, configured to acquire environmental image data in a classroom environment through an image sensor, wherein the environmental image data includes at least a color image, depth information, and a backlight scene; The sound data acquisition submodule is used to use the channel ring microphone array to acquire individual student voice data, teacher lecture sound, student discussion sound, environmental noise and writing friction sound in the classroom environment to obtain environmental sound data; The temperature and humidity data acquisition submodule is used to obtain temperature data and humidity data in the classroom environment through temperature and humidity sensors to obtain ambient temperature and humidity data.

10. The classroom environment intelligent supervision system based on data analysis according to claim 8, characterized in that: The data alignment processing module includes the following submodules: The time alignment submodule is used to synchronize devices in noise-reduced multi-source heterogeneous data based on the PTP precise time protocol, eliminate network delay differences through hardware timestamps, and obtain time-aligned data; A spatial alignment submodule is used to map the physical positions of the sensor nodes in the time-aligned data to the camera coordinate system, establish the correspondence between the desks and the image pixels through the QR code calibration plate, obtain environmental mapping data, and dynamically correct the tracking path of the PTZ camera using the sound source localization results to obtain visual association data; The data processing and analysis submodule is used to identify the combined actions of standing discussion and dozing off on the table through skeleton key point tracking, determine the duration through time series analysis, and analyze facial orientation, gaze focus, and micro-expressions to construct a comprehensive scoring model.

Images