Classroom student behavior detection and analysis system based on deep learning

Abstract

The application relates to the field of behavior detection, and discloses a classroom student behavior detection and analysis system based on deep learning, which comprises a data set construction, a target detection model, a target tracking algorithm, a behavior detection and tracking, and a data storage and analysis module; in the application, a classroom student behavior data set is constructed, a YOLOv8s model is improved, and a ByteTrack target tracking algorithm is improved, so that accurate detection and tracking of classroom student behavior can be realized in combination; thus, video data can be processed in real time, student behavior can be automatically captured and classified and analyzed, a data report and a visual chart can be generated, support can be provided for education decision-making, meanwhile, the application has the advantages of high accuracy, real-time efficiency, high automation degree and data-driven decision-making.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of behavior detection, in particular to a classroom student behavior detection and analysis system based on deep learning. BACKGROUND

[0002] The existing computer vision technology collects classroom data through a classroom camera, cleans and pretreats, trains a YOLO model, deploys the model to realize student behavior detection and outputs the category. However, it still has the following defects:

[0003] 1. Computer vision and deep learning model: In a complex classroom environment, the traditional target detection model (such as YOLOv5) has the problem of insufficient target detection accuracy. In low light, occlusion, and multiple overlapping situations, small targets (such as looking down at the mobile phone, turning around, and lying on the table) are prone to be missed, and the dynamic response to student posture changes (such as standing and turning around) is not accurate, which is prone to false detection or missed detection. The YOLO series algorithm has poor detection ability for small targets, and is prone to detection omissions when shooting at a distance in a complex classroom scene. The model is not good at handling occluded and overlapping objects, and is prone to false identification or tracking when multiple targets are close or occluded.

[0004] 2. Target tracking algorithm: The existing target tracking algorithm is not good at handling target occlusion, such as ByteTrack, which is prone to tracking loss when the target is partially occluded for a long time. When the target appearance, posture, or motion pattern changes greatly, it is prone to tracking mismatch and drift, and false matching of targets. When processing target motion information, it mainly relies on appearance features, ignoring dynamic information such as motion trajectory and speed, which is not accurate in fast motion scenes and difficult to maintain accuracy when targets interact complexly. SUMMARY

[0005] The present application aims to provide a classroom student behavior detection and analysis system based on deep learning, which solves at least one of the above technical problems.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] The classroom student behavior detection and analysis system based on deep learning comprises:

[0008] A data set construction module is used to obtain real classroom videos, frame the videos to obtain picture data, and use the MakeSense tool for labeling to obtain a YOLO format data set;

[0009] A target detection model module uses YOLOv8s as a baseline model, adds a multi-head self-attention mechanism MHSA in the Backbone part, replaces the loss function CIoU with Focaler-DIoU, and adds a small target detection head in the Head part to obtain a classroom student behavior detection model after training.

[0010] The target tracking algorithm module adopts the ByteTrack algorithm, introduces adaptive noise adjustment for Kalman filtering, introduces an improved CIoU and speed feature when calculating the similarity measure, and calculates the weighted combination;

[0011] The behavior detection and tracking module uses an improved YOLOv8 model as a target detector to detect the t-frame video frames to be detected, divides them into high and low score frames according to the score threshold, and then performs target matching and trajectory prediction through the ByteTrack algorithm.

[0012] The data storage and analysis module integrates the detected data into a data packet, including student ID, behavior category, coordinate position, confidence and timestamp, stores it in the database, and uses Python to perform student behavior quantity statistics and individual student behavior timing statistics.

[0013] As a further technical solution, the adaptive noise adjustment includes adaptive adjustment of measurement noise and adaptive adjustment of process noise.

[0014] In each filtering step, first calculate the innovation vector v k , the innovation vector v k is the difference between the current prediction value of the Kalman filter and the actual measurement value, representing the error of the model; the calculation formula is as follows:

[0015] z k is the actual measurement value at time k, is the predicted value at time k, based on the state of the model at the last time k-1 and the prediction result of the current time state by the prediction model, and H is the observation matrix.

[0016] According to the norm of the innovation vector v k , the measurement noise covariance matrix R is dynamically adjusted:

[0017] R k = R k-1 +(1+||v k ||)·I4;

[0018] Where I4 is an identity matrix, ||v k || is the norm of the innovation vector, R k is the measurement noise covariance matrix at time k, and R k-1 is the measurement noise covariance matrix at time k-1.

[0019] According to the target motion state, adjust the process noise covariance matrix Q:

[0020]

[0021] β is the process noise adjustment factor, Q k The process noise covariance matrix at time k, Q k-1 Let k be the process noise covariance matrix at time k-1. For innovation vector v k The transpose of .

[0022] As a further technical solution, in the object detection model module, the Focaler-DIoU Loss loss function combines the dynamic weighting mechanism of Focal Loss with the bounding box localization optimization capability of DIoU Loss, and the calculation formula is as follows:

[0023] Focaler enhances its focus on difficult-to-detect samples by dynamically assigning weights to each sample. The formula for calculating the weight coefficient ω is: ω=(1-IoU) γ γ is an adjustment factor, γ is greater than zero, and IoU is the intersection-union ratio of the predicted box and the ground truth box. gt () represents the center point b of the predicted bounding box and the center point b of the ground truth bounding box. gt The Euclidean distance between the predicted and ground truth boxes is c, where c is the diagonal length of the smallest bounding box of the predicted and ground truth boxes.

[0024] As a further technical solution, the similarity calculation formula in the target tracking algorithm module is: T s =γ·CIoU+δ·S s (v1,v2); γ and δ are weighting coefficients, S s (v1,v2) is the similarity vector of velocity features;

[0025] in, CIoU is a similarity metric that takes into account the center distance of the bounding boxes, aspect ratio, and overlapping regions; b and b gt These are the coordinates of the center points of the predicted and detected bounding boxes, respectively; v is the aspect ratio of the bounding box; and α is the balance parameter.

[0026] As a further technical solution Among them, w gt w gt These are the width and height of the ground truth bounding box, respectively, and w and h are the width and height of the predicted bounding box, respectively.

[0027] As a further technical solution, the speed characteristic S s The calculation is performed by measuring the changes in the target's position information across different frames, using the following formula:

[0028]

[0029] In the formula, v1 and v2 are the speed vectors of the historical trajectory target and the current detection frame respectively; and sigma is a normalization parameter for adjusting the weight of the speed feature.

[0030] As a further technical solution, in the target detection model module, an MHSA is added between the ninth layer C2f module and the tenth layer SPPF of the Backbone part, and after the feature map with a specification of (20, 20, 512) is reshaped, it is converted into a query matrix, a key matrix and a value matrix through three learnable projection matrices, the self-attention weight is calculated through the scaled dot-product attention, and after the outputs of multiple attention heads are spliced, they are mapped back to the original dimension through linear transformation; the multi-head attention mechanism output formula is as follows:

[0031] MultiHead(Q,K,V)=Concat(h1,h2……h h )·W;

[0032] Q, K and V are respectively a query matrix, a key matrix and a value matrix matrix set shared by multiple heads in the multi-head attention mechanism, h1, h2……h h are respectively the attention outputs calculated by each head in the multi-head attention mechanism, Concat represents the splicing operation, and W is a weight matrix for linear transformation of the outputs of multiple heads and mapping back to the original dimension.

[0033] As a further technical solution, the calculation formula for calculating the self-attention weight through the scaled dot-product attention is as follows:

[0034]

[0035] In the formula, Q i , K i and V i represent respectively a query matrix, a key matrix and a value matrix, and d k is the dimension of the query or the key.

[0036] The beneficial effects of the present application are as follows:

[0037] The improved YOLOv8s model is combined with the optimized ByteTrack target tracking algorithm, which greatly enhances the detection and tracking ability of the classroom student behavior; in a complex classroom environment, whether it is light change, personnel occlusion or small movements of students, it can accurately identify and continuously track; compared with the traditional technology, the judgment of various behaviors is more accurate, effectively reducing the false detection and missing detection, providing a reliable data basis for subsequent analysis, and ensuring that the behavior performance of each student in the classroom can be accurately grasped. BRIEF DESCRIPTION OF DRAWINGS

[0038] The application will be further described below with reference to the drawings.

[0039] Figure 1 The standard table for student behavior labeling.

[0040] Figure 2 The category label quantity table.

[0041] Figure 3 The improved YOLOv8 model structure diagram.

[0042] Figure 4 The multi-head self-attention mechanism (MHSA) structure diagram.

[0043] Figure 5 The ablation experiment result comparison table.

[0044] Figure 6 The improved ByteTrack algorithm flowchart.

[0045] Figure 7 The CIoU structure diagram.

[0046] Figure 8 The tracking result comparison table before and after improvement.

[0047] Figure 9 The student behavior positivity score table. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the application.

[0049] Please refer to Figures 1-9 The application is a classroom student behavior detection and analysis system based on deep learning, which comprises:

[0050] A dataset construction module is configured to obtain real classroom videos, extract picture data from the video, and label the data using a MakeSense tool to obtain a YOLO format dataset; in the dataset construction module, the data collection obtains real classroom videos of different schools and classroom layouts from multiple domestic schools through authorization, and uses OpenCV to process the video file to extract frames and save them as pictures; the data sorting filters out pictures with diverse scenes, diverse behaviors, and good quality from the front angle, with a resolution of 1920x1080; the data labeling uses the MakeSense labeling tool to label and save each bounding box in YOLO format for six behaviors, including using a mobile phone, sleeping on the desk, staring at the blackboard / teacher, reading and writing, standing, and turning around.

[0051] A target detection model module uses YOLOv8s as a baseline model, adds a multi-head self-attention mechanism MHSA in the Backbone part to improve feature extraction capability, replaces the loss function CIoU with Focaler-DIoU, and uses the Focaler-DIoU loss function to solve the problem of class imbalance and bounding box prediction; a small target detection head is added in the Head part, and a small target detection layer is added to improve the detection capability of long-distance and small-size targets; a classroom student behavior detection model is obtained after training; in the target detection model module, the added small target detection layer P2 has a resolution of 1 / 4 of the input image size, i.e., 160x160, which locates and extracts high-resolution feature maps in the backbone network, and performs target detection after multi-scale feature fusion with other layer feature maps through upsampling and Concat splicing.

[0052] A target tracking algorithm module uses the ByteTrack algorithm, introduces adaptive noise adjustment to Kalman filtering, improves the Kalman filtering in the tracking algorithm, adjusts the observation noise R and process noise Q according to the innovation, and dynamically optimizes the filter according to the environment and state changes, which can avoid excessive dependence on prediction and observation, and improve the overall precision; when calculating the similarity measure, the improved CIoU and speed features are introduced and weighted and combined to calculate, which can obtain more accurate matching results.

[0053] A behavior detection and tracking module uses the improved YOLOv8 model as a target detector to detect the t-frame video frames to be detected, divides them into high and low score boxes according to the score threshold, and then performs target matching and trajectory prediction through the ByteTrack algorithm.

[0054] The data storage and analysis module integrates the detected data into a data packet, including student ID, behavior category, coordinate position, confidence and timestamp, and stores it in the database. The student behavior quantity statistics and student individual behavior time sequence statistics are performed using Python (Pandas, Matplotlib). In the data storage and analysis module, the system can load video in real time through a high-definition camera or select video from the local, and the detection result is stored in the database in a structured form. When performing category quantity statistics, real-time single-row behavior statistical column chart and cumulative behavior statistical column chart are realized, and a positivity score is given to the student behavior. When analyzing the student individually, a time sequence graph is drawn in the form of a point-line graph.

[0055] In the target detection model module, MHSA is added between the ninth layer C2f module and the tenth layer SPPF of the Backbone part. After reshaping the feature map with a specification of (20, 20, 512), it is converted into a query matrix, a key matrix and a value matrix through three learnable projection matrices. The self-attention weight is calculated through the scaled dot-product attention, and the outputs of multiple attention heads are spliced and mapped back to the original dimension through linear transformation.

[0056] MultiHead(Q,K,V)=Concat(h1,h2……h h )·W;

[0057] Q, K and V are respectively the query, key and value matrix sets shared by multiple heads in the multi-head attention mechanism, h1, h2……h h are respectively the attention outputs calculated by each head in the multi-head attention mechanism, Concat represents the splicing operation, and W is the weight matrix used for linear transformation of the outputs of multiple heads to map back to the original dimension.

[0058] The calculation formula of the self-attention weight calculated by the scaled dot-product attention is as follows:

[0059]

[0060] Where Q i , K i and V i represent the query matrix, the key matrix and the value matrix respectively, and d k is the dimension of the query or the key.

[0061] In this embodiment, the detection layers in the YOLOv8 base model are an 8-fold down-sampled P3 layer, a 16-fold down-sampled P4 layer, and a 32-fold down-sampled P5 layer; the input image resolution of the training picture after YOLOv8 preprocessing is 640x640, the corresponding feature map resolutions of the three layers are 80x80, 40x40, and 20x20, respectively, and the small target detection layer P2 added has a feature map resolution of the size of the input image, that is, 160x160; first, high-resolution feature maps are located and extracted in the backbone network, then multi-scale feature fusion is performed with other layer feature maps through up-sampling and Concat splicing, target detection is performed on the multi-scale feature maps, and multi-scale results are combined and output; the performance comparison of each experiment before and after improvement is shown in the ablation experiment result table in the description; it can be found from the data that each index of the improved model is significantly improved, the precision and recall rate are balanced, and the robustness and accuracy of the model in the actual scene are improved;

[0062] The adaptive noise adjustment includes adaptive adjustment of measurement noise and adaptive adjustment of process noise.

[0063] In each filtering step, first, the innovation vector v k is calculated k , which is the difference between the current prediction value of the Kalman filter and the actual measurement value, representing the error of the model; the calculation formula is as follows:

[0064] z k is the actual measurement value at time k, is the prediction value at time k, which is based on the state of the model at the previous time k-1 and the prediction result of the prediction model for the current time state, and H is an observation matrix.

[0065] The measurement noise covariance matrix R is dynamically adjusted according to the norm of the innovation vector v k :

[0066] R k = R k-1 +(1+‖v k ‖)·I4;

[0067] where I4 is an identity matrix, ||v k || is the norm of the innovation vector, R k is the measurement noise covariance matrix at time k, and R k-1 is the measurement noise covariance matrix at time k-1.

[0068] The process noise covariance matrix Q is adjusted according to the target motion state:

[0069]

[0070] β is the process noise adjustment factor, Q k process noise covariance matrix at time k, Q k-1 is the process noise covariance matrix at time k-1, is the innovation vector v k .

[0071] In this embodiment, the target of adaptive noise adjustment is to dynamically adjust the noise covariance matrix according to the innovation, so that the Kalman filter can achieve better performance in different environments. It mainly consists of two steps, as follows:

[0072] 1) Adaptive adjustment of measurement noise R

[0073] In each filtering step, first calculate the innovation vector v k , which is the difference between the current prediction value of the Kalman filter and the actual measurement value, representing the error of the model. Its calculation formula is as follows:

[0074] Then calculate the norm of the innovation vector ||v k || and dynamically adjust the measurement noise covariance matrix R according to the value. The measurement noise adjustment coefficient is:

[0075] R k = R k-1 +(1+||v k ||)·I4

[0076] Where I4 is a unit matrix used to increase the diagonal elements of the measurement noise. According to the above formula, it can be seen that the innovation norm is the core of adjusting the noise size. The larger the innovation vector, the larger the model error, and the larger the measurement noise, so R needs to be increased.

[0077] 2) Adaptive adjustment of process noise Q

[0078] Process noise Q is generally related to the motion state of the target. The more uncertain the motion state, the process noise should be appropriately increased. Based on this, it is proposed to use the size of the innovation to adjust the process noise in the Kalman filtering process. If the norm of the innovation is small, it means that the model prediction is accurate, and the process noise Q can be appropriately reduced, thereby improving the tracking accuracy of the system. The process noise adjustment formula is:

[0079]

[0080] In the formula, β is the process noise adjustment factor, which controls the speed of noise update.

[0081] Combining the following Kalman filter overall formula, by dynamically adjusting Q, the adaptive strategy optimizes the credibility of state prediction; when the system dynamics is strong, increase Q to improve the uncertainty of prediction to capture changes; in a stable environment, reduce Q to improve the accuracy of prediction. And the Kalman gain K k in the correction stage is directly affected by R, after gain adjustment, the filter can suppress the influence of noise when the innovation is large, and fully utilize the measurement value information when the innovation is small, so as to dynamically optimize the accuracy and stability of state estimation.

[0082] The overall formula of Kalman filter is as follows, which is used for state prediction and update in the improved ByteTrack target tracking algorithm to optimize the target tracking effect;

[0083]

[0084] State prediction equation: The role is: according to the state estimation value x k-1|k-1 of the last time (k-1) and the control input u k of the current time, predict the state of the current time (k). A is the state transition matrix, which describes the change rule of state with time; B is the control matrix, which determines the influence of control input on state; for example: in the classroom student behavior detection scene, if the position and speed of students are taken as state variables, the position and speed of students at the last time are known, combined with the possible movement control at the current time (such as the teacher asking students to stand up and answer questions, which will affect the position state of students), the state of students at the current time can be predicted.

[0085] Covariance prediction equation: The role is: predict the covariance of state estimation at the current time, which combines the covariance P k-1|k-1 of state estimation at the last time (k-1), state transition matrix A and process noise covariance Q. Covariance reflects the uncertainty of state estimation, and Q represents the process noise, which embodies the uncertainty existing in the system itself. For example, in tracking student behavior, the student's movement may not be completely regular, and this uncertainty is represented by Q. This formula considers the uncertainty at the last time and the uncertainty of the current process to predict the uncertainty of state estimation at the current time.

[0086] Kalman gain calculation equation: The role is: calculate the Kalman gain K k , which determines how to combine the predicted value and the measurement value to update the state estimation. K k is affected by the predicted covariance The influence of the observation matrix H and the measurement noise covariance R. In practical applications, measurements (such as student location information captured by a camera) contain noise, and predictions also have uncertainty. Kalman gain K k The key is to balance these two factors by determining the weights to be given to the predicted and measured values ​​based on the magnitudes of the prediction covariance and the measurement noise covariance, in order to obtain a more accurate state estimate.

[0087] State update equation: Its function is to utilize the Kalman gain K k Combined with predicted values and measured value z k residual To update the current state estimate x k|k For example, when tracking student behavior, there may be a difference between the predicted student location and the actual measured student location; this difference is the residual. By weighting the residual using Kalman gain and adding it to the predicted value, we can obtain a more realistic estimate of the student's state.

[0088] Covariance update equation: Its function is to: based on the Kalman gain K k and predicted covariance Update the covariance P of the current state estimate k|k This is used for prediction and update calculations at the next time step. Example: After each update of the state estimate, the covariance needs to be updated accordingly to reflect the uncertainty of the new state estimate. I is the identity matrix. This formula adjusts the covariance size by considering the Kalman gain and the prediction covariance, providing a more accurate measure of uncertainty for state prediction and updates at the next time step.

[0089] In the target detection model module, the Focaler-DIoU Loss loss function combines the dynamic weighting mechanism of FocalLoss with the bounding box localization optimization capability of DIoU Loss. The calculation formula is as follows:

[0090] Focaler-DIoU combines the dynamic weighting mechanism of Focal Loss with the bounding box localization optimization capability of DIoU Loss to form a more effective loss function for object detection tasks. The formula represents adding a penalty term for the center point distance to the IoU, which solves the problem of insensitivity to center point offset when relying solely on IoU to evaluate bounding box localization.

[0091] Focaler enhances its focus on difficult-to-detect samples by dynamically assigning weights to each sample. The formula for calculating the weight coefficient ω is: ω=(1-IoU) γγ is an adjustment factor used to control the dynamics of weight allocation; γ is greater than zero and is usually set to 2. IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. (b,b) gt () represents the center point b of the predicted bounding box and the center point b of the ground truth bounding box. gt The Euclidean distance between the predicted and ground truth boxes is c, where c is the diagonal length of the smallest bounding box of the predicted and ground truth boxes.

[0092] In the target tracking algorithm module, the similarity calculation formula is: T s =γ·CIoU+δ·S s (v1,v2); γ and δ are weighting coefficients, S s (v1,v2) is the similarity vector of velocity features;

[0093] in, CIoU is a similarity metric that takes into account the center distance of the bounding boxes, aspect ratio, and overlapping regions; b and b gt These are the coordinates of the center points of the predicted and detected bounding boxes, respectively; v is the aspect ratio of the bounding box; and α is the balance parameter. Among them, w gt w gt These are the width and height of the ground truth bounding box, respectively, and w and h are the width and height of the predicted bounding box, respectively. Velocity characteristic S s The calculation is performed by measuring the changes in the target's position information across different frames, using the following formula:

[0094]

[0095] In the formula, v1 and v2 are the velocity vectors of the historical trajectory target and the current detection box, respectively; σ is a normalization parameter used to adjust the weight of the velocity feature.

[0096] In this embodiment, ByteTrack uses target location information as association clues and adopts IoU as a method to calculate the similarity between the detection box and the prediction box. However, it does not fully consider factors such as the shape of the target, aspect ratio, and center point distance. CIoU considers the center distance, aspect ratio, and overlapping area of ​​the target box, thereby further optimizing the target matching based on IoU. CIoU can improve the calculation of IoU through three aspects: center point distance, aspect ratio, and overlap.

[0097] The MOTA and MOTP values of the improved tracking algorithm are improved by 2.8% and 2.4% compared with those before improvement, which indicates that the improved algorithm has higher accuracy in trajectory prediction and bounding box assignment, and the mis-matching condition is reduced; the increase of IDP and IDR indexes reflects that the algorithm better maintains the consistency of target identity, and the tracking trajectory quality is higher. The decrease of evaluation index IDSW indicates that the target identity switching problem is reduced, and the tracking result is more coherent, and in the scene where students frequently interact, are shielded or have large action amplitude, the target can be more stably identified and tracked.

[0098] It should be noted that the calculation formula and each parameter participating in operation in the application are all pre-processed by dimensionless processing, and the process of dimensionless processing is known in the industry, which is not described here.

[0099] The above has carried out the detailed explanation to one embodiment of the application, but the content described is only the preferred embodiment of the application, cannot be considered for limiting the implementation scope of the application. All equivalent changes and improvements made according to the scope of the application should still belong to the patent coverage range of the application.

Claims

1. A deep learning based classroom student behavior detection and analysis system, characterized in that, The method comprises the following steps: A dataset construction module is used to obtain real classroom videos, frame pictures from the video, and label the pictures using the MakeSense tool to obtain a YOLO format dataset; A target detection model module is used to take YOLOv8s as a baseline model, add a multi-head self-attention mechanism MHSA in the Backbone part, replace the loss function CIoU with Focaler-DIoU, and add a small target detection head in the Head part to obtain a classroom student behavior detection model after training; A target tracking algorithm module is used to adopt the ByteTrack algorithm, introduce adaptive noise adjustment to Kalman filtering, and introduce an improved CIoU and speed feature and weighted combination calculation when calculating the similarity measure; A behavior detection and tracking module is used to take the improved YOLOv8 model as a target detector, detect the t-frame video frames to be detected, divide them into high and low score boxes according to the score threshold, and then perform target matching and trajectory prediction through the ByteTrack algorithm; A data storage and analysis module is used to integrate the detected data into a data packet, including student ID, behavior category, coordinate position, confidence and timestamp, store it in the database, and use Python to perform student behavior quantity statistics and individual student behavior time sequence statistics; In the target detection model module, the Focaler-DIoU Loss loss function combines the dynamic weight mechanism of the Focal Loss and the frame positioning optimization capability of the DIoU Loss, and the calculation formula is: ; Wherein, Focaler enhances the attention to difficult-to-detect samples by dynamically allocating weights for each sample, and the weight coefficient The calculation formula is: , is an adjustment factor, greater than zero, is the intersection over union of the prediction box and the real box, is the Euclidean distance between the center point of the prediction box and the center point of the real box, is the diagonal length of the minimum bounding box of the prediction box and the real box. The similarity calculation formula in the target tracking algorithm module is: ; and is a weight coefficient, is a similarity vector of the speed feature; wherein, , is a similarity measure that takes into account the center distance, aspect ratio and overlap area of the target box; and are the predicted and detected box center point coordinates, respectively, is a measure of the width-height ratio of the box, is a balancing parameter. 2.The deep learning based classroom student behavior detection and analysis system according to claim 1, wherein, The adaptive noise adjustment includes adaptive adjustment of measurement noise and adaptive adjustment of process noise; At each filtering step, the innovation vector is first computed , the innovation vector is the difference between the current prediction of the Kalman filter and the actual measurement, representing the error of the model; the calculation formula is as follows: ; is the actual measurement value at time is the predicted value at time based on the state of the model at the previous time and the prediction of the current time state by the prediction model, is the observation matrix;​ According to innovation vectors The norm of dynamically adjusted measurement noise covariance matrix : ； in, For an identity matrix, || || represents the norm of the innovation vector; For at any time The measurement noise covariance matrix, For at any time The measurement noise covariance matrix; Adjusting process noise covariance matrix according to target motion state : ； a process noise adjustment factor, time instant a process noise covariance matrix, a process noise covariance matrix, a process noise covariance matrix, the transpose of the innovation vector the transpose of the innovation vector 3.The deep learning based classroom student behavior detection and analysis system according to claim 1, wherein, ; wherein, , are the width and height of the true box, respectively, , are the width and height of the predicted box, respectively; . 4.The deep learning based classroom student behavior detection and analysis system of claim 1, wherein, Speed feature The speed feature is calculated by the change of the position information of the target between different frames, and the calculation formula is as follows: ； wherein , are the velocity vectors of the historical trajectory target and the current detection box, respectively; is a normalization parameter for adjusting the weight of the velocity feature. 5.The deep learning based classroom student behavior detection and analysis system according to claim 1, wherein, In the target detection model module, an MHSA is added between the ninth layer C2f module and the tenth layer SPPF in the Backbone part, the feature map with a specification of (20, 20, 512) is reshaped, then converted into a query matrix, a key matrix and a value matrix through three learnable projection matrices, the self-attention weight is calculated through the scaled dot-product attention, and the outputs of multiple attention heads are spliced and then linearly transformed to map back to the original dimension; the output formula of the multi-head attention mechanism is as follows: ； , , are a set of query, key and value matrix matrices shared by multiple heads in multi-head attention mechanism, respectively, are attention outputs computed by each head in multi-head attention mechanism, respectively, denotes a concatenation operation, is a weight matrix used to linearly transform the outputs of multiple heads and map them back to the original dimension. 6.The deep learning based classroom student behavior detection and analysis system according to claim 5, wherein, The calculation formula for calculating the self-attention weight through the scaled dot-product attention is as follows: ； wherein, , , denote a query matrix, a key matrix and a value matrix, respectively, is the dimension of the query or key.

Images