Classroom concentration evaluation method and system based on improved YOLOv9s

Abstract

The invention relates to a classroom concentration evaluation method and system based on improved YOLOv9s, and the method comprises the following steps: collecting a classroom video stream, and obtaining a video frame according to the classroom video stream; detecting student behaviors in the video frame through a behavior detection model to obtain a behavior detection result; extracting a face region in the video frame to obtain a face image, and performing image analysis on the face image through a DeepFace model to obtain a face analysis result; performing dynamic weighted fusion on the behavior detection result and the face analysis result to obtain a multi-modal fusion feature; according to the multi-modal fusion features, calculating through a personalized evaluation parameter model to obtain a concentration score of the student; and smoothing the concentration score to obtain a concentration evaluation result. Compared with the prior art, the stability and accuracy of concentration evaluation and the detection performance and generalization ability of the model in a complex detection task are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and educational informatization, and in particular to a classroom concentration evaluation method and system based on an improved YOLOv9s. Background Art

[0002] With the continuous advancement of educational concepts such as "smart education" and "precision teaching," the comprehensive, objective, and real-time assessment of student focus in the classroom has become a key area of ​​research in educational informatization. Student focus, as a key indicator for teaching feedback and learning outcome assessment, is directly related to the efficiency and quality of classroom instruction. However, in current teaching practices, the assessment of student focus still primarily relies on subjective observation, empirical judgment, or questionnaire-based methods. These methods are not only inefficient but also lack scientificity, systematicity, and practicality.

[0003] In recent years, the application of cutting-edge technologies such as artificial intelligence, computer vision, and deep learning in educational settings has gradually emerged. Some research has begun exploring the use of image processing, gesture recognition, facial expression recognition, and speech recognition to assess student performance and engagement in class. However, existing technologies and systems still have the following significant problems and shortcomings:

[0004] (1) Single-modal perception methods have great limitations: Most existing methods are based only on visual or audio modalities and lack a multimodal fusion mechanism, making them unable to fully perceive the complex state of students. Single modality is easily affected by interference such as occlusion, lighting, shielding, and background complexity, resulting in reduced recognition accuracy and insufficient robustness.

[0005] (2) Insufficient temporal information processing: Classroom concentration is a dynamic process. Evaluating it based solely on the behavior or emotional state at a certain moment cannot reflect the continuity and fluctuation trend of students' concentration throughout the class. It lacks temporal smoothing processing and long-term state tracking mechanisms.

[0006] (3) Lack of personalized assessment mechanism: There are significant individual differences among students, such as personality, behavioral expression, and emotional response. Existing methods mostly use unified assessment standards. For example, patent application CN113326729A discloses a multimodal classroom concentration detection method and device. This method fully records the emotional characteristics of the subjects through multimodal emotion detection, constructs an emotion model, and then identifies changes in concentration based on the emotional characteristics. This method ignores the modeling and adaptive adjustment of students' individual characteristics, resulting in the evaluation results being neither personalized nor practical.

[0007] (4) The presentation of evaluation results is not intuitive: Although some studies have basic analytical capabilities, they lack effective visualization of analysis results and cannot provide teachers with intuitive and actionable teaching feedback data, making it difficult to truly apply them in teaching practice.

[0008] (5) Low system integration and high application threshold: Some intelligent analysis systems currently on the market have problems such as complex deployment, high software and hardware requirements, and inconsistent interfaces. As a result, they are not widely used in actual school teaching scenarios and are difficult to meet the real-time, visibility, and operability needs of ordinary schools.

[0009] In summary, existing classroom concentration assessment methods have obvious technical shortcomings in multimodal perception, time series modeling, personalized adaptation and visual feedback. There is an urgent need for a concentration assessment system that integrates multi-source data, has time series dynamic analysis capabilities, supports individual difference modeling and has good interactivity, so as to improve the scientificity and intelligence of classroom teaching. Summary of the Invention

[0010] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a classroom concentration assessment method and system based on improved YOLOv9s, which significantly improves the stability and accuracy of concentration assessment as well as the detection performance and generalization ability of the model in complex detection tasks.

[0011] The purpose of the present invention can be achieved by the following technical solutions:

[0012] A classroom concentration evaluation method based on improved YOLOv9s includes the following steps:

[0013] Collecting classroom video streams, and obtaining video frames according to the classroom video streams;

[0014] Detecting the student behavior in the video frame using a behavior detection model to obtain a behavior detection result;

[0015] Extracting a facial region from the video frame to obtain a facial image, and performing image analysis on the facial image using a DeepFace model to obtain a facial analysis result;

[0016] Dynamically weighting the behavior detection results and the facial analysis results to obtain a multimodal fusion feature;

[0017] Calculating the student's concentration score through a personalized evaluation parameter model based on the multimodal fusion features;

[0018] The concentration score is smoothed to obtain a concentration evaluation result.

[0019] Furthermore, the video frames simultaneously contain the behavioral actions and facial emotional states of multiple students.

[0020] Furthermore, the behavior detection model is an EM-YOLOv9s model, and the original loss function is replaced by the FocalWIoU loss function. The behavior detection model includes a backbone network and a detection head. The RepNCSPELAN4 module of the backbone network is provided with a GhostConv lightweight convolution structure, and the detection head is provided with an EMA attention mechanism.

[0021] Furthermore, the behavior detection result is a behavior category, and the behavior category includes multiple types of hand raising, reading and writing.

[0022] Furthermore, the image analysis results include student identity and emotion category. The identification of the student identity adopts facial feature vector comparison to achieve identity mapping. The emotion category is obtained by using a Softmax multi-classifier to judge the facial emotional state. The emotion category includes multiple types of neutral, anger, joy, sadness and surprise.

[0023] Furthermore, the behavior detection results and the facial analysis results are dynamically weighted and fused to obtain multimodal fusion features. The specific steps are:

[0024] Extracting behavioral features and confidence levels from the behavioral detection results, emotional features and confidence levels from the facial analysis results, and spatiotemporal context features from the video frames;

[0025] According to the importance and confidence of each modal feature, a weight calculation method based on the attention mechanism is used to dynamically allocate the proportion of each modality in the multimodal fusion feature;

[0026] Construct multimodal fusion features according to the proportion of each modality in the multimodal fusion features.

[0027] Furthermore, the steps for constructing the personalized evaluation parameter model are specifically as follows:

[0028] Based on the student identity identified in the image analysis results, the student's personal feature vector is constructed according to the student's behavior category and emotion category in the history class record;

[0029] The analysis strategy parameters are dynamically adjusted according to the student's personal feature vector, thereby realizing adaptive modeling of individual differences among different students. The analysis strategy parameters include behavior detection threshold, emotion classification weight and concentration calculation weight.

[0030] Furthermore, the concentration score is:

[0031]

[0032] Where F is the behavior category B at the kth frame i The concentration score of the behavior score is α, and the concentration calculation weight of the behavior score is S B Score for behavior, S E For sentiment score, B i is the behavior category, Behavior category B i The weight of Behavior category B at the kth frame i The confidence level, E j is the emotion category, Emotion category E j The weight of is the emotion category E at the kth frame j confidence level.

[0033] Furthermore, a time series smoothing mechanism is adopted to perform time-weighted averaging on the concentration scores of consecutive frames using a sliding time window to obtain a concentration evaluation result. The concentration score after the time-weighted averaging is:

[0034]

[0035] Where, Behavior category B at the t-th frame i The time-weighted average concentration score of is, T is the frame number window used for weighted average, Behavior category B at the kth frame i concentration score.

[0036] According to another aspect of the present invention, a classroom concentration assessment system based on an improved YOLOv9s is provided, comprising:

[0037] A data acquisition module is used to collect classroom video streams and obtain video frames based on the classroom video streams;

[0038] A behavior detection module is used to detect the student behavior in the video frame using a behavior detection model to obtain a behavior detection result;

[0039] A facial analysis module is used to extract the facial area in the video frame to obtain a facial image, and perform image analysis on the facial image using a DeepFace model to obtain a facial analysis result;

[0040] A feature fusion module is used to dynamically weight the behavior detection results and the facial analysis results to obtain multimodal fusion features;

[0041] A concentration score calculation module is used to calculate the student's concentration score through a personalized evaluation parameter model based on the multimodal fusion features;

[0042] The concentration evaluation module is used to smooth the concentration score to obtain a concentration evaluation result.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention obtains video frames by collecting classroom video streams, dynamically weightedly fuses the behavior detection results and facial analysis results to obtain multimodal fusion features, calculates the student's concentration score through a personalized evaluation parameter model based on the multimodal fusion features, and performs smoothing to obtain the concentration evaluation result. By designing the fusion of two types of modal data, namely student behavior and facial emotions, the complementary fusion of multimodal features is achieved, which effectively improves the stability and accuracy of concentration evaluation.

[0045] 2. This paper introduces a number of improvement measures to construct the EM-YOLOv9s behavior detection model, replaces the original loss function with the Focal WIoU loss function, sets a GhostConv lightweight convolution structure in the RepNCSPELAN4 module of the backbone network, and sets the EMA attention mechanism in the detection head. This not only improves the expressive power of feature extraction and fusion, but also optimizes the gradient information flow path and multi-scale detection mechanism, achieving a balance between the model's detection accuracy and computational efficiency for small targets, and improving the model's detection performance in complex detection tasks.

[0046] 3. This invention proposes a dynamic weighting mechanism based on context awareness, targeting the different weights of students' different behavioral and emotional characteristics in classroom scenarios. It assigns different weights to behavioral and emotional categories in different time periods, thereby achieving personalized and dynamic concentration scoring. This strategy can adapt to the changes in students' concentration in different teaching stages and different teaching styles, thereby improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a classroom concentration assessment method based on improved YOLOv9s proposed in the present invention;

[0048] Figure 2 This is a schematic diagram of the network structure of the behavior detection model;

[0049] Figure 3 Comparison of the map curves of the models before and after improvement, where (3a) is the map curve of YOLOv9s and (3b) is the map curve of EM-YOLOv9s;

[0050] Figure 4 A behavior-emotion time series change diagram for personalized analysis. DETAILED DESCRIPTION

[0051] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0052] English abbreviations involved:

[0053] Efficient Multi-scale Attention: Efficient Multi-scale Attention, EMA

[0054] Student Classroom Behavior (SCB)

[0055] Example 1

[0056] This embodiment provides a classroom concentration evaluation method based on improved YOLOv9s, such as Figure 1 As shown, the following steps are included:

[0057] S1. Collect classroom video stream and obtain video frames based on the classroom video stream.

[0058] According to the actual needs of the multimodal fusion student classroom concentration assessment task, the classroom video stream acquisition equipment includes fixed or mobile high-definition cameras. The captured video images have a frame rate of no less than 60 frames per second and support a resolution of no less than 1080p.

[0059] The video frames simultaneously contain the behavioral actions and facial emotional states of multiple students, covering the entire classroom scene range and are adaptable to multi-target detection.

[0060] S2. Detect student behaviors in video frames using a behavior detection model to obtain behavior detection results.

[0061] The behavior detection model is the EM-YOLOv9s model, which replaces the original loss function with the Focal WIoU loss function. The behavior detection model includes a backbone network (Backbone) and a detection head (Head), as shown in Figure 2As shown in the figure, the Backbone module serves as the backbone network for feature extraction, integrating a reparameterization mechanism and an ELAN architecture. This module enhances the depth and breadth of feature extraction through the fusion of multi-path structures, particularly for small object detection in complex backgrounds, achieving stronger semantic expression and discriminative capabilities. The reparameterization mechanism allows the model to maintain high performance during training and merge parameters during inference to reduce computational effort, achieving a balance between performance and efficiency. The Head module undergoes multiple optimizations based on the original YOLOv9. First, the PGI mechanism is introduced. This structure bypasses the traditional feature pyramid network (FPN) and directly feeds gradient information back to different layers of the backbone network, significantly improving gradient transfer efficiency, accelerating network convergence, and reducing information loss. Second, three auxiliary detection heads are added to the Head module. Their parallel branching structure improves the recognition of objects of different scales, further enhancing the model's detection accuracy and stability.

[0062] The RepNCSPELAN4 module of the backbone network is equipped with a GhostConv lightweight convolutional structure, and the detection head is equipped with an EMA attention mechanism.

[0063] The EM-YOLOv9s model integrates the EMA attention mechanism on top of the original YOLOv9s architecture. This mechanism weights feature maps of different scales, enhancing the model's ability to focus on small target behavior areas and maintaining stable feature representation. To focus on key areas, the EMA attention mechanism was added to YOLOv9s using the CBAM attention mechanism, the SEA attention mechanism, the BiFormer attention mechanism, and the EMA attention mechanism. Comparative experiments are shown in Table 1.

[0064] Table 1 Comparative experiment of attention mechanism

[0065] Model P％ R％ mAP 50% mAP50: 90% YOLOv9s 0.689 0.711 0.752 0.585 +CBAM 0.689 0.716 0.749 0.580 +SEA 0.714 0.697 0.754 0.581 +BiFormer 0.721 0.717 0.775 0.616 +EMA 0.705 0.741 0.781 0.622

[0066] According to Table 1, the YOLOv9s model with the EMA attention mechanism performs better in various indicators, especially in the mAP50:90% indicator, where the YOLOv9s model with the EMA attention mechanism performs best.

[0067] By introducing the GhostConv lightweight convolution structure into the RepNCSPELAN4 module in the backbone network, some redundant feature maps are generated and the computational complexity of traditional convolution operations is reduced, thereby reducing the number of model parameters and inference latency, and improving real-time detection performance and resource utilization efficiency.

[0068] The Focal WIOU loss function is introduced to amplify the loss of difficult samples, helping the behavior detection model to focus more on small targets and accelerate convergence.

[0069] The behavioral test results are behavioral categories, which include various types of raising hands, reading, and writing.

[0070] The EM-YOLOv9s model significantly improves the accuracy and robustness of student behavior detection. The experimental results are as follows: Figure 3 As shown, Figure 3 (3a) is the YOLOv9s baseline model, Figure 3 (3b) is the EM-YOLOv9s model. Its mAP50 on the SCB dataset reaches 78.1%, which is 2.9% higher than the original YOLOv9s and reduces the number of parameters by 8.95%.

[0071] The performance of different algorithms for behavior detection is compared on the SCB dataset. The comparison results are shown in Table 2.

[0072] Table 2 Comparison of behavior detection performance of different algorithms on the SCB dataset

[0073] Model Rise hand / % Read / % Write / % mAP50 / % YOLOv5s 82.3 73.4 60.4 72.0 YOLOv9s 84.5 77.3 63.9 75.2 YOLOv10s 84.0 75.0 66.8 75.3 EM-YOLOv9s 87.0 78.5 68.8 78.1

[0074] Experimental results show that the EM-YOLOv9s model performs well across all categories, particularly in detecting "Rise Hand," "Read," and "Write," achieving significantly higher accuracy than other models, ultimately achieving a mAP50% of 78.1%. In comparison, YOLOv5s performs poorly in several categories, particularly in detecting "Write," where it has the lowest accuracy of only 60.4%. Overall, EM-YOLOv9s combines greater robustness and accuracy, demonstrating excellent small object detection capabilities in complex scenes.

[0075] S3. Extract the facial area in the video frame to obtain a facial image, perform image analysis on the facial image using the DeepFace model, and obtain a facial analysis result.

[0076] The image analysis results include student identity and emotion category. The student identity is identified by using facial feature vector comparison to achieve identity mapping. The emotion category is obtained by using a Softmax multi-classifier to judge the facial emotional state. The emotion category includes neutral, angry, happy, sad and surprised.

[0077] The specific steps of image analysis are:

[0078] After completing the student behavior detection, the facial area of ​​each student is accurately extracted based on the detection frame to remove redundant background interference;

[0079] The extracted face images are input into the DeepFace model for student identity recognition and facial expression emotion analysis. The student identity is recognized by using facial feature vector comparison to achieve identity mapping, and the emotion category is obtained by using the Softmax multi-classifier to judge the facial emotional state.

[0080] Bind the identified student ID to its corresponding emotion category as one of the key input information for subsequent multimodal fusion and concentration evaluation

[0081] Because each student's facial features are unique to a certain extent, each student is assigned a separate identity tag (PersonID) and accurately matched based on their facial features. When new facial features appear in an image, the model can automatically determine whether the features match those of a known student. If there is no match, a new PersonID is assigned and recorded. The facial feature matching process uses methods such as cosine similarity to calculate the similarity between students' facial features. When the similarity between facial features and stored facial features exceeds a set threshold, the model will consider the student to have been recognized. If the similarity is lower, the model will determine that the face is new and assign it a new identity.

[0082] S4. Dynamically weight the behavior detection results and the facial analysis results to obtain multimodal fusion features.

[0083] The specific steps for dynamically weighted fusion of behavior detection results and facial analysis results to obtain multimodal fusion features are as follows:

[0084] Extracting behavioral features and their confidence levels from behavioral detection results, emotional features and their confidence levels from facial analysis results, and spatiotemporal context features from video frames;

[0085] According to the importance and confidence of each modal feature, a weight calculation method based on the attention mechanism is used to dynamically allocate the proportion of each modality in the multimodal fusion feature;

[0086] Multimodal fusion features are constructed according to the proportion of each modality in the multimodal fusion features, which serves as input data for the subsequent personalized evaluation module to achieve a comprehensive description of the student's current status.

[0087] S5. The student’s concentration score is calculated based on the multimodal fusion features through a personalized evaluation parameter model.

[0088] The specific steps for building a personalized evaluation parameter model are as follows:

[0089] Based on the student identity identified in the image analysis results, the student's personal feature vector is constructed according to the student's behavior category and emotion category in the history class record;

[0090] The analysis strategy parameters are dynamically adjusted according to the students' personal feature vectors to achieve adaptive modeling of individual differences among different students. The analysis strategy parameters include behavior detection threshold, emotion classification weight and concentration calculation weight.

[0091] A personalized fine-tuning mechanism can be used to perform lightweight local parameter updates on the personalized assessment model, improving sensitivity to individual behavioral and emotional changes and analytical accuracy. During subsequent classroom analysis, behavior recognition, emotional judgment, and concentration scores are optimized in real time based on the student's personalized parameter model, ensuring that the analysis results are more tailored to individual characteristics.

[0092] The concentration scores are:

[0093]

[0094] Where F is the behavior category B at the kth frame i The concentration score of the behavior score is α, and the concentration calculation weight of the behavior score is S B Score for behavior, S E For sentiment score, B i is the behavior category, Behavior category B i The weight of Behavior category B at the kth frame i The confidence level, E j is the emotion category, Emotion category E j The weight of is the emotion category E at the kth frame j confidence level.

[0095] S6. Smoothing the concentration score to obtain a concentration evaluation result.

[0096] To improve the stability and continuity of the evaluation results, a time series smoothing mechanism is introduced. The sliding time window is used to perform weighted averaging on the concentration scores of consecutive frames to reduce the influence of short-term fluctuations and obtain the concentration evaluation result. The concentration score after time-weighted averaging is:

[0097]

[0098] Where, Behavior category B at the t-th frame i The time-weighted average concentration score of is, T is the frame number window used for weighted average, Behavior category B at the kth frame i concentration score.

[0099] Through this continuous smoothing mechanism, a concentration change curve reflecting the students' actual classroom status is output, realizing dynamic and accurate tracking of students' concentration status.

[0100] like Figure 4 This image shows a personalized assessment of classroom focus for a student with PersonID 1. Over 18 consecutive frames, the student's behavior was mostly "reading," indicating that they were focused on their learning tasks. However, their emotional state was mostly "sad," with some periods of time when their emotions shifted to "neutral."

[0101] In another preferred embodiment, the steps are further included:

[0102] S7. Visualize the behavior, emotions, and concentration of individual and group students to generate classroom feedback data for teachers’ reference.

[0103] Through a visual human-computer interaction platform, combined with a teacher-side control module and a student-side data collection module, real-time monitoring and trend analysis of concentration are achieved. The platform provides features such as individual student concentration curves, class-wide concentration heat maps, and abnormal fluctuation warnings. It also supports data export and teaching feedback, assisting teachers in adjusting personalized teaching strategies and making classroom management decisions.

[0104] Example 2

[0105] This embodiment provides a classroom concentration assessment system based on an improved YOLOv9s, including:

[0106] The data acquisition module is used to collect classroom video streams and obtain video frames based on the classroom video streams;

[0107] The behavior detection module is used to detect the student behavior in the video frame through the behavior detection model to obtain the behavior detection results;

[0108] The facial analysis module is used to extract the facial area in the video frame to obtain the facial image, and perform image analysis on the facial image through the DeepFace model to obtain the facial analysis result;

[0109] Feature fusion module, used to dynamically weighted fuse behavior detection results and facial analysis results to obtain multimodal fusion features;

[0110] The concentration score calculation module is used to calculate the student's concentration score based on the multimodal fusion features through a personalized evaluation parameter model;

[0111] The concentration evaluation module is used to smooth the concentration score to obtain the concentration evaluation result.

[0112] The rest is the same as in Example 1.

[0113] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A classroom concentration evaluation method based on improved YOLOv9s, characterized by: The following steps are involved: Collecting classroom video streams, and obtaining video frames according to the classroom video streams; Detecting the student behavior in the video frame using a behavior detection model to obtain a behavior detection result; Extracting a facial region from the video frame to obtain a facial image, and performing image analysis on the facial image using a DeepFace model to obtain a facial analysis result; Dynamically weighting the behavior detection results and the facial analysis results to obtain a multimodal fusion feature; Calculating the student's concentration score through a personalized evaluation parameter model based on the multimodal fusion features; The concentration score is smoothed to obtain a concentration evaluation result.

2. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that, The video frames simultaneously contain the behavioral actions and facial emotional states of multiple students.

3. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that, The behavior detection model is the EM-YOLOv9s model, and the original loss function is replaced by the Focal WIoU loss function. The behavior detection model includes a backbone network and a detection head. The RepNCSPELAN4 module of the backbone network is provided with a GhostConv lightweight convolution structure, and the detection head is provided with an EMA attention mechanism.

4. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that The behavior detection result is a behavior category, which includes multiple types of hand raising, reading, and writing.

5. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that, The image analysis results include student identity and emotion category. The identification of the student identity adopts facial feature vector comparison to achieve identity mapping. The emotion category is obtained by using a Softmax multi-classifier to judge the facial emotional state. The emotion category includes multiple types of neutral, anger, joy, sadness and surprise.

6. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that The specific steps of dynamically weighted fusion of the behavior detection results and the facial analysis results to obtain multimodal fusion features are as follows: Extracting behavioral features and confidence levels from the behavioral detection results, emotional features and confidence levels from the facial analysis results, and spatiotemporal context features from the video frames; According to the importance and confidence of each modal feature, a weight calculation method based on the attention mechanism is used to dynamically allocate the proportion of each modality in the multimodal fusion feature; Construct multimodal fusion features according to the proportion of each modality in the multimodal fusion features.

7. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that The steps for constructing the personalized evaluation parameter model are specifically as follows: Based on the student identity identified in the image analysis results, the student's personal feature vector is constructed according to the student's behavior category and emotion category in the history class record; The analysis strategy parameters are dynamically adjusted according to the student's personal feature vector, thereby realizing adaptive modeling of individual differences among different students. The analysis strategy parameters include behavior detection threshold, emotion classification weight and concentration calculation weight.

8. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that The concentration scores are: Where F is the behavior category B at the kth frame i The concentration score of the behavior score is α, and the concentration calculation weight of the behavior score is S B Score for behavior, S E For sentiment score, B i is the behavior category, Behavior category B i The weight of Behavior category B at the kth frame i The confidence level, E j is the emotion category, Emotion category E j The weight of is the emotion category E at the kth frame j confidence level.

9. The classroom concentration evaluation method based on improved YOLOv9s according to claim 1 is characterized in that A time series smoothing mechanism is used to perform a time-weighted average of the concentration scores of consecutive frames using a sliding time window to obtain a concentration evaluation result. The concentration score after the time-weighted average is: Where, Behavior category B at the t-th frame i The time-weighted average concentration score of is, T is the frame number window used for weighted average, Behavior category B at the kth frame i concentration score.

10. A classroom concentration evaluation system based on improved YOLOv9s, characterized by: include: A data acquisition module is used to collect classroom video streams and obtain video frames based on the classroom video streams; A behavior detection module is used to detect the student behavior in the video frame using a behavior detection model to obtain a behavior detection result; A facial analysis module is used to extract the facial area in the video frame to obtain a facial image, and perform image analysis on the facial image using a DeepFace model to obtain a facial analysis result; A feature fusion module is used to dynamically weight the behavior detection results and the facial analysis results to obtain multimodal fusion features; A concentration score calculation module is used to calculate the student's concentration score through a personalized evaluation parameter model based on the multimodal fusion features; The concentration evaluation module is used to smooth the concentration score to obtain a concentration evaluation result.

Images