A method for detecting student classroom behavior based on improved YOLOv8 model

Abstract

The invention discloses a student classroom behavior detection method based on an improved YOLOv8 model, which belongs to the field of behavior detection and recognition. The method comprises: constructing an improved YOLOv8 model, optimizing the YOLOv8 model to obtain an improved YOLOv8 model: introducing a dynamic group convolution shuffle module in a Backbone network of the YOLOv8 model to replace the fourth C2f module in the Backbone network of the original YOLOv8 model; constructing an adaptive polarization feature fusion module in a Neck network of the YOLOv8 model to replace the C2f module in the Neck network of the original YOLOv8 model; creating a task dynamic alignment detection head module to replace the detection layer of the original YOLOv8 model, thereby improving the accuracy and efficiency of student classroom behavior detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to a method for detecting student classroom behavior based on an improved YOLOv8 model. Background Art

[0002] In a classroom environment, real-time monitoring of students' behavior status not only helps teachers grasp the teaching effect, but also improves students' classroom participation and learning efficiency. However, traditional classroom management and teaching evaluation mainly rely on teachers' subjective observation and evaluation, which is not only labor-intensive but also easily affected by personal bias and difficult to ensure the quality of classroom teaching. At present, commonly used behavior detection methods mainly rely on video surveillance and image processing technology, among which target detection is one of the core technologies.

[0003] As an efficient target detection method, the YOLO (You Only Look Once) series of models are widely used due to their real-time performance and high accuracy. As the latest version of the YOLO series, the YOLOv8 model significantly improves the detection accuracy and speed by introducing a deeper feature extraction network and an optimized loss function. However, in practical applications, especially in complex classroom environments, the YOLOv8 model still has some problems. For example, the C2f module does not fully consider the semantic information between different features in feature fusion, resulting in the loss of some feature information of the target; the detection head has problems such as poor multi-scale target processing, large and redundant parameters, and insufficient interaction due to task separation. In addition, in order to meet the real-time requirements of classroom behavior detection, the computational complexity and parameter quantity of the model need to be further optimized. At the same time, the detection accuracy of the model is also affected by the low-quality annotation of training samples. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the present invention provides a method for detecting student classroom behavior based on an improved YOLOv8 model. It solves multiple problems existing in existing methods for detecting student classroom behavior, improves detection accuracy and real-time performance, and improves the quality of classroom teaching. By improving the YOLOv8 model, the present invention improves the feature fusion effect, enhances the multi-scale target detection capability, improves the real-time detection capability, and significantly improves the detection accuracy. It achieves efficient and accurate student classroom behavior detection, helps teachers better grasp the classroom teaching effect, improves students' classroom participation and learning efficiency, and provides strong support for the digitization and intelligence of education.

[0005] To achieve the above effects, the technical solution of the present invention is as follows:

[0006] The present invention provides a method for detecting student classroom behavior based on an improved YOLOv8 model, comprising the following steps:

[0007] Step 1: Optimize the YOLOv8 model to obtain an improved YOLOv8 algorithm model: introduce the DGCST module into the Backbone network of the YOLOv8 model to replace the fourth C2f module in the Backbone network of the original YOLOv8 model;

[0008] Construct an APFF module in the Neck network of the YOLOv8 model to replace the C2f module in the Neck network of the original YOLOv8 model;

[0009] The APFF module includes a sequentially connected convolutional Conv module, a Split module, an APFF Bottleneck module, a feature fusion module and a convolutional Conv module; the APFF Bottleneck module includes a sequentially connected convolutional Conv module, an LDConv convolutional module and a PSA attention mechanism;

[0010] Create a TDADH module to replace the detection layer of the original YOLOv8 model;

[0011] Step 2: Iteratively train the improved YOLOv8 algorithm model to obtain the trained improved YOLOv8 algorithm model;

[0012] Step 3: Input the image to be tested into the trained improved YOLOv8 algorithm model for testing to obtain the detection results of students' classroom behavior.

[0013] Furthermore, the working process of the DGCST module in step 1 includes:

[0014] The image of the student's classroom behavior to be identified is used as the input feature map. After the input feature map is adjusted by the 1×1 convolution layer, the channel number is divided into the first part and the second part at a ratio of 3:1. The first part retains the original features as the first feature map; the second part is subjected to 3×3 group convolution and channel shuffle operations to obtain the second feature map;

[0015] The first feature map and the second feature map are concatenated to obtain a third feature map, and the third feature map is input into a convolutional feed-forward network with a residual structure, and features are fused and compressed through two consecutive 1×1 convolutional layers to obtain a fourth feature map;

[0016] The third feature map and the fourth feature map are fused through residual connection.

[0017] Further, the TDADH module includes a feature splicing module, a task disassembly module, a positioning prediction module and a classification prediction module connected in sequence, wherein the task disassembly module includes a first task disassembly module TAP and a second task disassembly module TAP;

[0018] The working process of the TDADH module is as follows: the three feature maps of different scales output by the Neck network are extracted through two 3×3 shared convolution modules Conv_GN, and the extracted features are spliced ​​in the channel dimension to form task interaction features; the offset and the first mask and the second mask are obtained from the task interaction features;

[0019] The task interaction features are respectively input into the first task decomposition module TAP and the second task decomposition module TAP, and the task interaction features are respectively decomposed into classification features and positioning features;

[0020] The positioning feature, the offset and the first mask are positioned using the deformable convolution module DCNv2 to obtain the output positioning feature;

[0021] Multiply the classification feature and the second mask and perform classification alignment to obtain the output classification feature;

[0022] The output positioning features are convolved through the positioning convolution module Conv_Reg, and the output classification features are convolved through the classification convolution module Conv_Cls to obtain output feature maps of three different scales.

[0023] Furthermore, after step 1, the following steps are also included:

[0024] The Wise-IoU loss function is used to replace the original CIoU module in the YOLOv8 model as the bounding box regression loss function.

[0025] 5. The method according to claim 4 is characterized in that the specific formula of the Wise-IoU loss function is as follows:

[0026]

[0027] L IoU =1-I IoU

[0028]

[0029] Where, L Wise-IoU is the Wise-IoU loss function, L IoU is the IoU loss function, I IoU is the overlap between the real box and the predicted box of the input feature map; * indicates that W g ,H g Separate from the computational graph; x and y are the horizontal and vertical positions of the center point of the bounding box of the input feature map in the image; x g and g is the coordinate of the center point of the real frame; W g and H gare the width and height of the smallest circumscribed rectangular box of the predicted box and the true box, respectively; r is the non-monotonic focusing coefficient, β is the outlier degree of the anchor box, α and δ are hyperparameters, and when β=δ, r=1.

[0030] Furthermore, the training set in step 3 is input into the improved YOLOv8 algorithm model for iterative training, including:

[0031] A student classroom behavior image dataset to be identified is obtained, and the student classroom behavior image dataset to be identified is divided into a training set and a test set; the training set is input into an improved YOLOv8 algorithm model, and the improved YOLOv8 algorithm model is iteratively trained for multiple rounds through an SGD optimizer until the improved YOLOv8 algorithm model converges, thereby obtaining a trained improved YOLOv8 algorithm model.

[0032] Furthermore, the student classroom behavior image dataset includes: listening to lectures, reading, taking notes, lowering the head, turning the head, sleeping, and playing with mobile phones.

[0033] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0034] The present invention designs a method for detecting student classroom behavior based on an improved YOLOv8 model. By constructing an APFF module and a TDADH module, introducing a DGCST module, and adopting a Wise-IoU loss function, the accuracy, efficiency, and real-time performance of student classroom behavior detection are significantly improved. The feature fusion and multi-scale target processing capabilities are improved, the model parameters and computational complexity are reduced, and the adaptability of the detector to anchor frames of different qualities is enhanced. Through the above improvements, the present invention significantly improves the accuracy, efficiency, and real-time performance of student classroom behavior detection, and has high application value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the APFF module structure of the present invention;

[0036] Figure 2 It is a schematic diagram of the structure of the TDADH module of the present invention;

[0037] Figure 3 It is a schematic diagram of the DGCST module structure of the present invention;

[0038] Figure 4 It is a schematic diagram of the regression of the Wise-IOU loss function of the present invention;

[0039] Figure 5 It is a schematic diagram of the YOLOv8 model before the improvement of the present invention;

[0040] Figure 6It is a schematic diagram of the improved YOLOv8 algorithm model of the present invention;

[0041] Figure 7 It is a schematic diagram of the training results of the improved YOLOv8 algorithm model of the present invention;

[0042] Figure 8 It is a schematic diagram of the confusion matrix of the improved YOLOv8 algorithm model of the present invention. DETAILED DESCRIPTION

[0043] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0044] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0045] Example

[0046] This embodiment proposes a method for detecting student classroom behavior based on an improved YOLOv8 model. Figure 1 , including the following steps:

[0047] Step 1: Optimize the YOLOv8 model to obtain an improved YOLOv8 algorithm model: introduce the DGCST module into the Backbone network of the YOLOv8 model to replace the fourth C2f module in the Backbone network of the original YOLOv8 model;

[0048] Introduce the APFF module into the Neck network of the YOLOv8 model to replace the C2f module in the Neck network of the original YOLOv8 model;

[0049] The APFF module includes a sequentially connected convolutional Conv module, a Split module, an APFF Bottleneck module, a feature fusion module and a convolutional Conv module; the APFF Bottleneck module includes a sequentially connected convolutional Conv module, an LDConv convolution (lightweight dynamic convolution) module and a PSA attention mechanism;

[0050] Create a TDADH module to replace the detection layer of the original YOLOv8 model;

[0051] Step 2: Iteratively train the improved YOLOv8 algorithm model to obtain the trained improved YOLOv8 algorithm model;

[0052] Step 3: Input the image to be tested into the trained improved YOLOv8 algorithm model for testing to obtain the detection results of students' classroom behavior.

[0053] It is understandable that the APFF module uses a 1×1 standard convolution SC and a dynamic convolution LDConv in series, so that the network can focus more on information in different areas when performing feature fusion; at the same time, the normalization layer (BatchNormalization, BN) and the activation layer (Sigmoid-weighted Linear Unit, SiLU) are indispensable components for building high-performance neural networks, but if these layers are overused, the diversity of target features will be limited, thereby affecting the model detection performance and causing the calculation speed to slow down; therefore, the APFF module only places the normalization layer and the activation layer after LDConv, so as to achieve lower calculation delay while maintaining feature diversity; after batch normalization processing and activation function, the PSA attention mechanism is integrated to reduce the information loss caused by dimensionality reduction and better capture the fine-grained details of the target; LDConv, SC and PSA are integrated to form a new Neck network, replacing the Backbone in the original improved YOLOv8 algorithm model C2f, and the new network is called the APFF module.

[0054] As a preferred technical solution, in this embodiment, Figure 3 The working process of the dynamic group convolution shuffle DGCST module in step 1 includes:

[0055] The image of the student's classroom behavior to be identified is used as the input feature map. After the input feature map is adjusted by the 1×1 convolution layer, it is split at a ratio of 3:1 to obtain the first part and the second part respectively. The first part (i.e. 75%) retains the original features as the first feature map; the second part (i.e. 25%) undergoes 3×3 group convolution (GConv) and channel shuffle operations to enhance the feature mixability and expression ability, and obtains the second feature map;

[0056] The first feature map and the second feature map are concatenated to obtain a third feature map, and the third feature map is input into a convolutional feed-forward network (ConvFFN) with a residual structure, and feature fusion and compression are performed through two consecutive 1×1 convolutional layers to obtain a fourth feature map;

[0057] The third feature map and the fourth feature map are fused through skip connection to enhance the feature expression ability and maintain the stability of gradient transfer.

[0058] The structure of the DGCST module is as follows Figure 3 As shown in the figure, the specific implementation process is as follows: the input feature map is passed through a 1×1 convolution layer to adjust the number of channels and is divided into two parts at a ratio of 3:1; one part (75%) retains the original features, and the other part (25%) undergoes 3×3 grouped convolution (GConv) and channel shuffle operations to enhance feature mixing and expression capabilities; the two parts of the feature map are spliced ​​and enter a convolutional feedforward network (ConvFFN) with a residual structure; the ConvFFN draws on the design idea of ​​Vision Transformer, performs feature fusion and compression through two consecutive 1×1 convolution layers, and fuses the input features with the processed features through a skip connection to enhance feature expression capabilities and maintain the stability of gradient transfer; by combining grouped convolution, channel shuffle technology and visual Transformer ideas, the DGCST module improves the detection performance of the model while maintaining computational efficiency, realizes lightweight improvements to the network, and thus achieves the effect of real-time detection.

[0059] As a preferred technical solution, in this embodiment, the structure of the TDADH module is as follows: Figure 2 As shown; the TDADH module includes a feature splicing module, a task disassembly module, a positioning prediction module and a classification prediction module connected in sequence, wherein the task disassembly module includes a first task disassembly module TAP and a second task disassembly module TAP;

[0060] The working process of the TDADH module is as follows: the three feature maps of different scales output by the Neck network are extracted through two 3×3 shared convolution modules Conv_GN, and the extracted features are spliced ​​in the channel dimension to form task interaction features; the offset and the first mask and the second mask are obtained from the task interaction features;

[0061] The task interaction features are respectively input into the first task decomposition module TAP and the second task decomposition module TAP, and the task interaction features are respectively decomposed into classification features and positioning features;

[0062] The positioning feature, the offset and the first mask are positioned using the deformable convolution module DCNv2 (DeformableConvNets version 2) to obtain the output positioning feature;

[0063] Multiply the classification feature and the second mask and perform classification alignment to obtain the output classification feature;

[0064] The output positioning features are convolved through the positioning convolution module Conv_Reg, and the output classification features are convolved through the classification convolution module Conv_Cls to obtain output feature maps of three different scales.

[0065] In the specific implementation process, the structure of TDADH is shown in the figure. Three feature maps of different scales (sizes are 80×80, 40×40, and 20×20 respectively) will extract image features through two shared 3×3 convolutions, and the extracted features will be spliced ​​in the channel dimension to form task interaction features. After two task decomposition modules TAP, the task is decomposed into aligned classification and positioning prediction branches. Among them, the positioning feature uses the deformable convolution module DCNv2 (DeformableConvNets version 2, DCNv2), and generates offsets and masks through interactive features to obtain output feature maps (sizes are 80×80, 40×40, and 20×20 respectively), realizing the deformation of the convolution kernel, thereby enhancing the adjustment ability of the receptive field. This enables the convolution to better adapt to target deformation and posture changes, and improve the accuracy of target positioning.

[0066] The classification feature draws on the dynamic idea of ​​Dyhead, multiplies the second mask with the classification feature for classification alignment, and forms dynamic feature selection to enhance the interaction between classification and positioning tasks, thereby improving the multi-scale perception ability of the detection head. In order to deal with the problem of inconsistent target scales detected by each detection head, the Scale layer is used to rescale the specific output of each detection head to enhance the multi-scale detection capability.

[0067] As a preferred technical solution, in this embodiment, after step 1, the following steps are further included:

[0068] The Wise-IoU loss function is used to replace the original CIoU module in the YOLOv8 model as the bounding box regression loss function.

[0069] As a preferred technical solution, in this embodiment, the regression diagram of the Wise-IoU loss function is as follows: Figure 4 As shown in the figure, a dynamic non-monotonic focusing mechanism is combined to use "outliers" to evaluate the quality of the anchor box. The larger the outlier, the lower the quality of the anchor box, and its anchor box will be assigned a smaller gradient gain, thereby focusing the bounding box regression on the anchor box with higher quality. The specific formula of the Wise-IoU loss function is as follows:

[0070]

[0071] LIoU =1-I IoU

[0072]

[0073] Where, L Wise-IoU is the Wise-IoU loss function, L IoU is the IoU loss function, I IoU is the overlap between the real box and the predicted box of the input feature map; * indicates that W g ,H g Separate from the computational graph, thereby effectively eliminating factors that hinder convergence; x and y are the horizontal and vertical positions of the center point of the bounding box of the input feature map in the image, respectively; x g and g is the coordinate of the center point of the real frame; W g and H g are the width and height of the smallest circumscribed rectangular box of the predicted box and the true box, respectively; r is the non-monotonic focusing coefficient, β is the outlier degree of the anchor box, α and δ are hyperparameters, and when β=δ, r=1.

[0074] As a preferred technical solution, in this embodiment, the training set in step 3 is input into the improved YOLOv8 algorithm model for iterative training, which includes:

[0075] A student classroom behavior image dataset to be identified is obtained, and the student classroom behavior image dataset to be identified is divided into a training set and a test set; the training set is input into an improved YOLOv8 algorithm model, and the improved YOLOv8 algorithm model is iteratively trained for multiple rounds through an SGD optimizer until the improved YOLOv8 algorithm model converges, thereby obtaining a trained improved YOLOv8 algorithm model.

[0076] In one embodiment, the above improvement method can be applied to any model size of YOLOv8. In the embodiment of the present application, the YOLOv8 model size is used as a training example. When the improvement is performed based on the YOLOv8 model and an improved YOLOv8 algorithm model is obtained, the initial learning rate can be set to 0.01 in the SGD optimizer, the batch size is set to 16, the learning rate reduction method of cosine annealing is adopted, the number of iterations is 300, the IoU threshold is 0.5, and the optimized weight parameter model is obtained through multiple rounds of iterative training as the target improved YOLOv8 algorithm model.

[0077] As a preferred technical solution, in this embodiment, the student classroom behavior image data set includes: listening to lectures, reading, taking notes, lowering the head, turning the head, sleeping, and playing with mobile phones.

[0078] It should be noted that the test and result analysis of the present invention are as follows:

[0079] 1.1 Dataset

[0080] The test uses the publicly available student classroom behavior dataset (POCO) to evaluate the effectiveness of the classroom behavior detection method proposed in this invention. The POCO dataset after data enhancement includes 5709 images, which divides the student's class status into three types: positive, negative and neutral, covering a variety of behaviors such as listening to lectures, reading, writing, chatting, sleeping, playing with mobile phones, etc. This test randomly divides the data into training set and validation set in a ratio of 9:1, and puts them into model training and testing to evaluate the performance of the entire framework.

[0081] 1.2 Evaluation indicators

[0082] In order to objectively evaluate the performance of the student behavior detection model, the evaluation indicators used include precision (Precision, P), recall (Recall, R), average precision AP, mean average precision mAP, GFLOPs, model size (MB), and parameter quantity Params (M). GFLOPs is used to quantify the time complexity of the model, and the parameter quantity is used to measure the spatial complexity of the model. P, R, AP, and mAP reflect the detection effect of the model. The larger these values ​​are, the better the model detection effect is. The specific calculation formula is shown in the following formula. GFLOPs, model size, and model parameter quantity reflect the degree of lightweight of the model. The smaller these values ​​are, the higher the degree of lightweight of the model is, and the lower the hardware performance requirements are.

[0083]

[0084]

[0085] Among them, TP is the number of correctly predicted positive samples, TN is the number of correctly predicted negative samples, FP is the number of negative samples classified as positive samples, FN is the number of positive samples classified as negative samples, and N is the number of behavior categories.

[0086] 1.3 Test Setup

[0087] The hardware platform and environment parameter configuration used in the test training phase are shown in Table 1. During training, the input image size is set to 640×640 pixels, the number of training iterations is 300, the batch size is 16, and the initial learning rate is 0.01.

[0088] Table 1 Training environment and parameter configuration table

[0089]

[0090] 2 Evaluation of the Improved YOLOv8 Algorithm Model Based on the POCO Dataset

[0091] When training the improved YOLOv8 algorithm model for student behavior detection, the model is optimized using the Stochastic Gradient Descent (SGD) optimizer, and the mosaic data augmentation technique is used in the last 10 training stages. This adjustment is designed to improve the robustness of the model in detecting student behavior. The training results are shown in Figure 2. Figure 7 shown.

[0092] These graphs show the changes in loss and evaluation indicators of the improved model improved YOLOv8 algorithm model during training and validation. Each graph shows how different losses or evaluation indicators change with the number of training rounds (epochs). The following is an analysis of each graph: The first three columns show the positioning loss, target loss, and classification loss of the improved model. These curves depict the loss trend, where the X-axis represents epoch and the Y-axis represents the overall loss value. It can be seen from the curves that as the training progresses, the overall loss value gradually decreases and tends to stabilize. These results show that the proposed improved YOLOv8 algorithm model exhibits good fitting performance, high stability and accuracy. The last two columns show the important evaluation indicators of the model, where the X-axis represents the training time and the Y-axis represents the changes in accuracy, recall rate, and average precision under different IoU thresholds. These curves show the changes in the evaluation of target detection performance as the number of training rounds increases: the curve value gradually rises and tends to stabilize, and the performance of the model gradually improves and reaches convergence. From Figure 7 It can be seen that the proposed improved YOLOv8 algorithm model is effective.

[0093] Figure 8 The confusion matrix of the proposed improved YOLOv8 model is shown, showing the prediction results of the model for different student behaviors in the data set, and explaining the confusion between the behaviors. In the figure, the horizontal axis represents the true category label, the vertical axis represents the predicted category label, and the diagonal elements represent the correct detection rate. The darker the color, the more accurate the model's prediction on the category. It can be observed that the model proposed in the present invention has achieved a good accuracy rate in each category.

[0094] 3 Ablation test

[0095] In order to verify the performance advantages of the new module and improved model proposed in the present invention in detecting students' classroom behavior, an ablation study was conducted on the improved YOLOv8 algorithm model. As shown in Table 2 below, the present invention improves the YOLOv8n basic model, improving the model detection accuracy while achieving model lightweight.

[0096] Table 2 Comparison of ablation test results

[0097]

[0098] In YOLOv8n, the C2f in the original feature fusion stage is replaced by the APFF module, which strengthens the model's attention to information polarity, improves its adaptability to target changes, enables the network to focus more on key information in different areas, and reduces the loss of detailed information. Compared with the original YOLOv8n model, the precision, recall, mAP50, and mAP50-95 are increased by 0.8%, 0.8%, 0.8%, and 1.1%, respectively, which shows that the improved model with the introduction of the APFF module has significantly improved detection accuracy.

[0099] After replacing the original detection head module of YOLOv8 with the TDADH module, the multi-scale perception ability of the detection head is enhanced, and the interaction ability between positioning and classification tasks is also enhanced. Compared with the original YOLOv8n model, the accuracy, recall rate, mAP50 and mAP50-95 are increased by 1.4%, 1.5%, 1.8% and 2.9% respectively, while the number of model parameters is reduced by 25.5% and the model size is reduced by 25.4%. Although the model calculation amount is slightly increased, the model volume is reduced and the detection accuracy is also significantly improved. This shows that the addition of the TDADH module effectively improves the performance of the detection head in positioning and classification tasks.

[0100] In order to further achieve lightweight, the DGCST module is introduced in the feature extraction stage of the original model. The improved model reduces the amount of calculation by 6.2%, the number of parameters by 12.0%, and the model size by 11.1% while maintaining the detection accuracy.

[0101] In order to further improve the detection accuracy, the improved YOLOv8 algorithm model introduces the Wise-IoU loss function instead of the CIoU function. Without changing the amount of calculation, the number of parameters, and the model size, the recall rate, mAP50, and mAP50-95 are increased by 1.1%, 1.0%, and 1.0%, respectively.

[0102] From the ablation test results, the improved four modules effectively improve the accuracy of the entire model and achieve lightweight, indicating that these improved methods have good coupling. Therefore, the improved YOLOv8 algorithm model has significantly improved performance compared to the original YOLOv8n.

[0103] 4 Comparative test

[0104] In order to further verify the accuracy and lightweight degree of the model proposed in the present invention, 7 representative Faster R-CNN, SSD, YOLOv5n, YOLOv5s, YOLOv6s, YOLOv7-tiny, and YOLOv8n were selected for comparative testing with the improved YOLOv8 algorithm model of the present invention, and all models were trained using the same data set. Precision, recall rate, and mAP were selected as evaluation indicators of model accuracy, and the amount of calculation, the number of parameters, and the model size were selected as evaluation indicators of model speed. The test results are shown in Table 3.

[0105] Compared with the above seven models, the improved YOLOv8 algorithm model proposed in this invention has the best comprehensive performance in various indicators. Compared with Faster R-CNN, SSD, YOLOv5s, YOLOv6s, YOLOv7-tiny, and YOLOv8n, the accuracy is improved by 16.1%, 20.2%, 3.5%, 1.4%, 6.1%, 11.9%, and 1.8%, respectively, the recall rate is improved by 14.1%, 11.3%, 1.8%, 6.3%, 11.0%, and 4.0%, respectively, and the mAP is improved by 14.8%, 22.2%, respectively. , 7.2%, 11.7%, 5.6% and 3.1%, the number of parameters was reduced by 134.7M, 16.5M, 4.7M, 15.3M, 3.7M and 0.6M respectively, the amount of calculation was reduced by 360.8B, 27.8B, 8.3B, 37.7B, 5.7B and 0.6B respectively, and the model size was reduced by 101.1MB, 60.7MB, 9.4MB, 35.5MB, 7.4MB and 1.3MB respectively. Compared with YOLOv5n, although it is slightly insufficient in lightweight, it has achieved significant improvement and optimization in accuracy, and has improved 3.9%, 13.5% and 10.3% in precision, recall rate and mAP respectively. In summary, the improved YOLOv8 algorithm model proposed in the present invention has better effect on student behavior detection in classroom scenarios, and has certain advantages in detection speed, taking into account the balance between accuracy and model lightweight, and has high practicality.

[0106] Table 3 Performance comparison of different models

[0107]

[0108] Implementing the embodiments of the present invention has the following beneficial effects:

[0109] On the one hand, in order to solve the problem that the C2f module in the YOLOv8 model does not fully consider the semantic information between different features in feature fusion, resulting in the loss of some feature information of the target. The present invention constructs an adaptive polarized feature fusion (APFF) module, which pays attention to information polarity and adaptively adjusts the input while ensuring the lightweight of the model, so that the network focuses more on information in different areas and reduces the loss of detailed information. In order to adapt to the characteristics of student behavior detection scenarios and make up for the shortcomings of using only standard convolution (SC), the present invention introduces low-parameter dynamic convolution (LDConv). LDConv adaptively adjusts the receptive field capability according to the different characteristics of the input image while maintaining a low amount of calculation, so that the convolution layer can better capture the fine-grained details of the target and adapt to the changes in the pose of the target. Secondly, due to the limitation of camera image quality, the sensitivity of student pose estimation to image resolution is very high. In order to achieve accurate pose estimation of the target and reduce the interference of background information, the present invention embeds the polarized self-attention mechanism (PSA) into the module. The new Neck network is formed by fusing LDConv convolution, standard convolution SC and PSA attention mechanism, replacing the Backbone in the original improved YOLOv8 algorithm model C2f, and the new network is called APFF module. The specific process can be seen in Section 8 Specific Implementation Method.

[0110] On the second aspect, due to the problems of poor multi-scale target processing, large and redundant parameters, and insufficient interaction caused by task separation in the detection head of the YOLOv8 model, the present invention specifically designs a task dynamic alignment detection head (Task DynamicAlign Detection Head, Task Dynamic Alignment Detection Head TDADH module) module, aiming to improve the multi-scale perception ability of the detection head, thereby obtaining higher detection accuracy and efficiency, and improving the performance of the detection head in positioning and classification tasks. The group normalization layer (Group Normalization, GN) is introduced to replace BN. GN normalizes the features from the dimension of the channel and is not limited by the video memory. Secondly, the detection head structure is improved to a shared parameter structure, and the original dual-branch structure is shared and parameterized to form a smaller and lighter detection head, which can reduce the number of parameters and reduce the redundant calculation amount. Drawing on the design concept of TOOD, on the basis of the introduction / adoption of the task alignment idea in the label allocation strategy, the present invention innovatively customizes the task alignment structure on the detection head, so that the detection head can learn the task interaction features from multiple convolutional layers through the feature extractor, and enhances the interaction ability between tasks. The specific process can be seen in the specific implementation method of Section 8.

[0111] Thirdly, in order to meet the requirements of real-time classroom behavior detection, the present invention introduces a dynamic group convolution shuffle DGCST module (Dynamic Group Convolution ShuffleTransformer, DGCST) module into the backbone network of the YOLOv8 model, aiming to maintain the original detection accuracy while reducing the model parameters and computational complexity, thereby improving the detection speed of the network. This module has an innovative structural design and integrates the advantages of ShuffleNet v2 and VisionTransformer. Using the channel grouping convolution in ShuffleNet v2, the input feature map is grouped for convolution operation, which significantly reduces the number of parameters and the amount of calculation. Secondly, the channel shuffling technology of ShuffleNet v2 is used to enhance the cross-channel information exchange capability, while maintaining the resolution of the feature map, the feature expression capability is improved, and it helps to reduce the feature information loss caused by downsampling and pooling. Combined with the design idea of ​​Vision Transformer, this module can better capture global information and understand the correlation between different positions in the feature map, thereby improving the quality and resolution of the feature map, and enhancing the richness and accuracy of feature expression. In this way, a lightweight improvement of the network is achieved, so that it can achieve the effect of real-time detection. The specific process can be found in Section 8 Specific Implementation Methods.

[0112] Fourthly, to address the problem of decreased detection accuracy caused by low-quality training sample annotations, the present invention adopts the Wise-IoU loss function to replace the traditional loss function CIoU, so that the detector can take into account anchor boxes of different qualities, enhance the fitting ability of the bounding box loss, and significantly improve the detection accuracy, thereby improving the overall performance of the model.

[0113] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for detecting student classroom behavior based on an improved YOLOv8 model, characterized in that: The following steps are involved: Step 1: Optimize the YOLOv8 model to obtain an improved YOLOv8 algorithm model: introduce the dynamic group convolution shuffle DGCST module into the Backbone network of the YOLOv8 model to replace the fourth C2f module in the Backbone network of the original YOLOv8 model; The working process of the dynamic group convolution shuffle DGCST module includes: The image of the student's classroom behavior to be identified is used as the input feature map. After the input feature map is adjusted by the 1×1 convolution layer, the channel number is divided into the first part and the second part at a ratio of 3:

1. The first part retains the original features as the first feature map; the second part is subjected to 3×3 group convolution and channel shuffle operations to obtain the second feature map; The first feature map and the second feature map are concatenated to obtain a third feature map, and the third feature map is input into a convolutional feed-forward network with a residual structure, and features are fused and compressed through two consecutive 1×1 convolutional layers to obtain a fourth feature map; The third feature map and the fourth feature map are connected through residual connection to achieve feature fusion; Construct an APFF module in the Neck network of the YOLOv8 model to replace the C2f module in the Neck network of the original YOLOv8 model; The APFF module includes a sequentially connected convolutional Conv module, a Split module, an APFF Bottleneck module, a feature fusion module and a convolutional Conv module; the APFF Bottleneck module includes a sequentially connected convolutional Conv module, an LDConv convolutional module and a PSA attention mechanism; Create a TDADH module to replace the detection layer of the original YOLOv8 model; The TDADH module includes a feature splicing module, a task disassembly module, a positioning prediction module and a classification prediction module connected in sequence, wherein the task disassembly module includes a first task disassembly module TAP and a second task disassembly module TAP; The working process of the TDADH module is as follows: the three feature maps of different scales output by the Neck network are extracted through two 3×3 shared convolution modules Conv_GN, and the extracted features are spliced ​​in the channel dimension to form task interaction features; the offset and the first mask and the second mask are obtained from the task interaction features; The task interaction features are respectively input into the first task decomposition module TAP and the second task decomposition module TAP, and the task interaction features are respectively decomposed into classification features and positioning features; The positioning feature, the offset and the first mask are positioned using the deformable convolution module DCNv2 to obtain the output positioning feature; Multiply the classification feature and the second mask and perform classification alignment to obtain the output classification feature; The output positioning features are convolved through the positioning convolution module Conv_Reg, and the output classification features are convolved through the classification convolution module Conv_Cls to obtain output feature maps of three different scales; Step 2: Iteratively train the improved YOLOv8 algorithm model to obtain the trained improved YOLOv8 algorithm model; Step 3: Input the image to be tested into the trained improved YOLOv8 algorithm model for testing to obtain the detection results of students' classroom behavior.

2. The method according to claim 1, characterized in that After step 1, also include: The Wise-IoU loss function is used to replace the original CIoU module in the YOLOv8 model as the bounding box regression loss function.

3. The method according to claim 2, characterized in that The specific formula of the Wise-IoU loss function is as follows: L IoU =1-I IoU Where, L Wise-IoU is the Wise-IoU loss function, L IoU is the IoU loss function, I Iou It is the overlap degree between the real box and the predicted box of the input feature map; The * in the text box indicates that W g ,H g Separate from the computational graph; x and y are the horizontal and vertical positions of the center point of the bounding box of the input feature map in the image; x gt and gt is the coordinate of the center point of the real frame; W g and H g are the width and height of the smallest circumscribed rectangular box of the predicted box and the true box, respectively; r is the non-monotonic focusing coefficient, β is the outlier degree of the anchor box, α and δ are hyperparameters, and when β=δ, r=1.

4. The method according to claim 1, characterized in that: Step 2 iteratively trains the improved YOLOv8 algorithm model, and the trained improved YOLOv8 algorithm model includes: A student classroom behavior image dataset to be identified is obtained, and the student classroom behavior image dataset to be identified is divided into a training set and a test set; the training set is input into an improved YOLOv8 algorithm model, and the improved YOLOv8 algorithm model is iteratively trained for multiple rounds through an SGD optimizer until the improved YOLOv8 algorithm model converges, thereby obtaining a trained improved YOLOv8 algorithm model.

5. The method according to claim 4, characterized in that The student classroom behavior image dataset includes: listening to lectures, reading, taking notes, lowering the head, turning the head, sleeping, and playing with mobile phones.

Images