Student classroom behavior detection method based on YOLOv8 improvement

Abstract

The invention discloses a student classroom behavior detection method based on YOLOv8 improvement. According to the method, a C2F-SCG convolution module is used in a small target detection model, and compared with a traditional convolution module and an existing mainstream feature extraction module, key features of a small target can be captured more accurately, and better performance is shown in small target detection in student classroom behaviors; a small target enhanced pyramid network is used for replacing a traditional feature fusion network, it is guaranteed that the model does not have negative effects on classroom scene complex feature extraction performance, meanwhile, the model parameter quantity is effectively reduced, and in addition, a light linear adjustment detection head is used, and the detection efficiency is improved. The balance between lightweight design and linear adjustment function optimization detection precision and calculation efficiency is realized; and finally, the accuracy of bounding box regression is improved by using an Inner-WiseV3-Shape IoU loss function. According to the method, the model parameter quantity and the detection precision are greatly improved, and the balance between light weight and high performance is achieved compared with a traditional model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing technology in educational scenarios, and in particular to a student classroom behavior detection method based on improved YOLOv8. BACKGROUND

[0002] Traditional machine vision technology takes image processing and computer vision algorithm as the core, relies on optical acquisition module, image sensor and computing unit to complete image related processes, and is widely used in industrial quality inspection and other fields. The advantages are detection accuracy and response speed in structured scenes. However, in complex dynamic scenes, the adaptability and robustness are insufficient due to the reliance on artificial design features, and the generalization ability in unstructured environments is weak, and the detection stability decreases under noise.

[0003] Deep learning driven target detection technology innovates intelligent analysis in educational scenarios. YOLO series algorithm has advantages in end-to-end architecture, real-time performance, multi-scale fusion and global modeling, and performs outstandingly in behavior recognition and other tasks. Its multi-task joint learning flexibility adapts to multi-dimensional information perception in classroom scenes, providing support for intelligent teaching.

[0004] Although YOLOv8 has made progress in lightweight and precision balance, it still faces challenges in classroom behavior detection. Small target features are weak, and context integration is insufficient, which can easily lead to missed detection and false detection. Terminal deployment is constrained by computing resources, and the model parameter quantity and complexity are high, which makes it difficult to meet the low power consumption and real-time requirements. Therefore, by improving and enhancing small target expression and optimizing the model, a YOLOv8 improved method that adapts to classroom scenes is developed, which is the key to the landing of intelligent teaching analysis.

[0005] Therefore, the present application proposes a student classroom behavior detection method based on improved YOLOv8. SUMMARY

[0006] The present application aims to solve the problems of insufficient detection accuracy, high model parameter quantity and computing cost, and poor adaptability of terminal deployment in student classroom behavior target detection scenarios.

[0007] In order to achieve the above technical effects, the present application is realized by the following technical scheme: 1. A student classroom behavior detection method based on improved YOLOv8, characterized by comprising the following steps:

[0008] S1, collect student behavior videos and image files in a classroom scene, use the collected video images to make a data set based on the improved YOLOv8 network, and divide the data set into training set, validation set and test set according to the proportion; the specific steps are as follows:

[0009] S1.1, convert the target detection image into YOLO training format using self-made Python code;

[0010] S1.2, divide the self-built data set into training set, validation set and test set according to the proportion of 8:1:1;

[0011] S1.3, the train data of the self-built data set is used as the training set; the value data of the self-built data set is used as the validation set; and the test data of the self-built data set is used as the test set;

[0012] S2, use YOLOv8 improved network to train the training set, and obtain the optimal model with student behavior detection and classification ability; the specific steps are as follows:

[0013] S2.1, introduce four C2F-SCG convolution modules in the head network of YOLOv8 improved network; including first C2F-SCG module, second C2F-SCG module, third C2F-SCG module and fourth C2F-SCG module; through cascade processing, the features from low-level details to high-level behavior patterns are extracted from the classroom image layer by layer;

[0014] S2.2, use small target enhanced pyramid network in the neck network of YOLOv8 model, including CSP-OmniKernel module and SPDConv convolution module, wherein CSP-OmniKernel module effectively learns global features, obtains more efficient information utilization rate, and shows stronger generalization ability; introduce SPDConv convolution module, which increases the depth of feature map while retaining spatial information, ensures the integrity of information transmission, and small target enhanced pyramid network STEPN enhances the information exchange between different feature layers, accurately captures small target features, effectively improves the detection performance of small targets, and optimizes the multi-scale target feature fusion;

[0015] S2.3, introduce light shared convolution detection head (LSCD); including first LDConv convolution module and second LDConv convolution module, which reduces the calculation load through light convolution design, realizes behavior candidate region screening and fine classification through two-level cascade, improves the precision of similar behavior differentiation, and realizes accurate detection and classification of student classroom behavior;

[0016] S2.4, use Inner-WiseV3-Shape IoU loss function to optimize the traditional intersection over union IoU, combine the internal features, shape factors and dynamic intelligent weighting mechanism, and improve the positioning accuracy of the model;

[0017] The Inner-WiseV3-Shape IoU loss function is as follows:

[0018] L Inner-WiseV3-shape = rR WIoU (IoU-D shape -0.5Ω shape )+IoU-IoU inner (1)

[0019] wherein r represents a scale adjustment factor, D shape represents a shape distance, Ω shape represents a penalty term for penalizing the shape difference between the predicted box and the real box;

[0020] S3, using the trained optimal model to evaluate the test set, and evaluating the performance of the model in student classroom behavior detection according to the accuracy P, recall rate R and average precision;

[0021] S4, using ablation experiments and comparative experiments to evaluate the effectiveness of the improved student classroom behavior detection method based on YOLOv8; specifically, introducing the two key indicators of measuring model complexity, GFLOPs and model parameter quantity Params

[0022] GFLOPs and model parameter quantity Params;

[0023] GFLOPs and Params indicators are calculated according to equations (2)-(7):

[0024]

[0025] Parameters = r 2 × a × v + v (7)

[0026] Wherein P is the accuracy; R is the recall rate; AP represents the average precision of a single behavior class; mAP represents the average precision mean; FLOPs represents the number of floating point operations per second; Parameters represents the parameter quantity; TP represents that the model correctly predicts the student's behavior, FP represents that the model incorrectly predicts other student behaviors as student behaviors, FN represents that the model incorrectly predicts student behaviors as other student behaviors, H × W is the size of the output feature map, C in is the input channel, K is the kernel size, C out is the output channel, r is the convolution kernel size, a is the input size, and v is the output size.

[0027] S5, according to the model evaluation results, further optimize the network structure and training strategy, and improve the behavior recognition performance of the model in the actual classroom scene.

[0028] Further, in S1, the types of the dataset of the improved network based on YOLOv8 are divided into six categories: low head, hand under the table, looking left and right, normal, standing, and lifting hands.

[0029] Further, in S2.1, the C2F-SCG convolution module performs efficient feature transformation and channel interaction through a moving point convolution SMPConv, and combines a convolution gate linear unit CGLU to realize dynamic selection and enhancement of features, adopts a feature grouping and parallel sub-network structure, and effectively captures multi-scale context information through a cross-space learning mechanism; specifically as follows:

[0030] 1) The input features are first subjected to a basic convolution layer Conv for preliminary channel and spatial dimension transformation to extract basic features, and then split into multiple paths through a Split operation;

[0031] 2) Then, each branch feature first enters the SMPConv layer for convolution transformation, introduces nonlinearity through the Silu activation function, and then performs gate processing to filter information through the CGLU, and then randomly discards part of the path through the Droppath layer to improve robustness, and finally adds and fuses the initial input and the processed features element by element through the residual connection to output the final features, thereby realizing efficient feature extraction and enhancement;

[0032] Given an input C∈R C×H×W , the pooling size is H P ×W P , and the window size is k×k, the complexity Ω of the CGLU module is calculated according to equation (8),

[0033]

[0034] In the formula, R represents the expansion ratio, H P and W P represent the height and width of the to-be-transformed pooling layer, C represents the feature map channel of the input data, and k represents the convolution kernel size;

[0035] 3) Then, the branch features processed by each module are spliced and fused in the channel dimension through a Concat splicing layer, integrating the differentiated features of multiple branches to enrich the overall information; finally, the spliced and fused features are subjected to a layer of Conv convolution for final transformation, adjusting the channel number and optimizing the feature distribution, and outputting the features adapted to the subsequent network.

[0036] Further, in S2.2, the small target enhancement pyramid network includes the following steps:

[0037] 1) High-resolution feature maps are extracted from the head network, and the original feature map with size SxSxC is sliced into four sub-feature maps with size S / 2xS / 2xC by the SPDConv processing with a scaling factor, and then the channel connection is performed to obtain a feature map with size S / 2xS / 2x4C, and then the non-step convolution processing is performed to obtain a feature map with size S / 2xS / 2xC, and then the feature map is transmitted to the upper layer fusion;

[0038] 2) The fused features are input into the OmniKernel module: the input features are first preprocessed by 1x1 convolution to adjust the distribution and enhance the expression, and then respectively processed by 1x1 depth separable convolution to reserve local details, 63x63 depth separable convolution to capture context and long-range association, and double-channel attention mechanism module DCAM; finally, the features processed above are superimposed, and the final output feature is generated by 1x1 convolution modulation;

[0039] The output of the OmniKernel module is calculated as shown in formula (9)

[0040] X OKM =W out *(X Local +X Large +X Global ) (9)

[0041] In the formula, X Local represents the output of the local branch, X Large represents the output of the large branch, X Global represents the output of the global branch, and W out represents the fused feature output after 1x1 convolution;

[0042] 3) Then input the CSP_OmniKernel module, divide the features into two parts, one part is processed by the OmniKernel, and the other part is connected by jumping, and then the convolution interaction is performed to improve the efficiency;

[0043] 4) Finally, the fused features are up-sampled to the same size as the shallow features of the shallow network and fused to supplement the small target information, and the final detection result is output after the decoupling head processing.

[0044] Further, the double-channel attention mechanism module DCAM processing refers to the processing of combining the frequency domain convolution theorem, spatial attention, and frequency selective attention module FSAM to optimize the features and suppress noise.

[0045] Further, the ablation experiment and the comparative experiment are based on the YOLOv8n model, a series of improvement attempts and combinations are made on the C2f-SCG module, the small target strengthening pyramid network, the lightweight linear regulation detection head and the Inner-WiseV3-Shape IoU loss function, an ablation experiment is carried out on each combination, and a comparative experiment is carried out on the improved method and the mainstream target detection method, the specific influence of each different convolution module, feature fusion network and loss function improvement on the model performance is determined, so as to evaluate the influence on the overall algorithm performance.

[0046] The beneficial effects of the present application are:

[0047] The present application uses the C2F-SCG convolution module in the small target detection model, compared with the traditional convolution module and the existing mainstream feature extraction module, it can more accurately capture the key features of small targets, and has better performance in small target detection in student classroom behavior, effectively solving the omission and misjudgment problems of traditional detection methods.

[0048] The present application uses the small target strengthening pyramid network to replace the traditional feature fusion network, which effectively reduces the model parameter quantity while ensuring the performance of the model in extracting complex features in the classroom scene, and complements the C2F-SCG convolution module to cooperatively alleviate the problem of increased calculation caused by the introduction of a new feature extraction module.

[0049] The present application uses the lightweight linear regulation detection head to realize the balance between lightweight design and linear regulation function optimization of detection precision and calculation efficiency.

[0050] The present application uses the Inner-WiseV3-Shape IoU loss function to improve the accuracy of boundary box regression, especially for small target positioning. In summary, the present application greatly improves the model parameter quantity and detection accuracy, realizes the balance between lightweight and high performance compared with the traditional model, and lays a solid technical foundation for transplanting it to the teaching terminal device while ensuring the accuracy of small target detection in student classroom behavior. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 Flow chart of the student classroom behavior detection method based on YOLOv8 improvement;

[0053] Figure 2A structure diagram of an improved YOLOv8n network;

[0054] Figure 3 A structure diagram of a C2F-SCG convolution module;

[0055] Figure 4 A structure diagram of an SMPCGLU module;

[0056] Figure 5 A structure diagram of a CGLU module;

[0057] Figure 6 A structure diagram of a small target enhancement pyramid network (STEPN) module;

[0058] Figure 7 A structure diagram of a CSP-OmniKernel module;

[0059] Figure 8 A structure diagram of an SPDConv convolution module;

[0060] Figure 9 A structure diagram of an OmniKernel module;

[0061] Figure 10 A structure diagram of a light linear regulation detection head (LLDHead) module;

[0062] Figure 11 A structure diagram of an LDConv convolution module;

[0063] Figure 12 A structure diagram of a student classroom behavior detection system based on an improved YOLOv8

[0064] Figure 13 A comparison diagram of actual detection differences between an improved YOLOv8n detection algorithm and an original target detection algorithm. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0066] Embodiment 1

[0067] In this embodiment, as shown in the following figure, the student classroom behavior detection method based on the improved YOLOv8 is as follows: Figure 1

[0068] ​Step 1): Collect student behavior videos and image files in a classroom scene, and use the collected video images to create training sets, validation sets, and test sets based on the improved YOLOv8 network.

[0069] 5280 JPG images, using self-made Python code to convert the target detection images into YOLO training format, and dividing the self-built dataset into training set, validation set and test set according to the ratio of 8:1:1. The train and value data of the self-built dataset are used as the training set, with 4224 and 528 images respectively, a total of 4752 images for model training. The test data of the self-built dataset is used as the test set, with a total of 528 images for model testing.

[0070] Step 2): Use the improved YOLOv8 network to train the training set to obtain the optimal model with student behavior detection and classification capabilities. The network includes the following innovative modules: C2f-SCG module combines mobile point convolution (SMPConv) and convolution gate linear unit (CGLU) to improve feature extraction efficiency and solve the problem of occlusion; STEPN module optimizes multi-scale feature fusion through CSP-OmniKernel and SPD convolution, significantly improving small target detection performance; LLDHead module designs linear deformable convolution (LDConv) based on lightweight shared convolution detection head to improve adaptability and detection accuracy; Inner-WiseV3-Shape IoU loss function optimizes the intersection over union (IoU) by combining target shape and internal features to improve the positioning accuracy of small targets.

[0071] Step 3): Convolution module comparison experiment compares different feature extraction convolution modules to test the effectiveness of each convolution module in the feature extraction process; fusion network comparison experiment compares different feature fusion strategies to test the effectiveness of each model enhancement module in feature fusion to optimize overall detection performance; detection head comparison experiment evaluates the performance of different detection heads in improving small target and occluded target detection accuracy; loss function comparison experiment analyzes the influence of different loss functions on target positioning accuracy and regression effect.

[0072] In this embodiment, as shown in Figure 2 , the construction of the student classroom behavior detection model based on the improved YOLOv8 is divided into three steps:

[0073] The first step 2.1 is to introduce four C2F-SCG convolution modules in the head network of the YOLOv8 model, and then construct a high-efficiency multi-scale target feature extraction method to improve the model's ability in student behavior target detection.

[0074] In this embodiment, as shown in Figure 3As shown, the C2F-SCG convolution module performs efficient feature transformation and channel interaction through a moving point convolution (SMPConv), and realizes dynamic selection and enhancement of features in combination with a convolution gate linear unit (CGLU). The C2F-SCG convolution module includes the following steps:

[0075] The input features are first subjected to a basic convolution layer Conv for preliminary channel and spatial dimension transformation to extract basic features, and then split into multiple paths through a Split operation, and each branch is transmitted into an SMPCGLU module for fine feature extraction and nonlinear transformation to strengthen feature expression. Subsequently, the branch features processed by each module are spliced and fused in the channel dimension through a Concat splicing layer, integrating the differentiated features of multiple branches to enrich the overall information. Finally, the fused features after splicing are subjected to a layer of Conv convolution for final transformation, adjusting the channel number and optimizing the feature distribution, and outputting the features adapted for subsequent networks.

[0076] In this embodiment, as shown in Figure 4-5 The specific implementation process of the SMPCGLU convolution module is as follows: the input features first enter the SMPConv layer for convolution transformation, introduce nonlinearity through the Silu activation function, and then perform gate processing through the CGLU to filter information. Subsequently, the Droppath layer is used to randomly discard part of the paths to improve robustness. Finally, the initial input and the processed features are element-wise added and fused through a residual connection, and the final features are output, thereby realizing efficient feature extraction and enhancement. Given an input C∈R C×H×W , the pooling size is H P ×W P , and the window size is k×k, the complexity Ω of the CGLU module is calculated according to equation (8),

[0077]

[0078] In equation (8), R represents the expansion ratio, H P and W P represent the height and width of the to-be-transformed pooling layer, C represents the feature map channel of the input data, and k represents the convolution kernel size.

[0079] Step 2.2 is to use a small target enhancement pyramid network in the neck network of the YOLOv8 model. The small target enhancement pyramid network uses a CSP-OmniKernel module to effectively learn global features and obtain more efficient information utilization and stronger generalization capability. An SPDConv convolution module is introduced to increase the depth of the feature map while retaining spatial information and ensuring the integrity of information transmission. The small target enhancement pyramid network STEPN enhances the information exchange between different feature layers, accurately captures small target features, and effectively improves the detection performance of small targets.

[0080] As shown in FIG. 2B, the small target enhancement pyramid network includes the following steps: Figure 6-7

[0081] The P2 layer high-resolution feature map is extracted from the head network, processed by the SPDConv to retain small target details and adjust the scale, and then transmitted to the P3 layer fusion. The fusion features are input into the OmniKernel module to obtain multi-dimensional features through parallel processing of global, large, and local branches to improve detection accuracy and efficiency. Then, the features are input into the CSP_OmniKernel module, which divides the features into two parts, one part is processed by the OmniKernel, and the other part is connected by a skip connection. After fusion, the convolution interaction is improved. Finally, the fused features are upsampled to the same size as the shallow features of the shallow network and fused to supplement small target information. The decoupling head processes the output to output the final detection result.

[0082] As shown in FIG. 2B, the small target enhancement pyramid network includes the following steps: Figure 8

[0083] The original feature map with a size of SxSxC is sliced into four sub-feature maps with a size of S / 2xS / 2xC according to the scaling factor. After channel connection, the feature map with a size of S / 2xS / 2x4C is obtained, and then processed by non-step convolution to obtain a feature map with a size of S / 2xS / 2xC. This process increases the depth of the features and retains spatial information to ensure the integrity of information transmission.

[0084] As shown in FIG. 2B, the small target enhancement pyramid network includes the following steps: Figure 9

[0085] The input features are first preprocessed by 1x1 convolution to adjust the distribution and enhance the expression. Then, the features are processed by 1x1 depth separable convolution to retain local details, 63x63 depth separable convolution to capture context and long-range association, and double-channel attention mechanism module DCAM (combining frequency domain convolution theorem, spatial attention, and frequency selective attention module FSAM to optimize features and suppress noise). Finally, the features processed by the above methods are stacked and modulated by 1x1 convolution to generate the final output features.

[0086] ​​​The OmniKernel module outputs are calculated according to equation (9),

[0087] X OKM = W out *(X Local + X Large + X Global ) (9)

[0088] In equation (9), X Local represents the output of the local branch, X Large represents the output of the large branch, X Global represents the output of the global branch, and W out represents the fused feature output after 1x1 convolution.

[0089] The OmniKernel module combines global receptive fields, large-scale information, and local detail features, processes them in parallel through three branches, and finally fuses and outputs them, which helps to improve the detection accuracy and efficiency of the model.

[0090] Step 2.3 is to use a lightweight linear adjustment detection head, as shown in Figure 10 The lightweight linear adjustment detection head includes the following steps:

[0091] The P3, P4, and P5 output feature maps are first adjusted by 1x1Conv_GN shared convolution to adjust the channel, and the upper mixed features are then aggregated by 2 3x3LDConv_GN shared convolution. The fused features are divided into Conv_Reg target box regression, Scale layer scaling, and Conv_Cls target classification branch. The regression branch generates auxiliary boxes through Inner-IoU, assigns gradient weights through Wise-IoUv3, and optimizes small target bias through Shape-IoU. The total loss is obtained by weighting the losses of the three.

[0092] In this embodiment, as shown in Figure 11 , the linear deformable convolution LDConv first dynamically adjusts the convolution kernel position and samples the features, then learns the offset through two-dimensional convolution to adjust the position, generates the final sampling features, and finally resamples and reshapes the feature maps, outputs through convolution, normalization, and activation function, which can adapt to target changes and bring high flexibility and adaptability to the convolution network.

[0093] In this embodiment, the Inner-WiseV3-Shape IoU loss function combines target internal features, intelligent weighting, and shape factors to optimize the intersection over union index, improve the detection accuracy, convergence speed, and generalization ability. The lightweight linear adjustment detection head flexibly adapts to different datasets and targets in different positions, improving the detection accuracy and adaptability.

[0094] The Inner-WiseV3-Shape IoU loss function is calculated according to equation (1),

[0095] L Inner-WiseV3-shape = rR WIoU (IoU-D shape - 0.5Ω shape ) + IoU-IoU inner (1)

[0096] In equation (1), r represents a scale adjustment factor, D shape represents a shape distance, and Ω shape represents a penalty term for penalizing the shape difference between the predicted box and the real box.

[0097] The Inner-WiseV3-Shape IoU comprehensively considers the internal feature information of the target, is based on an intelligent weighting mechanism, and focuses on the shape factor, so as to more accurately and comprehensively measure the similarity between the predicted box and the real box.

[0098] Embodiment 2

[0099] In this embodiment, the parameter configuration used in the training process is shown in Table 1.

[0100] Table 1 Test environment and parameter configuration

[0101] Configuration Parameter Operation system Ubuntu 22.04 Graphics processing unit NVIDIA GeForce RTX 4090 Accelerated environment CUDA 12.1, CUDNN 8.9.5 Development environment PyCharm Community Edition 2024.2.2 Programming language Python 3.10 Pytorch version PyTorch 2.3.0 Data form VOC Image size 640×640 Training rounds 300 Initial learning rate 0.01 Batch size 32 Adapter SGD

[0102] Step 3): input the test set in step 1) into the optimal target detection model obtained in step 2) for verification, and combine the enhanced detection capability of the model for small target behaviors in the classroom scene to comprehensively complete the student classroom behavior detection in this method.

[0103] As Figure 12 shown, the student classroom behavior detection system improved based on YOLOv8 includes a data set construction module, a model construction module, a model training module, a model improvement module, and a model detection module; based on the test set, the optimal model

[0104] The data set construction module loads target detection image data from a self-built image database and converts it into VOC format, and divides it into a training set, a validation set, and a test set;

[0105] The model construction module uses a C2F-SCG convolution module, a small target enhancement pyramid network, and a lightweight linear adjustment detection head to optimize the YOLOv8n target detection algorithm, and constructs a student classroom behavior detection model improved based on YOLOv8;

[0106] The model training module is based on a training set to train the student classroom behavior detection model improved based on YOLOv8, and then obtain the optimal target detection model.

[0107] The model improvement module is to use the C2F-SCG convolution module, the small target strengthening pyramid network, the light linear regulation detection head, and replace and optimize the loss function of the YOLOv8 target detection algorithm to construct the student classroom behavior detection model improved based on YOLOv8.

[0108] The model detection module is to input the test set into the optimal target detection model, and then output the target detection result.

[0109] In this embodiment, the model training module focuses on using the algorithm improved based on YOLOv8, aiming at the target detection task, carrying out deep optimization and training work on a specific training set, and striving to generate a target detection model with excellent performance. This process clearly reflects the technical strategy of enhancing the model's ability to identify small targets through data-driven methods;

[0110] And the student classroom behavior detection system improved based on YOLOv8 successfully realizes the smooth connection from model learning to practical application, and can provide strong support for various application scenarios that have high requirements for the accuracy of student behavior target identification.

[0111] In this embodiment, in order to verify the effectiveness of the C2F-SCG convolution module, the small target strengthening pyramid network, the light linear regulation detection head, and the replacement and optimization of the loss function in this research, this embodiment carries out ablation experiments and comparison experiments. These experiments aim to evaluate the specific influence of the proposed improvement measures and other existing improvement methods on the performance of the YOLOv8 model. Through these experiments, the specific influence of various improvements on the performance of the model can be understood in depth.

[0112] In this embodiment, first, the evaluation criteria are determined, and the performance of the YOLO series model is usually measured by the following core indicators: precision (P), recall (R), and mean average precision (mAP); in this experiment, the embodiment selects mAP@0.5 as the key indicator to measure performance, mAP@0.5 represents the average value of mAP when the IoU threshold is 0.5, the higher the mAP value, the higher the overall accuracy of the model; when discussing the balance between the lightweight and performance of the model, in addition to focusing on the precision and speed of the model, the parameter size and computational complexity of the model also need to be considered, therefore, the embodiment introduces two key indicators to measure the complexity of the model, GFLOPs (Giga Floating Point Operations, i.e. 1 billion floating point operations per second) and Params (model parameter quantity), through these two indicators, the trade-off between resource consumption and performance of the model can be more comprehensively evaluated.

[0113] The GFLOPs and Params indicators are calculated according to equations (2)-(7):

[0114]

[0115] Parameters=r 2 ×a×v+v(7)

[0116] Where P is the accuracy; R is the recall; AP represents the average precision of a single behavior class; mAP represents the average precision mean; FLOPs represents the number of floating point operations per second; Parameters represents the parameter quantity; TP represents that the model correctly predicts the student's behavior, FP represents that the model incorrectly predicts other student behavior as the student's behavior, FN represents that the model incorrectly predicts the student's behavior as other student behavior, HxW is the size of the output feature map, C in is the input channel, K is the kernel size, C out is the output channel, r is the convolution kernel size, a is the input size, and v is the output size.

[0117] In this embodiment, the effectiveness of the proposed improved algorithm is gradually verified through ablation experiments and comparative experiments, based on the YOLOv8n model, the C2f-SCG module, the small target strengthening pyramid network, the lightweight linear adjustment detection head, and the Inner-WiseV3-Shape IoU loss function are ablation tested, the results of the ablation experiment are shown in Table 1, through this method, the specific influence of the improvement of the convolution module, the feature fusion network, the detection head, and the loss function on the model performance can be determined, so as to evaluate its influence on the overall algorithm performance.

[0118] Table 2 ablation experiment

[0119]

[0120]

[0121] In this embodiment, a series of ablation experiments are conducted on different modules of the improved algorithm. The experimental results show that all these improved modules and loss functions improve the mAP@0.5 value of the model to some extent, while reducing the parameter quantity and computational quantity. Among them, the Inner-WiseV3-Shape IoU loss function performs relatively outstanding in improving the mAP@0.5 value, further improving the detection ability of small targets; the small target enhanced pyramid network follows closely, and the effect of improving the small target detection performance is significant; the C2f-SCG feature extraction module selects effective information through the information channel, significantly enhancing the feature acquisition and fusion ability; the LLDHead detection head constructed by combining LSCD and LDConv convolution has more advantages in reducing the burden of the model and realizing lightweight, improving the accuracy and adaptability of detection. The overall algorithm improved in this paper is the most effective, compared with the initial YOLOv8 algorithm, not only effectively reduces the model complexity and parameter quantity, but also improves the detection accuracy of small targets in the classroom. Given that the detection accuracy is improved while significantly reducing the model burden, and the effectiveness in student behavior recognition has been fully proven, the improved algorithm in this paper will be used as the final solution of this embodiment.

[0122] In order to evaluate the performance of the improved YOLOv8n model in student behavior detection, a series of performance comparison tests are conducted with other models such as YOLOv5n, YOLOv8n, YOLOv9s, YOLOv10n, YOLOv11n and RT-DETR, and the test results are shown in Table 3.

[0123] Table 3 model experiment comparison

[0124]

[0125]

[0126] In this embodiment, according to the test results in Table 3, the improved algorithm has a significant advantage in performance indicators over existing algorithms such as YOLOv5n, YOLOv6n, YOLOv7, YOLOv9, YOLOv10n, YOLOv11n, etc., specifically in terms of precision (P), recall (R), and mAP@0.5, etc. Key detection performance parameters, where compared to the more performant YOLOv10n, the improved algorithm reduces parameter quantity by 0.6% and computation by 1.5%, while achieving a 1.5% improvement in mAP@0.5. Compared to YOLOv11n, the parameter quantity is reduced by 11.2% and the computation is reduced by 1.6%. In addition, Faster-RCNN and RT-DETR have parameter quantities and computations that are several times larger than the other models, but the accuracy improvement is not significant. In summary, the improved algorithm has a significant advantage over other algorithms, not only in terms of robustness, but also in terms of lightweight processing speed and detection ability, with a more comprehensive performance, and significant technical progress.

[0127] In this embodiment, in the comparison of detection effects of different algorithms, YOLOv5n has false detection in single-person behavior detection scenarios, which may be due to the complex layout of desks and chairs in the classroom and the similar colors of students' clothes, affecting image recognition and reducing the distinction between specific behaviors and backgrounds, increasing the risk of false detection. In multi-person interaction, classroom activities, body occlusion, and dynamic scenarios, all algorithms show high detection confidence without missing or false detection. In backlight scenarios, YOLOv8n has missing detection due to insufficient light and varied student postures. In the multi-person gathering and occlusion scenario, although there is missing detection, the improved algorithm shows better robustness, effectively improving mAP and recall rate, while reducing model parameters and computation, proving its adaptability and lightweight advantage in different classroom environments. Overall, the improved algorithm reduces false detection and missing detection in student behavior image detection in the real verification set, improves detection performance, and the model is more suitable for mobile deployment. As shown in Figure 13 The experimental results confirm that the improved module in this paper has obvious improvement in the accuracy of student subtle behavior detection compared to YOLOv8n, and the overall performance is also enhanced, achieving a good balance between lightweight and algorithm performance.

Claims

1. A YOLOv8 improved student classroom behavior detection method, characterized in that: The method comprises the following steps: S1, collecting student behavior video and image files under the classroom scene, using the collected video images, making a data set based on the improved network of YOLOv8, and dividing the data set into training set, validation set and test set according to the proportion; The specific steps are as follows: S1.1, using self-made Python code to convert the target detection image into YOLO training format; S1.2, divide the self-built data set into training set, validation set and test set according to the proportion of 8:1:1; S1.3, the train data of the self-built data set is used as the training set; the value data of the self-built data set is used as the validation set; and the test data of the self-built data set is used as the test set; S2, using the improved network of YOLOv8, training through the training set, and obtaining the optimal model with student behavior detection and classification ability; the specific steps are as follows: S2.1, introduce four C2F-SCG convolution modules in the head network of the improved network of YOLOv8; including first C2F-SCG module, second C2F-SCG module, third C2F-SCG module and fourth C2F-SCG module; through cascade processing, the features from low-level details to high-level behavior patterns are extracted from the classroom image layer by layer; S2.2, use small target enhanced pyramid network in the neck network of YOLOv8 model, including CSP-OmniKernel module and SPDConv convolution module, wherein the CSP-OmniKernel module effectively learns global features, obtains more efficient information utilization rate, and has stronger generalization ability; the SPDConv convolution module is introduced, which increases the depth of feature map while retaining the spatial information, ensures the integrity of information transmission, and the small target enhanced pyramid network STEPN enhances the information exchange between different feature layers, accurately captures small target features, effectively improves the detection performance of small targets, and optimizes the multi-scale target feature fusion; S2.3, introduce light shared convolution detection head (LSCD); including first LDConv convolution module and second LDConv convolution module, which realizes behavior candidate region screening and fine classification through two-stage cascade, improves the precision of similar behavior differentiation, and realizes accurate detection and classification of student classroom behavior; S2.4, the Inner-WiseV3-Shape IoU loss function is used to optimize the traditional intersection over union IoU, which combines the internal features, shape factors and dynamic intelligent weighting mechanism of the target, and improves the positioning accuracy of the model; The Inner-WiseV3-Shape IoU loss function is as follows: L Inner-WiseV3-shape = rR WIoU (IoU-D shape -0.5Ω shape )+IoU-IoU inner (1) where r represents a scale adjustment factor, D shape represents a shape distance, Ω shape represents a penalty term for penalizing the shape difference between the predicted box and the real box; S3, use the optimal model obtained by training to evaluate the test set, and evaluate the performance of the model in student classroom behavior detection according to the accuracy P, recall rate R and average precision; S4, evaluate the effectiveness of the improved student classroom behavior detection method based on YOLOv8 by ablation and comparison experiments; Specifically, introduce two key indicators to measure model complexity: GFLOPs, which is 10 billion floating point operations per second, and Params, which is the number of model parameters; GFLOPs and Params indicators are calculated according to equations (2)-(7): Parameters = r 2 x a x v + v (7) Wherein, P is the accuracy; R is the recall rate; AP represents the average precision of a single behavior category; mAP represents the average precision mean; FLOPs represents the number of billions of floating point operations per second; Parameters represents the parameter quantity; TP represents that the model actually correctly predicts the student's such behavior, FP represents that the model actually other student behavior is incorrectly predicted as the student's such behavior, FN represents that the model actually the student's such behavior is incorrectly predicted as other student behavior, HxW is the size of the output feature map, C in is the input channel, K is the kernel size, C out is the output channel, r is the convolution kernel size, a is the input size, and v is the output size; S5, according to the model evaluation results, further optimize the network structure and training strategy to improve the behavior recognition performance of the model in the actual classroom scene.

2. The student classroom behavior detection method based on YOLOv8 improvement according to claim 1, characterized in that, In S1, the data set of the improved network based on YOLOv8 is divided into six categories: low head, hand under the table, left and right, normal, standing, and lifting hands.

3. The student classroom behavior detection method based on YOLOv8 improvement according to claim 1, characterized in that, In S2.1, the C2F-SCG convolution module performs efficient feature transformation and channel interaction through the moving point convolution SMPConv, and combines the convolution gating linear unit CGLU to realize dynamic selection and enhancement of features, adopts feature grouping and parallel sub-network structure, and effectively captures multi-scale context information through cross-space learning mechanism; Specifically as follows: 1) The input features are first transformed in channel and spatial dimensions by the basic convolution layer Conv to extract the basic features, and then split into multiple paths by the Split operation; 2) Then each branch feature first enters the SMPConv layer for convolution transformation, introduces nonlinearity through the Silu activation function, then does gate processing to filter information through CGLU, and finally adds the initial input and the processed features element by element through residual connection to fuse and output the final features, so as to realize efficient feature extraction and enhancement; Given an input C ∈ R C×H×W , a pooling size of H P × W P , and a window size of k × k, the complexity Ω of the CGLU module is calculated by equation (8), In the formula, R represents an expansion ratio, H P and W P represent the height and width of the to-be-variant-pooling layer, C represents a feature map channel of input data, and k represents a convolution kernel size. 3) After that, the branch features processed by each module are spliced and fused in the channel dimension through the Concat splicing layer to integrate the differentiated features of multiple branches to enrich the overall information; Finally, the fused features are further transformed by a layer of Conv convolution to adjust the channel number and optimize the feature distribution, and output the features adapted to the subsequent network.

4. The improved student classroom behavior detection method based on YOLOv8 according to claim 1, characterized in that, In S2.2, the small target enhancement pyramid network includes the following steps: 1) Extract high-resolution feature maps from the head network, and process them through SPDConv to slice the original feature maps with size SxSxC into four sub-feature maps with size S / 2xS / 2xC according to the scaling factor, then connect the channels to get a feature map with size S / 2xS / 2x4C, and then process it through non-striding convolution to get a feature map with size S / 2xS / 2xC, and then pass it to the upper layer for fusion; 2) Input the fused features into the OmniKernel module: first preprocess the input features through 1x1 convolution to adjust the distribution and enhance the expression, then pass them through 1x1 depth separable convolution to retain local details, 63x63 depth separable convolution to capture context and long-range associations, and DCAM module for double-channel attention mechanism; Finally, stack the features processed by the above, and generate the final output features through 1x1 convolution modulation; The output calculation of the OmniKernel module is shown in equation (9) X OKM = W out *(X Local + X Large + X Global ) (9) In the formula, X Local representing the output of the local branch, X Large representing the output of the large branch, X Global representing the output of the global branch, W out representing the fused feature output after 1x1 convolution; 3) Then input the CSP_OmniKernel module, divide the features into two parts, one part is processed by OmniKernel, and the other part is connected by skip connection, and then the convolution interaction is lifted; 4) Finally, the fused features are upsampled to the same size as the shallow features of the shallow network and fused to supplement small target information, and the final detection result is output after processing by the decoupling head.

5. The improved YOLOv8-based student classroom behavior detection method according to claim 4, characterized in that, The DCAM processing refers to the processing of combining the frequency domain convolution theorem, spatial attention and frequency selective attention module FSAM to optimize the features and suppress noise.

6. The improved student classroom behavior detection method based on YOLOv8 according to claim 1, characterized in that, The ablation experiment and the comparative experiment are based on the YOLOv8n model, through a series of improvement attempts and combinations of C2f-SCG module, small target enhancement pyramid network, lightweight linear regulation detection head and Inner-WiseV3-Shape IoU loss function, the ablation experiment is carried out for each combination, and the comparative experiment is carried out between the improved method and the mainstream target detection method, the specific influence of each different convolution module, feature fusion network and loss function improvement on the model performance is determined, so as to evaluate the influence on the overall algorithm performance.

Images