Student classroom behavior recognition algorithm based on YOLOv11n-RD

Abstract

The invention relates to the technical field of image recognition, in particular to a student classroom behavior recognition algorithm based on YOLOv11n-RD. Comprising the following steps: collecting student classroom behavior image data in different classroom scenes, and constructing a labeling data set comprising at least six classes of student classroom behaviors; the method comprises the steps that a YOLOv11n-RD model is constructed, the YOLOv11n model is improved, and a C3K2 module of the YOLOv11n is replaced with an RFCBAMCConv module; meanwhile, an original detection head of the YOLOv11n is replaced by a dynamic detection head DyHead; s2, training the YOLOv11n-RD network model constructed in the step S2 by using the annotation data set constructed in the step S1 to obtain a trained student classroom behavior recognition model; and inputting a to-be-recognized classroom image into the trained student classroom behavior recognition model, and outputting a student classroom behavior recognition result to complete recognition. According to the invention, based on the YOLOv11n algorithm, by optimizing the network structure and introducing multi-scale feature fusion and an attention mechanism, the detection capability of small targets is significantly improved, and the accuracy of classroom behavior recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a student classroom behavior recognition algorithm based on YOLOv11n-RD. BACKGROUND

[0002] With the deep integration of artificial intelligence and education informatization, the education industry has ushered in a profound revolution of digitalization and intelligentization. Under such a background, intelligent behavior recognition technology based on computer vision and deep learning has gradually become one of the key tools to improve the efficiency of classroom management and the quality of teaching. Student classroom behavior as an important indicator to evaluate student concentration, classroom participation and teacher teaching effect, its automatic recognition and analysis have been widely concerned by education informatization researchers. However, traditional classroom behavior recognition methods mostly rely on manual observation and recording, which not only has low efficiency and strong subjectivity, but also is difficult to adapt to large-scale and real-time education scenarios. In order to overcome these problems, researchers have introduced target detection, convolutional neural network (CNN), transfer learning and other technologies to develop automatic classroom behavior recognition systems, aiming to realize high-precision, high-efficiency and low-cost teaching behavior evaluation and intervention. Among many target detection algorithms, the YOLO (You Only Look Once) series algorithm has become the mainstream solution in the current education visual recognition field due to its real-time and detection accuracy. With the continuous iteration and upgrading of YOLOv5, YOLOv8 and other versions, the model's small target detection capability in complex background continues to improve, and has gradually possessed the potential to be applied in the classroom behavior recognition scene. However, limited by the characteristics of complex scenes such as small target size, posture change and occlusion interference, the existing algorithm still has certain missing detection and false detection problems in the recognition of student classroom behavior. In the classroom scene, student behavior as a small target is easily disturbed by occlusion and light change, and the existing model still has missing detection and false detection problems. Classroom behavior (such as raising hand, lowering head) usually occupies a small proportion of the image, and the background is complex (such as multiple occlusions, desk texture interference), and the traditional convolution operation is difficult to capture subtle feature differences due to fixed receptive field and parameter sharing mechanism. In order to further improve the detection accuracy and robustness of the model, the research of deep network structure combined with attention mechanism, dynamic detection head and receptive field optimization has become an important direction of current research.

[0003] With the rapid development of artificial intelligence and deep learning technology, its excellent feature extraction and global modeling capabilities make it have broad application potential in education informationization scenarios, especially for student behavior recognition tasks in complex environments. In recent years, researchers around the world have continuously introduced advanced target detection algorithms into teaching behavior analysis systems and have achieved remarkable results. Mainstream target detection frameworks such as YOLOv5, YOLOv8, etc. are increasingly widely used in classroom monitoring scenarios, especially in typical small target behavior recognition tasks such as "raising hands", "lowering heads", and "playing mobile phones". However, existing models still face problems such as feature loss, false detection and missed detection when dealing with small-scale targets in complex backgrounds, making it difficult to meet the requirements of real-time and high-precision coexistence.

[0004] To overcome the above limitations, researchers have tried to integrate attention mechanisms, multi-scale perception, and deformable convolution strategies into the detection network to improve the model's sensitivity and discrimination ability for student behavior details. By fusing semantic information from different feature layers to enhance the ability to capture detailed features, combining multi-scale feature flow with context diffusion mechanisms to improve the relevance between features, and finally improving the adaptability of the detection head when identifying targets of different scales through multi-scale perception and fusion mechanisms. Although the above methods have improved detection performance to some extent, they have problems such as high model complexity, large computational resource occupation, and limited adaptability, and there is currently a lack of a general recognition framework that takes into account detection accuracy, computational overhead, and practicality, suitable for real classroom complex scenarios.

[0005] Student classroom behavior is of great significance in the classroom environment, but existing methods have low accuracy when identifying student classroom behavior and are prone to false detection and missed detection. Since classroom behavior occupies a small proportion in images and is easily affected by occlusion and complex background, traditional models have difficulty in accurately capturing, and multiple downsampling can lead to loss of small target details, and traditional static detection heads cannot adapt to changes in scale, resulting in high false detection and missed detection rates. In view of this, we propose a student classroom behavior recognition algorithm based on YOLOv11n-RD. SUMMARY

[0006] The purpose of the present application is to provide a student classroom behavior recognition algorithm based on YOLOv11n-RD to solve the problems raised in the background art.

[0007] To solve the above technical problems, the purpose of the present application is to provide a student classroom behavior recognition algorithm based on YOLOv11n-RD, which includes the following steps:

[0008] S1, collect student classroom behavior image data under different classroom scenes (including light changes, student density differences), construct a labeled data set containing at least six types of student classroom behaviors, perform random horizontal flip, random cropping, brightness adjustment and Gaussian blur data enhancement processing on the data set, and divide it into training set and test set according to the set proportion; the student classroom behavior at least includes raising hand, reading, writing, using mobile phone, looking down, and crouching on the desk, etc.;

[0009] S2, construct YOLOv11n-RD model, improve YOLOv11n model, replace C3K2 module of YOLOv11n with RFCBAMConv module, fuse receptive field module RFB and convolution attention module CBAM, enhance multi-scale feature extraction ability through channel attention and spatial attention, for enhancing multi-scale feature extraction ability, solving the problem of fixed receptive field of original module and insufficient small target feature extraction; at the same time, replace the original detection head of YOLOv11n with dynamic detection head DyHead, realize adaptive feature fusion through three self-attention modules of hierarchical perception, spatial perception and task perception, solve the problems of feature loss caused by multiple down-sampling and poor adaptability of static detection head;

[0010] S3, train the YOLOv11n-RD network model constructed in step S2 using the labeled data set constructed in step S1 to obtain a trained student classroom behavior recognition model;

[0011] S4, input the classroom image to be identified (including dense scene and complex background) into the trained student classroom behavior recognition model, output the recognition result of student classroom behavior (including behavior category and confidence), and complete the recognition.

[0012] As a further improvement of the technical solution, in step S1, the construction of the labeled data set also includes data enhancement processing, which includes: performing random horizontal flip, random cropping, brightness adjustment and Gaussian blur on the original image to improve the model generalization ability; wherein the specific operation of data enhancement processing is:

[0013] Random horizontal flip is used to simulate left and right view changes;

[0014] Random cropping enhances the robustness of local features;

[0015] Brightness adjustment adapts to light changes;

[0016] Gaussian blur suppresses high-frequency noise;

[0017] Data set specifications: training set and test set are strictly divided according to 8:1, image size is uniformed to 640x640 (widthxheight), and annotation format adopts COCO format (including target bounding box coordinates and behavior category label);

[0018] The purpose of data enhancement is to improve the generalization ability of the model to different classroom scenes by expanding the data distribution diversity.

[0019] As a further improvement of the technical solution, in step S2, a receptive field attention convolution RFCBAMConv module is introduced in the C3K2 module, the RFCBAMConv module combines a receptive field module RFB and a convolution attention module CBAM, the convolution attention module CBAM is divided into channel attention and spatial attention two parts; the module first generates channel weights through channel attention (GlobalAvgPool+2 fully connected layers+Sigmoid), and then generates spatial weights through spatial attention (grouped convolution+AvgPool and MaxPool in parallel+1×1 convolution+Sigmoid), and the double weighted features are output through convolution; the implementation of the RFCBAMConv module includes the following sub-steps:

[0020] S2.1.1, channel attention processing: performing global average pooling GlobalAvgPool on the input feature map to obtain a channel description vector with a dimension of C; input the vector into the first fully connected layer and activate it through ReLU, and then input it into the second fully connected layer and activate it through Sigmoid to generate channel attention weights; multiply the weights with the input feature map channel by channel to highlight the key channel features;

[0021] S2.1.2, spatial attention and receptive field processing: performing grouped convolution on the output feature map of step S2.1.1 to extract local receptive field features, and after layer normalization LayerNorm and ReLU activation, performing average pooling AvgPool and maximum pooling MaxPool in parallel to obtain two spatial context feature maps; concatenate the two spatial context feature maps, and then perform 1×1 convolution and Sigmoid activation to generate spatial attention weights; multiply the spatial attention weights with the output feature map of step S2.1.1 pixel by pixel to strengthen the target spatial position;

[0022] S2.1.3, feature integration: integrating the feature map processed in step S2.1.2 through a 3×3 convolution layer to output an enhanced feature map, taking into account multi-scale receptive fields and detailed features.

[0023] As a further improvement of the technical solution, in step S2, the optimization of the dynamic detection head DyHead for YOLOv11 includes:

[0024] In YOLOv11, the output of the backbone network is regarded as a three-dimensional tensor, and the dimensions are horizontal L, spatial S and channel C; the dynamic detection head DyHead is added on the basis of the standard single-stage detection head, and three lightweight self-attention modules are sequentially concatenated , spatial perception and channel perception , dynamically reweighting the input feature map; introducing scale perception attention in the detection head, so that the weight of each feature level is adaptively adjusted according to the input, thereby realizing unified detection head design and enhancing feature representation capability; specifically, given a three-dimensional feature tensor , the attention function calculation formula is as follows:

[0025] ;

[0026] Wherein, π(*) is the attention function, π L (*)、π S (*)、π C (*) respectively represent three different attention mechanism functions suitable for dimensions L, S and C, and “·” represents element-wise multiplication.

[0027] By dynamically adjusting the feature weight, the problem of insufficient adaptability of static detection head to multi-scale targets and complex background is solved.

[0028] As a further improvement of the technical solution, in step S2, the level perception of the dynamic detection head DyHead is realized by the following way:

[0029] The input FPN level feature map is respectively subjected to global average pooling to obtain the corresponding level channel description vector;

[0030] The vector is input into a 1x1 convolution layer and processed by a ReLU activation function to generate a level channel attention weight, which is used to adaptively adjust the channel importance of FPN different scale feature maps;

[0031] The weight is multiplied with the corresponding level feature map channel by channel, and the channel weight of FPN different scale feature maps is adaptively adjusted to solve the problem of imbalance of feature attention degree of YOLOv11n to different size behavior targets.

[0032] As a further improvement of the technical solution, in step S2, the spatial perception of the dynamic detection head DyHead is realized by the following way:

[0033] The input level perception weighted feature map is subjected to scale indexing as additional spatial information;

[0034] The feature map is input into a deformable convolution layer to sample the offset position features of the target key area;

[0035] After weighting and summing the sampling results, the spatial attention weight is generated through Sigmoid activation.

[0036] The spatial attention weight is multiplied with the feature map pixel by pixel; the target key area is focused by the deformable convolution focusing behavior, and the effective features of the occlusion area are focused, so as to solve the occlusion interference problem of YOLOv11n in the student dense scene.

[0037] As a further improvement of the technical solution, in the step S2, the task perception of the dynamic detection head DyHead is This is achieved by the following way:

[0038] The spatial perception The weighted feature map is globally averaged and pooled to generate a channel description vector;

[0039] After two fully connected layers and a normalization layer processing: the vector is input into the first fully connected layer and the normalization layer BatchNorm, and then input into the second fully connected layer;

[0040] The output is normalized to the interval [0, 1] by the offset sigmoid function, generating a task weight, the positive channel is used for classification task, and the negative channel is used for regression task;

[0041] The dynamic switching of the classification and regression task special channel is realized; the classification and regression task channel is dynamically switched to solve the problem of insufficient adaptability of YOLOv11n detection head to classification and positioning tasks.

[0042] As a further improvement of the technical solution, in the step S4, after the to-be-identified classroom image is input into the trained YOLOv11n-RD model, the output recognition result of the student classroom behavior includes the behavior category and the confidence, wherein the output behavior recognition confidence is a core index for measuring the reliability of the model in predicting a certain behavior category, and the calculation fuses the classification task confidence and the positioning accuracy;

[0043] The behavior recognition confidence The classification confidence And the positioning confidence are weighted to obtain:

[0044] The classification confidence represents the probability prediction of the model on the target belonging to a certain behavior category; the positioning confidence represents the matching degree of the behavior region bounding box predicted by the model and the real region (measured by the intersection over union);

[0045] The final confidence formula is:

[0046] ;

[0047] Wherein, ​These are the weighting coefficients;

[0048] Furthermore, the confidence threshold is used for filtering and output: To reduce false detections, the model filters the output results using a confidence threshold.

[0049] when At that time, the behavior recognition result is retained, where The confidence threshold;

[0050] For multi-class predictions of the same target, the class with the highest confidence level is taken as the final output.

[0051] As a further improvement to this technical solution, the classification confidence level... The calculation is as follows:

[0052] The classification confidence comes from the task-aware module of the dynamic detection head DyHead. The output, specifically the predicted probabilities of the classification task channels, is normalized using the Sigmoid activation function:

[0053] ;

[0054] In the formula: The feature map after processing by the spatial perception module; This is a global average pooling operation that generates channel description vectors. , These are two fully connected layers in the task-aware module; For batch normalization operations; The Sigmoid activation function has an output range of... This indicates that the target belongs to the category. The probability of.

[0055] As a further improvement to this technical solution, the location confidence... The calculation is as follows:

[0056] Location reliability is determined by the intersection-union ratio (IUU) of the predicted bounding box and the actual behavior region. The measurement formula is:

[0057] ;

[0058] In the formula: The bounding box of the behavioral region predicted by the model (output by the regression task channel); The bounding box of the real behavior region (annotated in the training set); A larger value indicates a more accurate positioning.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] 1. In the YOLOv11n-RD-based student classroom behavior recognition algorithm, the receptive field attention mechanism and the dynamic detection head are used to optimize the network structure, improve the accuracy, robustness and real-time performance of classroom behavior recognition, and provide reliable technical support for teaching evaluation; the model proposed in the application introduces the RFCBAMConv module to improve the C3K2 module in the aspect of small target feature extraction, which can adaptively adjust the size of the receptive field and enhance the local feature extraction capability, thereby greatly improving the perception ability of the model to targets of different scales; in addition, the dynamic detection head DyHead is introduced to replace the original detection head, which can effectively deal with occlusion and background interference, enhance the small target feature representation ability without increasing the calculation overhead, and still accurately recognize subtle behaviors in dense classrooms and complex backgrounds, greatly reducing the missed detection rate and false detection rate;

[0061] 2. In the YOLOv11n-RD-based student classroom behavior recognition algorithm, the RFCBAMConv module and the DyHead structure are fused to significantly enhance the feature expression ability and small target detection ability of the model; the network model introduces multi-scale receptive field and attention mechanism to dynamically capture the micro behavior features in the classroom image, effectively avoiding the loss of key information;

[0062] 3. In the YOLOv11n-RD-based student classroom behavior recognition algorithm, based on the YOLOv11n algorithm, the network structure is optimized, multi-scale feature fusion and attention mechanism are introduced, and the detection ability of small targets is significantly improved, thereby improving the accuracy of classroom behavior recognition. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The algorithm flowchart in the application is shown in the figure;

[0064] Figure 2 The YOLOv11n-RD model structure diagram in the application is shown in the figure;

[0065] Figure 3 The RFCBAMConv model structure diagram in the application is shown in the figure;

[0066] Figure 4 The DyHead model structure diagram in the application is shown in the figure;

[0067] Figure 5 The model parameter comparison diagram before and after improvement in the application is shown in the figure;

[0068] Figure 6 The detection result comparison diagram of the improved model (right) before (left) in the application is shown in the figure. DETAILED DESCRIPTION

[0069] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0070] Embodiment 1

[0071] YOLOv11 is a target detection algorithm disclosed by Ultralytics Company on September 30, 2024, which has achieved significant enhancement in detection accuracy, inference speed and computational efficiency compared with previous YOLO series models. In the backbone network, the original C2F module is replaced by the C3K2 module, which uses the CBS (Convolution, Batch Normalization and SiLU Activation Function) module to first perform convolution processing, then performs batch normalization, and finally enhances the output effect through the SiLU activation function; After the Fast-SPP layer of the fast spatial pyramid pooling, a new pyramid slice attention C2PSA module is added, which combines the cross-stage local structure with the pyramid compression attention, and extracts the global features through the multi-head attention mechanism, so as to further strengthen the feature expression ability of the model. The neck network adopts the structure design of the fusion of the path aggregation network PAN and the feature pyramid network FPN, which effectively integrates the shallow position information and the deep semantic information, and enhances the positioning performance of targets of different scales. In addition, the standard convolution in the classification and regression detection head is replaced by the depth separable convolution DWConv, which reduces the model parameter quantity and the computational overhead while maintaining the detection performance.

[0072] As shown in Figures 1-4 The embodiment provides a student classroom behavior recognition algorithm based on YOLOv11n-RD, which comprises the following steps:

[0073] S1, student classroom behavior image data under different classroom scenes (including light changes and student density differences) is collected, a labeled data set containing at least six types of student classroom behaviors is constructed, data enhancement processing of random horizontal flipping, random cropping, brightness adjustment and Gaussian blur is performed on the data set, and the data set is divided into a training set and a test set (the image size is 640x640) according to a ratio of 8:1; the student classroom behavior at least includes raising hands, reading, writing, using a mobile phone, looking down, and crouching on a desk;

[0074] In this step, the construction of the labeled data set also includes data enhancement processing, which includes: performing random horizontal flipping, random cropping, brightness adjustment and Gaussian blur on the original image to improve the model generalization capability; wherein the specific operation of the data enhancement processing is:

[0075] Random horizontal flip (probability 50%) is used to simulate the change of left and right view;

[0076] Random cropping (cropping area is 30%-100% of the original image) enhances the robustness of local features;

[0077] Brightness adjustment (brightness coefficient range 0.5-1.5) adapts to changes in lighting;

[0078] Gaussian blur (kernel size 3x3 or 5x5) suppresses high-frequency noise;

[0079] Dataset specifications: The training set and test set are strictly divided according to 8:1, the image size is unified to 640x640 (widthxheight), and the annotation format uses COCO format (including target bounding box coordinates and behavior class labels);

[0080] The purpose of data augmentation is to expand the diversity of data distribution and improve the generalization ability of the model to different classroom scenes.

[0081] S2, build YOLOv11n-RD model, improve YOLOv11n model, replace C3K2 module of YOLOv11n with RFCBAMConv module (fusion receptive field module RFB and convolution attention module CBAM, through channel attention (GlobalAvgPool+2 fully connected layers+Sigmoid) and spatial attention (grouped convolution+AvgPool and MaxPool in parallel+1x1 convolution+Sigmoid) to enhance multi-scale feature extraction ability, used to enhance multi-scale feature extraction ability), solve the problem of fixed receptive field of original module and insufficient small target feature extraction; At the same time, replace the original detection head of YOLOv11n with dynamic detection head DyHead, realize adaptive feature fusion through three self-attention modules of hierarchical perception (GlobalAvgPool+Conv+ReLU), spatial perception (scale index+deformable convolution+Sigmoid) and task perception (GlobalAvgPool+2 fully connected layers+HardSigmoid), solve the problem of feature loss caused by multiple down-sampling and poor adaptability of static detection head;

[0082] In this step, a receptive field attention convolution RFCBAMConv module is introduced in the C3K2 module. The RFCBAMConv module combines a receptive field module RFB and a convolution attention module CBAM. The convolution attention module CBAM is divided into channel attention and spatial attention. The module first generates channel weights through channel attention (GlobalAvgPool+2 fully connected layers+Sigmoid), and then generates spatial weights through spatial attention (grouped convolution+AvgPool and MaxPool in parallel+1x1 convolution+Sigmoid). After double weighting, the features are output through convolution. The implementation of the RFCBAMConv module includes the following sub-steps:

[0083] S2.1.1, channel attention processing: performing global average pooling GlobalAvgPool on the input feature map to obtain a channel description vector with a dimension of C; inputting the vector into a first fully connected layer (dimension ) and performing ReLU activation, and then inputting into a second fully connected layer (dimension ) and performing Sigmoid activation to generate channel attention weights; multiplying the weights and the input feature map channel by channel to highlight key channel features (such as arm feature channels of the "hand raising" action);

[0084] S2.1.2, spatial attention and receptive field processing: performing grouped convolution ( ) on the output feature map of step S2.1.1 to extract local receptive field features, which are activated by layer normalization LayerNorm and ReLU, and then performing average pooling AvgPool and maximum pooling MaxPool (pooling kernel size 3x3) in parallel to obtain two spatial context feature maps. After splicing, 1x1 convolution (dimension reduction to 1 channel) and Sigmoid activation are performed to generate spatial attention weights. Multiply the spatial attention weights and the output feature map of step S2.1.1 pixel by pixel to strengthen the target spatial position (such as the hand region of "using a mobile phone");

[0085] S2.1.3, feature integration: integrating the feature map processed in step S2.1.2 through a 3x3 convolution layer (stride 1, padding 1) to output an enhanced feature map, which takes into account multi-scale receptive fields and detailed features.

[0086] In the C3K2 module, the receptive field attention convolution RFCBAMConv is introduced, which further expands the receptive field range and strengthens the multi-scale feature extraction capability through the variable receptive field and channel-spatial double attention mechanism, while considering the control of the calculation cost. Secondly, to deal with the problem of feature information loss caused by multiple downsampling, the detection head of the original model is replaced with a dynamic detection head DyHead with a self-attention mechanism. The detection head realizes unified adaptive feature fusion in the scale, space and task dimensions through the self-attention mechanism, significantly improving the representation ability and accuracy of small target detection. The structure diagram of the improved model network YOLOv11n-RD is as shown in Figure 2

[0087] In this process, first, the channel attention module, the input feature is subjected to global average pooling GlobalAvgPool in the spatial dimension to obtain a channel description vector; and through a fully connected layer Linear + ReLU activation, and then through a second fully connected layer Linear + Sigmoid, each channel obtains a weight coefficient; the channel attention weight is multiplied with the input feature channel by channel, highlighting important channels and suppressing redundant channels;

[0088] Secondly, the spatial attention and receptive field module, first, the local receptive field feature is extracted by the group convolution Group Conv, and then the layer normalization Norm + ReLU activation is performed; then the convolution output is adjusted to a shape suitable for subsequent pooling through Adjust Shape, and the average pooling AvgPool and the maximum pooling MaxPool are used in parallel to extract the spatial context information. The spliced or added pooling results are subjected to 1x1 convolution Conv, and then Sigmoid to obtain the attention weight of each spatial position. The spatial attention weight is multiplied with the channel-weighted feature again, highlighting the key spatial region; the feature processed by the above double attention mechanism RFCBAM can consider multi-scale receptive field and fine spatial position;

[0089] Finally, the final output feature map is generated through a layer of convolution integration. By introducing the receptive field attention mechanism and combining the receptive field spatial features, the attention degree of each channel and each spatial position can be dynamically adjusted, the receptive field can be expanded and covered in multiple scales, and the ability to capture small targets and detailed features can be strengthened without significantly increasing the calculation amount.

[0090] The C3K2 module in the original YOLOv11n model adopts standard convolution operation, the receptive field range is large and fixed, lacks dynamic adjustment ability, cannot flexibly adapt to the scale change of the target, and limits the capture ability of different position small target difference information. Therefore, the scheme introduces a new type of convolution operation named RFCBAMConv in the C3K2 module of YOLOv11n, and the structure diagram is as shown in​Figure 3 as shown.

[0091] CBAM focuses on the importance of feature channels and the relevance of spatial positions through two sub-modules, channel attention and spatial attention. Meanwhile, the channel attention of CBAM enhances the weight of feature channels related to small targets, and the spatial attention highlights the spatial position of small targets. This dual mechanism effectively suppresses the interference of background noise on small targets, enhances the pertinence of features, and significantly improves the recognizability of small targets. The multi-scale receptive field design of RFB provides rich contextual information for CBAM, while the attention mechanism of CBAM further highlights key features and suppresses the influence of irrelevant areas. This synergistic effect enables RFCBAMConv to finely focus on local features while retaining global context awareness, significantly improving the robustness of feature expression. Therefore, RFCBAMConv combines receptive field and attention mechanism, making the model more sensitive to small targets, reducing the miss rate, especially in dense scenes or long-distance target detection.

[0092] Further, the optimization of dynamic detection head DyHead for YOLOv11n includes:

[0093] Since the original detection head of YOLOv11n is not good at small target detection, this paper introduces a dynamic detection head to replace the original detection head, and the specific structure is as shown in Figure 4 .

[0094] In YOLOv11, the output of the backbone network is regarded as a three-dimensional tensor, whose dimensions are horizontal (i.e., FPN feature level) L, spatial S (where the spatial dimension is HxW), and channel C; the dynamic detection head DyHead is added on top of the standard single-stage detection head, which dynamically reweights the input feature map by sequentially cascading three lightweight self-attention modules: level perception , spatial perception and channel perception ; scale perception attention is introduced in the detection head to adaptively adjust the weight of each feature level according to the input, thereby realizing a unified detection head design and enhancing the feature representation capability; specifically, given a three-dimensional feature tensor , the attention function calculation formula is as follows:

[0095] ;

[0096] where π(*) is the attention function, π L (*), π S (*) and π C (*) represent three different attention mechanism functions applicable to dimensions L, S and C, respectively, and “·” represents element-wise multiplication.

[0097] Three self-attention modules cascade: hierarchical perception Adjust the weight of different levels of channels, spatial perception Focus on key spatial areas, task perception Adapt to classification / regression tasks

[0098] By dynamically adjusting the feature weight, the problem of insufficient adaptability of static detection head to multi-scale targets and complex background is solved.

[0099] First, the hierarchical perception of dynamic detection head DyHead It is realized in the following way:

[0100] Global average pooling is performed on the input FPN feature maps of each level to obtain the corresponding level channel description vector;

[0101] The vector is input into a 1x1 convolution layer (Conv, dimension ) and processed by a ReLU activation function to generate level channel attention weights for adaptively adjusting the channel importance of FPN feature maps of different scales;

[0102] The weights are multiplied with the corresponding level feature maps channel by channel to adaptively adjust the channel weights of FPN feature maps of different scales, solving the problem of imbalance of feature attention for different size behavior targets (such as "hand raising" small targets and "lying on the table" large targets) in YOLOv11n.

[0103] Second, the spatial perception of dynamic detection head DyHead It is realized in the following way:

[0104] The input hierarchical perception The weighted feature maps are indexed by scale (e.g. P3→0, P4→1, P5→2) as additional spatial information;

[0105] The feature maps are input into a deformable convolution layer (convolution kernel 3x3, offset range ) to sample the offset position features of the target key area;

[0106] The weighted sum of the sampling results is processed by Sigmoid activation to generate spatial attention weights;

[0107] The spatial attention weights are multiplied with the feature maps pixel by pixel; through deformable convolution, the behavior target key area is focused, and the effective features of the occluded area (such as the partially occluded "writing" hand in a crowded classroom) are focused, solving the problem of occlusion interference in YOLOv11n in a student-dense scene.

[0108] Third, the task perception of dynamic detection head DyHead This can be achieved in the following ways:

[0109] Spatial perception The weighted feature maps are then subjected to global average pooling to generate channel description vectors.

[0110] The vector is processed through two fully connected layers and a normalization layer: the vector is input into the first fully connected layer (dimension C → C / 2) and the normalization layer BatchNorm, and then input into the second fully connected layer (dimension C / 2 → C).

[0111] By offsetting the Sigmoid function (output range) Normalize the output to The interval is used to generate task weights. Positive values ​​are used for classification tasks (behavior category judgment), and negative values ​​are used for regression tasks (bounding box localization).

[0112] Enables dynamic switching between dedicated channels for classification and regression tasks; dynamically switching between classification and regression task channels solves the problem of insufficient adaptability of the YOLOv11n detection head to classification and localization tasks.

[0113] The first improvement is in hierarchical perception. The importance of each channel can be adaptively adjusted according to the feature map level of different scales of FPN. First, global average pooling AvgPool is performed to obtain a channel description vector of length C. The vector is then mapped to the dimension reduction space through linear convolution Conv. ReLU activation is then introduced to obtain a channel attention weight.

[0114] Secondly, spatial perception This is used to adaptively highlight key spatial locations within a single feature map; first, the processed... The weighted fused features are indexed by a scale and used as additional input. Then, deformable convolutions (Conv) are used to sample from a few offset positions. The sampling results are weighted and summed to obtain spatial attention weights, and finally, the output is passed through a sigmoid function. And multiply it pixel by pixel with the feature map;

[0115] Finally, to achieve joint learning and generalize different representations of objects, task-aware attention was deployed at the endpoint. It can adapt to different task requirements by dynamically switching feature channels; first, it performs global average pooling (AvgPool) on the spatially weighted features to reduce the dimensionality and generate a C-channel description vector. Then, through two fully connected layers and a normalization layer, the output is finally normalized to the [−1,1] interval by applying the offset Sigmoid function, thereby realizing the dynamic switching of dedicated channels for different tasks such as classification and regression.

[0116] Specifically, DyHead introduces a self-attention mechanism across each level of the feature pyramid to achieve a unified adaptive fusion of scale awareness, spatial awareness, and task awareness: enhancing multi-size target features through cross-scale attention, highlighting target location information through spatial attention, and optimizing feature representation for classification and regression tasks through task attention. This dynamic detection head significantly improves the representational ability and detection accuracy of small targets without increasing computational overhead. The DyHead structure enhances the ability to capture small-scale behavioral features by utilizing multi-level feature fusion and adaptive attention mechanisms. The scale awareness mechanism enables the model to dynamically adjust the feature extraction process according to the target size, ensuring that small target behaviors are fully represented while avoiding the loss of detailed information. Furthermore, the spatial awareness mechanism allows the detection head to distinguish adjacent individuals in densely populated classroom environments, thereby reducing detection errors caused by occlusion. The task awareness mechanism further strengthens the model's ability to balance classification and localization tasks, ensuring that behavioral features are learned and recognized more accurately.

[0117] S3. Use the labeled dataset constructed in step S1 to train the YOLOv11n-RD network model constructed in step S2 to obtain the trained student classroom behavior recognition model.

[0118] During model training, it is necessary to evaluate the model's recognition performance, and the evaluation metrics used include accuracy. Recall rate Average accuracy and mean average precision .

[0119] S4. Input the classroom image to be identified (including dense scenes and complex backgrounds) into the trained student classroom behavior recognition model, and output the recognition results of student classroom behavior (including behavior category and confidence level) to complete the recognition.

[0120] In this step, after the YOLOv11n-RD model trained on the classroom image to be identified is input, the output of the student classroom behavior recognition results includes behavior category and confidence level. Among them, the output behavior recognition confidence level is the core indicator for measuring the reliability of the model's prediction of a certain behavior category. Its calculation integrates the classification task confidence level and the localization accuracy.

[0121] Behavior recognition confidence Classification confidence Location reliability The weighted average of the two parts yields:

[0122] Classification confidence This represents the model's probability prediction of a target belonging to a certain behavioral category; location-based reliability. The matching degree (measured by the intersection over union) between the bounding box of the behavior region predicted by the model and the real region;

[0123] The final confidence formula is:

[0124] ;

[0125] wherein, is a weight coefficient (which can be set as 0.7, and the classification reliability is preferentially guaranteed);

[0126] The calculation of the classification confidence is specifically as follows:

[0127] The output of the task perception module of the dynamic detection head DyHead is specifically the prediction probability of the classification task channel, which is normalized by a Sigmoid activation function:

[0128] ;

[0129] wherein, is the feature map processed by the spatial perception module; is a global average pooling operation, generating a channel description vector; , are two fully connected layers in the task perception module; is a batch normalization operation; is a Sigmoid activation function, and the output range is , indicating the probability that the target belongs to the class ;

[0130] The calculation of the positioning confidence is specifically as follows:

[0131] The positioning confidence is measured by the intersection over union between the predicted bounding box and the real behavior region, and the formula is:

[0132] ;

[0133] wherein, is the bounding box of the behavior region predicted by the model (output by the regression task channel); is the bounding box of the real behavior region (annotated in the training set); The greater the value, the more accurate the positioning is;

[0134] Further, the confidence threshold is filtered and output: in order to reduce false detection, the model performs confidence threshold filtering on the output result:

[0135] When , the behavior recognition result is retained ( ​(The confidence threshold was set to 0.35 in the experiment).

[0136] For multi-class predictions of the same target, the class with the highest confidence level is taken as the final output.

[0137] The final output format is: (behavior category, Conf), for example (using mobile phone, 0.89), where the confidence score is rounded to two decimal places;

[0138] Furthermore, confidence level directly affects the model's recognition performance:

[0139] When confidence threshold As accuracy increases Improvement (reducing false positives), but recall rate It may decrease (increase the number of missed detections);

[0140] Average accuracy The calculation is based on different confidence thresholds. The line integral, i.e.: ,in For a given recall rate The maximum accuracy at any given time, whose value changes dynamically as the confidence threshold is adjusted.

[0141] In summary, this student classroom behavior recognition algorithm can be applied to smart classroom systems to monitor students' classroom focus in real time or evaluate teachers' teaching effectiveness, especially in educational deployments such as smart classrooms and integrated applications of classroom behavior analysis systems.

[0142] In addition, other fusion networks can be used as the basic backbone, and lightweight attention modules can be superimposed to enhance feature representation. Transform-based detection heads can be used to improve the ability to capture small targets.

[0143] Simulation Verification Examples

[0144] In order to verify the feasibility and effectiveness of the algorithm of the present application, we comprehensively evaluated YOLOv11n-RD on a computer simulation platform. First, in the Python3.10 virtual environment created in Miniconda, PyTorch2.1.0 was used for model training and testing; the hardware environment was AMD Ryzen 75800H (integrated Radeon Graphics) processor, and NVIDIA GeForce RTX3060 GPU was used for acceleration operation. The data came from the classroom behavior image dataset built by the present research, a total of 6351 labeled images, of which 5646 were used for training (including "raise hand", "read", "write", "use mobile phone", "lower head", "crouch desk" six types of behavior), and 705 were used for testing. The input image size in the training stage was fixed at 640x640, the training round was 500, the batch size was 16, the initial learning rate was 0.01, the learning rate decay factor was 0.01, and the confidence threshold was 0.35. The improved YOLOv11n-RD algorithm was used to train the dataset, and after the training was completed, the test set was tested.

[0145] The present application uses four indicators commonly used by researchers as evaluation indicators of the model to evaluate the recognition effect of the improved algorithm on students' classroom behavior, which are: precision , which represents the proportion of samples that are actually positive in the samples predicted by the model as positive; recall , which represents the proportion of samples that are actually positive and are correctly predicted by the model as positive; average precision , which represents the average value of precision at different recall levels; mean average precision , which represents the average value of all classes . The calculation formulas of the four evaluation indicators are as follows:

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] In the formula, represents the number of correctly detected targets, represents the number of incorrectly detected targets, represents the number of targets that actually exist but are not detected, represents the total number of classes, represents the th class value.

[0151] To evaluate the effectiveness of the improved YOLOv11n-RD algorithm for student behavior recognition, the present application conducts experiments on a self-built classroom image dataset, selects multiple mainstream target detection models as baseline models for performance comparison, and the results are shown in Table 1. As can be seen from the table, the improved YOLOv11n-RD algorithm performs excellently in detection performance. Specifically, the improved model achieves 97.5% and 79.6% in mAP@0.5 and mAP@0.5:0.95, respectively, which is an increase of 6.8% and 5.6% compared with the original YOLOv11n model. In addition, compared with mainstream algorithms such as YOLOv5s, YOLOv8s and YOLOv10s, the improved model proposed by the present application has significant advantages in detection accuracy, model complexity and computational efficiency. The experimental results fully show that the improved YOLOv11n-RD algorithm can better recognize and detect student classroom behavior, and improve the detection accuracy.

[0152] Table 1 Comparison of experimental results

[0153] To comprehensively verify the improvement effect of each improved module on the model performance, the present application conducts an ablation experiment on the self-built dataset. The experiment is based on the YOLOv11n model, and the improved modules are integrated step by step to quantify the contribution of each module to the detection accuracy and computational efficiency. The specific experimental process is as follows: first, the RFCBAMConv structure is used to improve the C3K2 module in the original network to enhance the model's ability to extract multi-scale features; second, the dynamic detection head DyHead is used to replace the original detection head to improve the model's dynamic perception ability of target position and category. In the experiment, the influence of each improved module on the experimental results is tested separately to determine its contribution. The results of the ablation experiment are shown in Table 2.

[0154] Table 2 Ablation experiment results

[0155] The results of the ablation experiment are visualized, and the original curve is smoothed to obtain the mAP@0.5, mAP@0.5:0.95, precision, and recall comparison results of the four groups of models before and after improvement as shown in Table 3. Figure 5

[0156] The difference comparison of the detection effect of the original YOLOv11 model and the improved YOLOv11n-RD model in the same classroom scene is shown in Table 4. Figure 6 ​As shown in the contrast diagram, compared with the YOLOv11n model, the YOLOv11n-RD model has stronger detection capability in the classroom scene with dense students and complex background, can more accurately identify and locate the student behavior target, and significantly improves the detection accuracy of student behavior, which verifies the effectiveness of the improved algorithm proposed in the application in dealing with actual classroom scenes and complex background tasks. The model introduces the receptive field attention mechanism and the dynamic detection head structure, which improves the behavior recognition accuracy while ensuring real-time performance, especially the feature capture capability of typical classroom behaviors such as raising hands, lowering heads and playing mobile phones is significantly enhanced.

[0157] From the perspective of target detection deep learning, the application combines three core structures of receptive field attention convolution (RFCBAMConv), dynamic detection head (DyHead) and standard convolution module, and uses image-behavior label pairs for training. The network can automatically learn the mapping relationship between pixel-level features and semantic information. Compared with other multi-stage detection schemes that require complex preprocessing or post-processing, this method does not need to perform tedious candidate box generation, non-maximum suppression iteration or feature rearrangement operation, and can be applied to behavior class recognition in various complex classroom environments. The application optimizes the network structure and hyperparameters in view of the characteristics of small targets, serious occlusion and subtle actions in the classroom, solves the problems of overfitting and precision fluctuation, effectively evaluates the detection performance, greatly reduces the computational complexity, and is no longer limited to specific behaviors or scenes. From the perspective of multi-scale receptive field and dynamic attention, the application provides a general and efficient technical solution for student behavior recognition in modern intelligent education scenarios. Through training on public data sets, the feasibility and effectiveness of the algorithm in the actual classroom scene are verified. The results show that the proposed method has low computational overhead, high detection accuracy and is suitable for various complex teaching environments, providing an efficient and reliable technical path for building a general student behavior recognition system.

[0158] Those skilled in the art can understand that the processes for implementing all or part of the steps of the above embodiments can be completed by hardware or by programs instructing related hardware.

[0159] The basic principles, main features and advantages of the application are shown and described above. Those skilled in the art should understand that the application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only preferred examples of the application and do not limit the application. Without departing from the spirit and scope of the application, various changes and improvements can be made to the application, and these changes and improvements all fall within the scope of the claimed application. The scope of protection of the application is defined by the appended claims and their equivalents.

Claims

1. A student classroom behavior recognition algorithm based on YOLOv11n-RD, characterized in that, Comprise the following steps: S1, collect student classroom behavior image data under different classroom scenes, construct a labeled data set containing at least six kinds of student classroom behaviors, and divide it into training set and test set according to the set proportion; S2, construct YOLOv11n-RD model, improve YOLOv11n model, replace C3K2 module of YOLOv11n with RFCBAMConv module; At the same time, replace the original detection head of YOLOv11n with dynamic detection head DyHead; S3, use the labeled data set constructed in step S1 to train the YOLOv11n-RD network model constructed in step S2, and obtain the trained student classroom behavior recognition model; S4, input the classroom image to be identified into the trained student classroom behavior recognition model, output the recognition result of the student classroom behavior, and complete the recognition.

2. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 1, characterized in that: In step S1, the construction of the labeled data set also includes data enhancement processing, which includes: performing random horizontal flip, random cropping, brightness adjustment and Gaussian blur on the original image; Wherein, the specific operation of data enhancement processing is: Random horizontal flip is used to simulate left and right view changes; Random cropping enhances the robustness of local features; Brightness adjustment adapts to light changes; Gaussian blur suppresses high-frequency noise; The data set labeling format adopts COCO format.

3. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 2, characterized in that: In step S2, the receptive field attention convolution RFCBAMConv module is introduced in the C3K2 module, the RFCBAMConv module combines the receptive field module RFB and the convolution attention module CBAM, and the convolution attention module CBAM is divided into channel attention and spatial attention two parts; The implementation of the RFCBAMConv module includes the following sub-steps: S2.1.1, channel attention processing: performing global average pooling GlobalAvgPool on the input feature map to obtain a channel description vector with dimension C; The vector is input into the first fully connected layer and activated by ReLU, and then input into the second fully connected layer and activated by Sigmoid to generate channel attention weights; Multiply the weights with the input feature map by channel; S2.1.2, spatial attention and receptive field processing: performing grouped convolution on the output feature map of step S2.1.1 to extract local receptive field features, which are activated by layer normalization LayerNorm and ReLU, and then perform average pooling AvgPool and maximum pooling MaxPool in parallel to obtain 2 spatial context feature maps; The two are spliced and activated by 1x1 convolution and Sigmoid to generate spatial attention weights; Multiply the spatial attention weights with the output feature map of step S2.1.1 by pixel; S2.1.3, feature integration: integrate the feature map processed in step S2.1.2 through a 3x3 convolution layer to output an enhanced feature map.

4. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 3, characterized in that: In step S2, the optimization of the dynamic detection head DyHead for YOLOv11n includes: In YOLOv11, the output of the backbone network is regarded as a three-dimensional tensor, whose dimensions are horizontal L, spatial S and channel C respectively; the dynamic detection head DyHead is added on the standard single-stage detection head, which dynamically reweights the input feature map by sequentially cascading three lightweight self-attention modules, i.e., level perception π L , spatial perception π S and channel perception π C ; given a three-dimensional feature tensor F∈R L×S×C , the attention function is calculated as follows: W(F) = π C (π S (πL(F) · F) · F); where π(*) is an attention function, π L (*), π S (*), π C (*) are three different attention mechanism functions applied to dimensions L, S and C, respectively, and "•" denotes element-wise multiplication.

5. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 4, characterized in that: In the step S2, the level awareness of the dynamic detection head DyHead This is achieved by: Perform global average pooling on the input FPN feature maps of each level to obtain the corresponding level channel description vector; The vector is input into a 1*1 convolution layer and processed by a ReLU activation function to generate hierarchical channel attention weights; The weights are multiplied with the corresponding hierarchical feature maps channel by channel to adaptively adjust the channel weights of FPN feature maps of different scales, solving the problem of imbalance of feature attention for different size behavior targets in YOLOv11n.

6. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 5, characterized in that: The spatial perception of the dynamic detection head DyHead in the step S2 This is achieved by: Hierarchical perception of input The weighted feature maps are indexed by scale as additional spatial information; The feature map is input into a deformable convolution layer to sample the offset position features of the target key region; The spatial attention weights are obtained by weighting and summing the sampling results and processing them through a Sigmoid activation function. The spatial attention weights are multiplied with the feature map pixel by pixel. The deformable convolution focuses on the key region of the behavior target and highlights the effective features of the occlusion region, solving the problem of occlusion interference in YOLOv11n in a dense student scene.

7. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 6, characterized in that: In the step S2, the task awareness of the dynamic detection head DyHead This is achieved by: Spatial perception The weighted feature map is globally averaged to generate a channel description vector. The vector is input into a first fully connected layer and a BatchNorm normalization layer, and then input into a second fully connected layer; The output is normalized to [0, 1] by the offset sigmoid function The interval is generated by the task weight, and the positive value channel is used for the classification task, and the negative value channel is used for the regression task. The dynamic switching of the classification and regression task channels is realized, and the dynamic switching of the classification and regression task channels solves the problem of insufficient adaptability of the YOLOv11n detection head to classification and positioning tasks.

8. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 7, characterized in that: In step S4, after the to-be-identified classroom image is input into the trained YOLOv11n-RD model, the output recognition result of the student classroom behavior includes the behavior category and the confidence, wherein the output behavior recognition confidence is a core indicator for measuring the reliability of the model's prediction of a certain behavior category, and the calculation integrates the classification task confidence and the positioning accuracy; behavior recognition confidence from the classification confidence and the localization confidence both parts are weighted to obtain: classification confidence represents the probability prediction of the model that the target belongs to a certain behavior class; classification confidence represents the matching degree of the behavior region bounding box predicted by the model and the real region; The final confidence formula is: ； wherein are weight coefficients; Furthermore, the confidence threshold is filtered and output: to reduce false positives, the model filters the output result by a confidence threshold: When the behavior recognition result is reserved, wherein is a confidence threshold value; For multi-class prediction of the same target, the class with the highest confidence is taken as the final output.

9. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 8, characterized in that, the classification confidence is calculated in particular as Classification confidence to automatically detect the task awareness module of the head DyHead The output of the classification task channel, specifically the prediction probability, is normalized by the Sigmoid activation function: ； In the formula: is the feature map processed by the space perception module; is a global average pooling operation, generating a channel description vector; , are two fully connected layers in the task perception module; is a batch normalization operation; is a Sigmoid activation function, and the output range is , indicating the probability that the target belongs to class .

10. The YOLOv11n-RD based student classroom behavior recognition algorithm according to claim 9, characterized in that, The position fix reliability The calculation of the position fix reliability is given by: Positioning confidence by intersection over union of predicted bounding box and true action region Measure, formula is: ； In the formula: is a model-predicted bounding box of a behavior region; is a real bounding box of a behavior region; The greater the value, the more accurate the positioning.

Images