A student classroom behavior recognition method based on feature aggregation and context calibration

Abstract

The application discloses a student classroom behavior recognition method based on feature aggregation and context calibration, which firstly pre-processes high-definition classroom student video data to obtain single student classroom meta-motion video, and samples key frames of the video, then extracts multi-layer features of classroom behavior by using a backbone network containing a MAN-FasterCGLU module, establishes the affinity relationship between pixels and multi-scale contexts through a context feature calibration module, and resamples and aligns cross-level features according to two-dimensional spatial offset through a spatial feature calibration module, and then uses a dynamic detection head to sequentially perform scale perception, spatial perception and task perception processing, and outputs student behavior categories and target positions. This method can enhance the expression of similar behavior subtle features, improve the spatial semantic mismatch caused by student shielding, and suppress the interference of scale change and classroom environment noise on the detection result.

Description

Technical Field

[0001] This invention relates to video understanding and analysis technology, particularly action and behavior recognition technology in video understanding, specifically a student classroom behavior recognition method based on feature aggregation and context calibration. Background Technology

[0002] Student classroom behavior recognition is a technology that uses image or video analysis to detect and determine the categories and spatial locations of student behaviors in a classroom setting. Students' learning behaviors in the classroom, such as raising their hands, reading, and writing, reflect their learning engagement. Accurately identifying these behaviors helps in generating data-driven classroom teaching analysis results.

[0003] Existing methods for student classroom behavior recognition mainly include video-based behavior recognition methods and static image-based target detection methods. Video-based methods extract spatiotemporal dynamic information using consecutive frames, but these methods have a large number of model parameters, inference time, and computational overhead, and are highly dependent on video quality. Static image-based target detection methods have high processing efficiency, but lack temporal dimension information, and still suffer from limited recognition accuracy in classroom scenarios with complex actions and changing backgrounds. Convolutional neural network-based methods are affected by fixed local receptive fields, and are insufficient in extracting subtle features of cross-regional associations and highly similar behaviors. Visual transformer-based methods can model global relationships, but their computational and memory overhead increases with input resolution, limiting their application in real-time classroom detection.

[0004] In real classroom images, students are densely seated, and the students in the front and back rows, along with their desks and chairs, can cause target occlusion. Reading and writing behaviors share high similarities in posture. The monitoring perspective creates a significant scale difference between front and back rows, and books and classroom clutter can also create background interference. During multi-layer feature fusion, repeated upsampling and downsampling can easily cause pixel-level misalignment between low-level spatial details and high-level semantic information, resulting in blurred behavioral boundaries, classification confusion, and missed detections. Static detection heads also struggle to adapt to different scales, spatial locations, and detection tasks simultaneously. Therefore, a student classroom behavior recognition method is needed that enhances fine-grained feature representation, calibrates cross-layer spatial semantics, and dynamically suppresses environmental noise while controlling computational overhead. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a student classroom behavior recognition method based on feature aggregation and context calibration. This method can improve the understanding of local spatial structure, capture global spatiotemporal feature information, and suppress the interference of scale changes and environmental noise on the detection results.

[0006] The technical solution to achieve the objective of this invention is: A method for recognizing student classroom behavior based on feature aggregation and context calibration includes the following steps: 1) Obtain images of the student's classroom behavior to be identified; 2) Input student classroom behavior images into a feature extraction backbone network. This backbone network includes, in sequence, basic convolutional units, cross-stage local aggregation units, a MANet-FasterCGLU module, a fast spatial pyramid pooling unit, and a spatial attention enhancement unit. Specifically, the basic convolutional units perform downsampling with a stride of 2 and channel expansion; the cross-stage local aggregation units enhance local semantics while preserving shallow textures; the MANet-FasterCGLU module performs high-order aggregation of similar action features in discontinuous spatial regions; the fast spatial pyramid pooling unit expands the receptive field through multi-scale max pooling; and the spatial attention enhancement unit... To enhance deep semantic regions, the feature extraction backbone network outputs at least three layers of classroom behavior feature maps at different resolutions, corresponding to shallow detail features, mid-level semantic features, and deep semantic features with strides of 8, 16, and 32, respectively. Based on the feature extraction backbone network including the MAN-FasterCGLU module, different levels of classroom behavior feature maps are generated. The MAN-FasterCGLU module is constructed based on a feature aggregation network and uses the PConv operator and CGLU channel mixer for feature processing. The MAN-FasterCGLU module includes an input compression branch, a local convolution branch, a depthwise convolution branch, a fast gated aggregation branch, and an output fusion branch. The input features... First, intermediate features are obtained through a 1×1 convolution. part Locally faithful features are formed through 1×1 convolution. Another part is processed by 1×1 convolution and k×k depthwise convolution to form local enhanced features. At the same time Divided into two sub-features along the channel dimension and ,in As a bypass feature, it directly participates in subsequent splicing and fusion. Given n FasterCGLU sub-cells, each FasterCGLU sub-cell is processed using PConv spatial extraction, CGLU channel gating, and residual summation to obtain... to Finally , The output features of each FasterCGLU level are concatenated along the channel dimension and compressed using a 1×1 convolution to obtain the output features. The PConv operator performs 3×3 convolution only on some channels, while maintaining the identity mapping for the remaining channels; the CGLU channel mixer completes channel-level selection through linear mapping, depthwise separable convolution, gated activation, and pointwise convolution, thereby reducing computational redundancy and enhancing subtle classroom action features. The calculation method of the MAN-FasterCGLU module is shown in formula (1): (1), In formula (1), The number of channels is ,and The number of channels for each feature is 1 The semantic information of the three types of features is fused through a concatenation operation, and then... The convolutional layer is compressed to generate a channel number of Output characteristics As shown in formula (2): (2), By employing the partial convolution PConv operator to select some channels for spatial feature extraction while maintaining the identity mapping of the remaining channels, memory access redundancy and model computation can be reduced, effectively improving feature extraction efficiency. Convolutionally gated linear units (CGLU) are used as channel mixers, and embedded lightweight depthwise separable convolutions are used to enable the model to achieve pixel-level channel filtering by using local feature-guided attention mechanisms when performing channel-level feature modulation, thereby enhancing the model's response strength to subtle motion features. 3) Feature Fusion: Classroom behavior feature maps from different levels are input into the feature fusion neck network. This neck network employs a bidirectional path aggregation structure, both top-down and bottom-up. First, the deep feature P5 output from the backbone network is upsampled and concatenated with the mid-level feature P4, then fused using the MANet-FasterCGLU module to obtain the mid-level fused feature. This mid-level fused feature is then upsampled again and concatenated with the shallow feature P3 to obtain the shallow fused feature. Subsequently, the shallow fused feature is fed back to the mid-level and deep layers via stride convolution or downsampling operations, and then concatenated with the corresponding level features to form enhanced multi-scale features. At the end of the neck network, a context-spatial feature calibration network (CSFCN) is set. The CSFCN includes a context feature calibration module (CFC) and a spatial feature calibration module (SFC) connected sequentially or in parallel. These modules perform context recalibration and spatial resampling on the fused three-layer features, outputting... , , The Contextual Feature Calibration (CFC) module calculates the affinity between pixels and multi-scale context for input features. The Context Feature Calibration (CFC) module first obtains low-dimensional query features through 1×1 convolution. ,in Less than Simultaneously, context key features are generated by performing pooling and convolution processes on the input features at different scales. and context value features ,Will Expand as The pixel query matrix will Expand as The context key matrix, where , To determine the number of multi-scale context locations, through and The matrix multiplication is performed and then normalized using Softmax to obtain the pixel-context affinity matrix. reuse right Weighted summation yields the context aggregation feature corresponding to each pixel, which is then fused with the original input feature residual to output the context calibration feature. Based on the spatial feature calibration module (SFC), cross-level features are resampled and spatially aligned using a learnable 2D spatial offset to obtain the calibrated multi-scale fused feature. The SFC receives low-level features. Characteristics of high-level personnel First, the high-level characteristics Adjust to low-level features The two are concatenated at the same spatial scale along the channel dimension; then a two-dimensional spatial offset is generated through a lightweight convolutional offset prediction branch. and modulation weights For the target location The Spatial Feature Calibration Module (SFC) is in accordance with Bilinear interpolation resampling or deformable sampling is performed on high-level semantic features to obtain semantic features aligned with low-level spatial details. Finally, the resampled high-level features are fused with the low-level features to obtain spatially calibrated multi-scale features. This process can alleviate boundary misalignment caused by occlusion, top-down view, and scale changes in classroom scenes. Specifically: First, regarding input features ,use Convolutional layers generate low-dimensional features ,in much smaller Simultaneously, multi-scale contextual features are extracted and contextual representations are generated, i.e. and Secondly, Reshaping and transposing and in and Matrix multiplication is performed between the pixels, and finally, a pixel-context affinity matrix is ​​generated through a Softmax layer. As shown in formula (3): (3), The Context Feature Calibration (CFC) module uses pooling operations to construct a multi-scale context feature library. By calculating the similarity matrix between pixels and context features, it adaptively aggregates the optimal semantic dependencies for each pixel to solve the classification bias problem. The definition of the Context Feature Calibration (CFC) module is shown in formula (4): (4), In formula (4), These represent the input, output, recalibration factor, and contextual features, respectively. The range of values ​​is , Represented as a pairwise function used to calculate the affinity between features, the Spatial Feature Calibration (SFC) module uses a lightweight network to predict the 2D spatial offset of the feature map during the cross-layer feature fusion stage, and resamples the low-resolution high-level semantic features based on the offset to align features at different levels in space, thereby enhancing the model's accuracy in defining the edges of students' limbs. 4) Detector localization and classification: Multi-scale fusion features calibrated by the Contextual Feature Calibration (CFC) module and the Spatial Feature Calibration (SFC) module. Input the dynamic detection header DyHead, DyHead first checks... The DyHead algorithm performs sequential attention enhancements for scale perception, spatial perception, and task perception. The enhanced features are then input into the target category prediction branch and bounding box regression branch, respectively, outputting the student classroom behavior category, target location, and confidence score. Based on the category and location prediction results, it outputs the student classroom behavior recognition result. DyHead includes scale perception attention modules, spatial perception attention modules, and task perception attention modules, unifying the three-layer calibration features into a feature tensor. ,in Indicates the number of feature levels. Represents the set of spatial locations at each level. The scale-aware attention module calculates the importance of different feature levels along the L dimension, representing the number of channels; the spatial-aware attention module calculates the importance of different feature levels along the L dimension. Dimensionality utilizes sparse sampling and deformable convolution to focus on discriminative spatial regions; the task-aware attention module along... The feature channels are dynamically enabled or disabled to adapt to classification and bounding box regression tasks. After processing by the above three modules, DyHead outputs the category prediction results, bounding box regression results, and confidence prediction results for student classroom behavior targets. For the feature tensor... The self-attention mechanism is expressed as shown in formula (5): (5), In formula (5), The attention function is represented by DyHead, which transforms the feature tensor into three sequentially executed attention operations, each focusing on a specific dimension as shown in Equation (6): (6), In formula (6), , and These represent the actions applied to the dimensions. , and Three different attention functions, scale-aware attention Features at different scales are dynamically fused based on the semantic importance of the feature tensor, as shown in formula (7): (7), In formula (7), Indicates a by A linear function approximated by a convolutional layer. The activation function is shown in formula (8): (8), Spatial perception and attention Used to focus on discriminative regions that coexist consistently across different spatial locations and feature levels, because Due to the high dimensionality of DyHead, attention learning is first made sparse through deformable convolution, and then features are aggregated across levels at the same spatial location, as shown in formula (9): (9), In formula (9), Indicates the number of sparse sampling locations; Indicates spatial offset through self-learning The moved position is used to focus on the discriminative region; Indicates the location The importance scalar learned at the midpoint of the input features, both are derived from the median level of the input features. The learning process revealed that attention is required during the final task deployment. The feature channels are dynamically turned on or off, as shown in formula (10): (10) As shown in formula (10), and This represents the learnable parameters related to the task. Indicates the first Feature slices of each channel Used to learn and control activation thresholds First of all Global average pooling is performed to reduce dimensionality, followed by two fully connected layers and a normalization layer. Finally, a shifted sigmoid function is applied to normalize the output to the interval. .

[0007] This technical solution is applied to student classroom behavior recognition in offline smart classrooms and has the following characteristics: 1) The MAN-FasterCGLU module establishes high-order feature associations across levels and spatial locations, and reduces redundant computations through the PConv operator; 2) The CGLU channel mixer guides channel-level gating filtering with local features, enhancing the response to lateral reading and small-amplitude handwriting behaviors; 3) The CFC module aggregates semantic information based on the affinity between pixels and multi-scale context, reducing classification bias caused by occlusion and complex backgrounds; 4) The SFC module improves the spatial alignment of student limb edges and behavioral regions by resampling high-level semantic features based on two-dimensional spatial offset. 5) DyHead unifies scale perception, spatial perception, and task perception, dynamically adjusts feature channels, and filters classroom environmental noise.

[0008] This method can improve the understanding of local spatial structure, capture global spatiotemporal feature information, and improve the accuracy of recognizing students' classroom learning behaviors. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the method flow in the embodiment; Figure 2 This is a schematic diagram of the MAN-FasterCGLU module in the embodiment; Figure 3 This is a schematic diagram of the context feature calibration module in the embodiment; Figure 4 This is a schematic diagram of the spatial feature calibration module in the embodiment; Figure 5 This is a schematic diagram of the dynamic detection head in the embodiment. Detailed Implementation

[0010] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this is not intended to limit the scope of the invention.

[0011] Example: Reference Figure 1A student classroom behavior recognition method based on feature aggregation and context calibration includes the following steps: 1) Obtain images of the student's classroom behavior to be identified; 2) Input student classroom behavior images into a feature extraction backbone network. This backbone network includes, in sequence, basic convolutional units, cross-stage local aggregation units, a MANet-FasterCGLU module, a fast spatial pyramid pooling unit, and a spatial attention enhancement unit. The basic convolutional units perform downsampling and channel expansion with a stride of 2; the cross-stage local aggregation units enhance local semantics while preserving shallow textures; the MANet-FasterCGLU module performs high-order aggregation of similar action features in discontinuous spatial regions; the fast spatial pyramid pooling unit expands the receptive field through multi-scale max pooling; and the spatial attention enhancement unit strengthens deep semantic regions. The feature extraction backbone network outputs at least three layers of classroom behavior feature maps at different resolutions, corresponding to shallow detail features, mid-level semantic features, and deep semantic features with strides of 8, 16, and 32, respectively. Different levels of classroom behavior feature maps are generated based on the feature extraction backbone network containing the MANet-FasterCGLU module. The MANet-FasterCGLU module is constructed based on the feature aggregation network and uses the PConv operator and CGLU channel mixer for feature processing, such as... Figure 2 As shown, the MANet-FasterCGLU module includes an input compression branch, a local convolution branch, a depthwise convolution branch, a fast gated aggregation branch, and an output fusion branch. The input features... First, intermediate features are obtained through a 1×1 convolution. part Locally faithful features are formed through 1×1 convolution. Another part is processed by 1×1 convolution and k×k depthwise convolution to form local enhanced features. At the same time Divided into two sub-features along the channel dimension and ,in As a bypass feature, it directly participates in subsequent splicing and fusion. Given n FasterCGLU sub-cells, each FasterCGLU sub-cell is processed using PConv spatial extraction, CGLU channel gating, and residual summation to obtain... to Finally , The output features of each FasterCGLU level are concatenated along the channel dimension and compressed using a 1×1 convolution to obtain the output features. The PConv operator performs 3×3 convolution only on some channels, while maintaining the identity mapping for the remaining channels; the CGLU channel mixer completes channel-level selection through linear mapping, depthwise separable convolution, gated activation, and pointwise convolution, thereby reducing computational redundancy and enhancing subtle classroom action features. The calculation method of the MAN-FasterCGLU module is shown in formula (1): (1), In formula (1), The number of channels is ,and The number of channels for each feature is 1 The semantic information of the three types of features is fused through a concatenation operation, and then... The convolutional layer is compressed to generate a channel number of Output characteristics As shown in formula (2): (2), By employing the partial convolution PConv operator to select some channels for spatial feature extraction while maintaining the identity mapping of the remaining channels, memory access redundancy and model computation can be reduced, effectively improving feature extraction efficiency. Convolutionally gated linear units (CGLU) are used as channel mixers, and embedded lightweight depthwise separable convolutions are used to enable the model to achieve pixel-level channel filtering by using local feature-guided attention mechanisms when performing channel-level feature modulation, thereby enhancing the model's response strength to subtle motion features. 3) Feature Fusion: Classroom behavior feature maps from different levels are input into the feature fusion neck network. This neck network employs a bidirectional path aggregation structure, both top-down and bottom-up. First, the deep feature P5 output from the backbone network is upsampled and concatenated with the mid-level feature P4, then fused using the MANet-FasterCGLU module to obtain the mid-level fused feature. This mid-level fused feature is then further upsampled and concatenated with the shallow feature P3 to obtain the shallow fused feature. Subsequently, the shallow fused feature is fed back to the mid-level and deep layers through stride convolution or downsampling operations, and then concatenated with the corresponding level features to form enhanced multi-scale features. A context-spatial feature calibration network, denoted as CSFCN, is set at the end of the neck network. Figure 3 , Figure 4 As shown, CSFCN includes a context feature calibration module (CFC) and a spatial feature calibration module (SFC) connected sequentially or in parallel. These modules perform context recalibration and spatial resampling on the fused three-layer features, outputting... , , The Contextual Feature Calibration (CFC) module calculates the affinity between pixels and multi-scale context for input features. The Context Feature Calibration (CFC) module first obtains low-dimensional query features through 1×1 convolution. ,in Less than Simultaneously, context key features are generated by performing pooling and convolution processes on the input features at different scales. and context value features ,Will Expand as The pixel query matrix will Expand as The context key matrix, where , To determine the number of multi-scale context locations, through and The matrix multiplication is performed and then normalized using Softmax to obtain the pixel-context affinity matrix. reuse right Weighted summation yields the context aggregation feature corresponding to each pixel, which is then fused with the original input feature residual to output the context calibration feature. Based on the spatial feature calibration module (SFC), cross-level features are resampled and spatially aligned using a learnable 2D spatial offset to obtain the calibrated multi-scale fused feature. The SFC receives low-level features. Characteristics of high-level personnel First, the high-level characteristics Adjust to low-level features The two are concatenated at the same spatial scale along the channel dimension; then a two-dimensional spatial offset is generated through a lightweight convolutional offset prediction branch. and modulation weights For the target location The Spatial Feature Calibration Module (SFC) is in accordance with Bilinear interpolation resampling or deformable sampling is performed on high-level semantic features to obtain semantic features aligned with low-level spatial details. Finally, the resampled high-level features are fused with the low-level features to obtain spatially calibrated multi-scale features. This process can alleviate boundary misalignment caused by occlusion, top-down view, and scale changes in classroom scenes. Specifically: First, regarding input features ,use Convolutional layers generate low-dimensional features ,in much smaller Simultaneously, multi-scale contextual features are extracted and contextual representations are generated, i.e. and Secondly, Reshaping and transposing and in and Matrix multiplication is performed between the pixels, and finally, a pixel-context affinity matrix is ​​generated through a Softmax layer. As shown in formula (3): (3), The Context Feature Calibration (CFC) module uses pooling operations to construct a multi-scale context feature library. By calculating the similarity matrix between pixels and context features, it adaptively aggregates the optimal semantic dependencies for each pixel to solve the classification bias problem. The definition of the Context Feature Calibration (CFC) module is shown in formula (4): (4), In formula (4), These represent the input, output, recalibration factor, and contextual features, respectively. The range of values ​​is , Represented as a pairwise function used to calculate the affinity between features, the Spatial Feature Calibration (SFC) module uses a lightweight network to predict the 2D spatial offset of the feature map during the cross-layer feature fusion stage, and resamples the low-resolution high-level semantic features based on the offset to align features at different levels in space, thereby enhancing the model's accuracy in defining the edges of students' limbs. 4) Detector localization and classification: Multi-scale fusion features calibrated by the Contextual Feature Calibration (CFC) module and the Spatial Feature Calibration (SFC) module. Input the dynamic detection header DyHead, the dynamic detection header DyHead is as follows Figure 5 As shown, DyHead first... The DyHead algorithm performs sequential attention enhancements for scale perception, spatial perception, and task perception. The enhanced features are then input into the target category prediction branch and bounding box regression branch, respectively, outputting the student classroom behavior category, target location, and confidence score. Based on the category and location prediction results, it outputs the student classroom behavior recognition result. DyHead includes scale perception attention modules, spatial perception attention modules, and task perception attention modules, unifying the three-layer calibration features into a feature tensor. ,in Indicates the number of feature levels. Represents the set of spatial locations at each level. The scale-aware attention module calculates the importance of different feature levels along the L dimension, representing the number of channels; the spatial-aware attention module calculates the importance of different feature levels along the L dimension. Dimensionality utilizes sparse sampling and deformable convolution to focus on discriminative spatial regions; the task-aware attention module along... The feature channels are dynamically enabled or disabled to adapt to classification and bounding box regression tasks. After processing by the above three modules, DyHead outputs the category prediction results, bounding box regression results, and confidence prediction results for student classroom behavior targets. For the feature tensor... The self-attention mechanism is expressed as shown in formula (5): (5), In formula (5), The attention function is represented by DyHead, which transforms the feature tensor into three sequentially executed attention operations, each focusing on a specific dimension as shown in Equation (6): (6), In formula (6), , and These represent the actions applied to the dimensions. , and Three different attention functions, scale-aware attention Features at different scales are dynamically fused based on the semantic importance of the feature tensor, as shown in formula (7): (7), In formula (7), Indicates a by A linear function approximated by a convolutional layer. The activation function is shown in formula (8): (8), Spatial perception and attention Used to focus on discriminative regions that coexist consistently across different spatial locations and feature levels, because Due to the high dimensionality of DyHead, attention learning is first made sparse through deformable convolution, and then features are aggregated across levels at the same spatial location, as shown in formula (9): (9), In formula (9), Indicates the number of sparse sampling locations; Indicates spatial offset through self-learning The moved position is used to focus on the discriminative region; Indicates the location The importance scalar learned at the midpoint of the input features, both are derived from the median level of the input features. In order to achieve joint learning and generalize representations of different objects, task-aware attention is deployed at the end. The feature channels are dynamically turned on or off, as shown in formula (10): (10) As shown in formula (10), and This represents the learnable parameters related to the task. Indicates the first Feature slices of each channel Used to learn and control activation thresholds First of all Global average pooling is performed to reduce dimensionality, followed by two fully connected layers and a normalization layer. Finally, a shifted sigmoid function is applied to normalize the output to the interval. .

Claims

1. A method for recognizing student classroom behavior based on feature aggregation and context calibration, characterized in that, Includes the following steps: 1) Obtain images of the student's classroom behavior to be identified; 2) Input student classroom behavior images into a feature extraction backbone network. This backbone network includes, in sequence, basic convolutional units, cross-stage local aggregation units, a MANet-FasterCGLU module, a fast spatial pyramid pooling unit, and a spatial attention enhancement unit. Specifically, the basic convolutional units perform downsampling with a stride of 2 and channel expansion; the cross-stage local aggregation units enhance local semantics while preserving shallow textures; the MANet-FasterCGLU module performs high-order aggregation of similar action features in discontinuous spatial regions; the fast spatial pyramid pooling unit expands the receptive field through multi-scale max pooling; and the spatial attention enhancement unit... To enhance deep semantic regions, the feature extraction backbone network outputs at least three layers of classroom behavior feature maps at different resolutions, corresponding to shallow detail features, mid-level semantic features, and deep semantic features with strides of 8, 16, and 32, respectively. Based on the feature extraction backbone network including the MAN-FasterCGLU module, different levels of classroom behavior feature maps are generated. The MAN-FasterCGLU module is constructed based on a feature aggregation network and uses the PConv operator and CGLU channel mixer for feature processing. The MAN-FasterCGLU module includes an input compression branch, a local convolution branch, a depthwise convolution branch, a fast gated aggregation branch, and an output fusion branch. The input features... First, intermediate features are obtained through a 1×1 convolution. part Locally faithful features are formed through 1×1 convolution. Another part is processed by 1×1 convolution and k×k depthwise convolution to form local enhanced features. At the same time Divided into two sub-features along the channel dimension and ,in As a bypass feature, it directly participates in subsequent splicing and fusion. Given n FasterCGLU sub-cells, each FasterCGLU sub-cell is processed using PConv spatial extraction, CGLU channel gating, and residual summation to obtain... to Finally , The output features of each FasterCGLU level are concatenated along the channel dimension and compressed using a 1×1 convolution to obtain the output features. The PConv operator performs 3×3 convolution only on some channels, while maintaining the identity mapping for the remaining channels; the CGLU channel mixer completes channel-level selection through linear mapping, depthwise separable convolution, gated activation, and pointwise convolution, thereby reducing computational redundancy and enhancing subtle classroom action features. The calculation method of the MAN-FasterCGLU module is shown in formula (1): （1）， In formula (1), The number of channels is ,and The number of channels for each feature is 1 The semantic information of the three types of features is fused through a concatenation operation, and then... The convolutional layer is compressed to generate a channel number of Output characteristics As shown in formula (2): （2）； 3) Feature Fusion: Classroom behavior feature maps from different levels are input into the feature fusion neck network. This neck network employs a bidirectional path aggregation structure, both top-down and bottom-up. First, the deep feature P5 output from the backbone network is upsampled and concatenated with the mid-level feature P4, then fused using the MANet-FasterCGLU module to obtain the mid-level fused feature. This mid-level fused feature is then upsampled again and concatenated with the shallow feature P3 to obtain the shallow fused feature. Subsequently, the shallow fused feature is fed back to the mid-level and deep layers via stride convolution or downsampling operations, and then concatenated with the corresponding level features to form enhanced multi-scale features. At the end of the neck network, a context-spatial feature calibration network (CSFCN) is set. The CSFCN includes a context feature calibration module (CFC) and a spatial feature calibration module (SFC) connected sequentially or in parallel. These modules perform context recalibration and spatial resampling on the fused three-layer features, outputting... , , The Contextual Feature Calibration (CFC) module calculates the affinity between pixels and multi-scale context for input features. The Context Feature Calibration (CFC) module first obtains low-dimensional query features through 1×1 convolution. ,in Less than Simultaneously, context key features are generated by performing pooling and convolution processes on the input features at different scales. and context value features ,Will Expand as The pixel query matrix will Expand as The context key matrix, where , To determine the number of multi-scale context locations, through and The matrix multiplication is performed and then normalized using Softmax to obtain the pixel-context affinity matrix. reuse right Weighted summation yields the context aggregation feature corresponding to each pixel, which is then fused with the original input feature residual to output the context calibration feature. Based on the spatial feature calibration module (SFC), cross-level features are resampled and spatially aligned using a learnable 2D spatial offset to obtain the calibrated multi-scale fused feature. The SFC receives low-level features. Characteristics of high-level personnel First, the high-level characteristics Adjust to low-level features The two are concatenated at the same spatial scale along the channel dimension; then a two-dimensional spatial offset is generated through a lightweight convolutional offset prediction branch. and modulation weights For the target location The Spatial Feature Calibration Module (SFC) is in accordance with Bilinear interpolation resampling or deformable sampling is performed on high-level semantic features to obtain semantic features aligned with low-level spatial details. Finally, the resampled high-level features are fused with the low-level features to obtain spatially calibrated multi-scale features. This process can alleviate boundary misalignment caused by occlusion, top-down view, and scale changes in classroom scenes. Specifically: First, regarding input features ,use Convolutional layers generate low-dimensional features ,in much smaller Simultaneously, multi-scale contextual features are extracted and contextual representations are generated, i.e. and Secondly, Reshaping and transposing and in and Matrix multiplication is performed between the pixels, and finally, a pixel-context affinity matrix is ​​generated through a Softmax layer. As shown in formula (3): （3）， The Context Feature Calibration (CFC) module uses pooling operations to construct a multi-scale context feature library. By calculating the similarity matrix between pixels and context features, it adaptively aggregates the optimal semantic dependencies for each pixel to solve the classification bias problem. The definition of the Context Feature Calibration (CFC) module is shown in formula (4): （4）， In formula (4), These represent the input, output, recalibration factor, and contextual features, respectively. The range of values ​​is , Represented as a pairwise function used to calculate the affinity between features, the Spatial Feature Calibration (SFC) module uses a lightweight network to predict the 2D spatial offset of the feature map during the cross-layer feature fusion stage, and resamples the low-resolution high-level semantic features based on the offset to align features at different levels in space, thereby enhancing the model's accuracy in defining the edges of students' limbs. 4) Detector localization and classification: Multi-scale fusion features calibrated by the Contextual Feature Calibration (CFC) module and the Spatial Feature Calibration (SFC) module. Input the dynamic detection header DyHead, DyHead first checks... The DyHead algorithm performs sequential attention enhancements for scale perception, spatial perception, and task perception. The enhanced features are then input into the target category prediction branch and bounding box regression branch, respectively, outputting the student classroom behavior category, target location, and confidence score. Based on the category and location prediction results, it outputs the student classroom behavior recognition result. DyHead includes scale perception attention modules, spatial perception attention modules, and task perception attention modules, unifying the three-layer calibration features into a feature tensor. ,in Indicates the number of feature levels. Represents the set of spatial locations at each level. The scale-aware attention module calculates the importance of different feature levels along the L dimension, representing the number of channels; the spatial-aware attention module calculates the importance of different feature levels along the L dimension. Dimensionality utilizes sparse sampling and deformable convolution to focus on discriminative spatial regions; the task-aware attention module along... The feature channels are dynamically enabled or disabled to adapt to classification and bounding box regression tasks. After processing by the above three modules, DyHead outputs the category prediction results, bounding box regression results, and confidence prediction results for student classroom behavior targets. For the feature tensor... The self-attention mechanism is expressed as shown in formula (5): （5）， In formula (5), The attention function is represented by DyHead, which transforms the feature tensor into three sequentially executed attention operations, each focusing on a specific dimension as shown in Equation (6): （6）， In formula (6), , and These represent the actions applied to the dimensions. , and Three different attention functions, scale-aware attention Features at different scales are dynamically fused based on the semantic importance of the feature tensor, as shown in formula (7): （7）， In formula (7), Indicates a by A linear function approximated by a convolutional layer. The activation function is shown in formula (8): （8）， Spatial perception and attention Used to focus on discriminative regions that coexist consistently across different spatial locations and feature levels, because Due to the high dimensionality of DyHead, attention learning is first made sparse through deformable convolution, and then features are aggregated across levels at the same spatial location, as shown in formula (9): （9）， In formula (9), Indicates the number of sparse sampling locations; Indicates spatial offset through self-learning The moved position is used to focus on the discriminative region; Indicates the location The importance scalar learned at the midpoint of the input features, both are derived from the median level of the input features. The learning process revealed that attention is required during the final task deployment. The feature channels are dynamically turned on or off, as shown in formula (10): （10）， As shown in formula (10), and This represents the learnable parameters related to the task. Indicates the first Feature slices of each channel Used to learn and control activation thresholds First of all Global average pooling is performed to reduce dimensionality, followed by two fully connected layers and a normalization layer. Finally, a shifted sigmoid function is applied to normalize the output to the interval. .

Images