Classroom scene target detection method based on multi-scale feature enhancement and semantic alignment

Abstract

This invention discloses a classroom scene object detection method based on multi-scale feature enhancement and semantic alignment, implemented according to the following steps: Step 1, obtain a classroom scene object detection dataset; Step 2, improve the YOLOv11 one-stage object detection network to obtain a student classroom scene object detection model; Step 3, train the student classroom scene object detection model using the classroom scene object detection dataset obtained in Step 1 to obtain a trained student classroom scene object detection model; Step 4, input the student classroom scene image to be detected into the trained student classroom scene object detection model for detection. This invention solves the problem in existing technologies where insufficient feature extraction and feature fusion mechanisms make it difficult to meet the accuracy requirements of actual classroom scene object detection.

Description

Technical Field

[0001] This invention belongs to the technical field of classroom scene target detection methods, and relates to a classroom scene target detection method based on multi-scale feature enhancement and semantic alignment. Background Technology

[0002] With the development of smart education, the demand for intelligent analysis and evaluation of classroom teaching processes is increasing. Classroom scene target detection, as a fundamental step, aims to automatically identify and locate various student behaviors in the classroom, such as listening, raising hands, reading, or writing. Traditional methods typically use general target detection models, such as the YOLO series or other one-stage detection models, but these methods have significant limitations in classroom scenarios. In classroom environments, students are densely distributed, with severe occlusion; small targets, such as those in the back rows, are easily lost due to missing details; and different behaviors have subtle differences in appearance, leading to false positives or false negatives in densely packed environments. Furthermore, the complex and variable lighting conditions in classrooms also affect the robustness of the models. Existing general models are not optimized for these specific classroom problems; their feature extraction, feature fusion, and attention mechanism design are insufficient, making it difficult to meet the accuracy and stability requirements of target detection in real-world classroom scenarios. Therefore, there is an urgent need for a dedicated behavior detection method optimized for small targets, dense occlusion, and semantic similarity in classroom scenarios to improve detection performance and adapt to complex teaching environments. Summary of the Invention

[0003] The purpose of this invention is to provide a classroom scene target detection method based on multi-scale feature enhancement and semantic alignment, which solves the problem that existing technologies have insufficient feature extraction and feature fusion mechanism design, making it difficult to meet the accuracy requirements of actual classroom scene target detection.

[0004] The technical solution adopted in this invention is a classroom scene target detection method based on multi-scale feature enhancement and semantic alignment, which is implemented according to the following steps:

[0005] Step 1: Obtain the classroom scene target detection dataset; Step 2: Improve the YOLOv11 one-stage object detection network to obtain the object detection model for student classroom scenes; Step 3: Use the classroom scene object detection dataset obtained in Step 1 to train the student classroom scene object detection model to obtain the trained student classroom scene object detection model; Step 4: Input the student classroom scene image to be detected into the trained student classroom scene target detection model for detection.

[0006] Preferably, the classroom scene target detection dataset in step 1 uses the publicly available classroom scene target detection dataset SCB.

[0007] Preferably, the improvement of the YOLOv11 one-stage object detection network in step 2 is specifically as follows: The Fast Spatial Pyramid Pooling (SPPF) module at layer 10 of the YOLOv11 one-stage object detection network backbone is replaced with a multi-scale feature enhancement module. The output of the C3k2 module at layer 9 of the backbone network is used as the input of the multi-scale feature enhancement module, and the output of the multi-scale feature enhancement module is used as the input of the C2PSA module at layer 11 of the backbone network. Two semantic feature alignment modules are set in the top-down feature fusion path of the neck network in the YOLOv11 one-stage object detection network. The first semantic feature alignment module is set between the 13th layer concatenation layer and the 14th layer C3k2 module of the neck network. The output of the 11th layer C2PSA module of the backbone network and the output of the 13th layer concatenation layer of the neck network are used as the input of the first semantic feature alignment module, and the output of the first semantic feature alignment module is used as the input of the 14th layer C3k2 module of the neck network. The second semantic feature alignment module is set between the 16th layer concatenation layer and the 17th layer C3k2 module of the neck network. The output of the 14th layer C3k2 module and the output of the 16th layer concatenation layer of the neck network are used as the input of the second semantic feature alignment module, and the output of the second semantic feature alignment module is used as the input of the 17th layer C3k2 module of the neck network. After the C3k2 module in layer 17, layer 20, and layer 23 of the neck network, a joint attention module is set up. The three scale features are reweighted by the corresponding joint attention module and then input into the detection layer.

[0008] Preferably, the multi-scale feature enhancement module includes a first CBS module, a deep excitation module, a multi-scale pooling and recombination module, a residual connection unit, and a second CBS module connected in sequence. The output features of the C3k2 module at layer 9 of the backbone network are used as the input features of the multi-scale feature enhancement module. The input features are processed sequentially through the first CBS module, the deep activation module, and the multi-scale pooling and recombination module. At the same time, the output features of the C3k2 module at layer 9 of the backbone network are subjected to residual mapping. The residual mapping result and the output of the scale pooling and recombination module are then added element-wise in the residual connection unit. After being processed by the second CBS module, the output is used as the input of the C2PSA module at layer 11 of the backbone network. When performing residual mapping on the output features of the C3k2 module at layer 9 of the backbone network, which is also the input features of the multi-scale feature enhancement module, the residual mapping result is obtained through 1×1 convolution when the number of input channels is inconsistent with the number of output channels; and when the number of input channels is consistent with the number of output channels, the residual mapping result is obtained through identity mapping.

[0009] Preferably, the deep activation module includes a deep convolutional layer, a pointwise convolutional layer, a global average pooling layer, a multilayer perceptron, and a sigmoid activation layer connected in sequence. The output of the first CBS module is used as the input of the deep activation module. That is, the output of the first CBS module is sequentially fed into the deep convolutional layer for channel-independent spatial convolution operations, followed by pointwise convolutional layers to complete inter-channel feature mixing, then a global average pooling layer to extract channel description vectors, and then a multilayer perceptron and a sigmoid activation layer to generate channel weights. The channel weights are then multiplied elementwise with the output features of the pointwise convolutional layer, and the result of the multiplication is used as the output of the deep activation module and input into the multi-scale pooling and recombination module. The multi-scale pooling recombination module includes multiple adaptive average pooling layers with different output sizes. Each adaptive average pooling layer is connected to a flattening layer. Multiple flattening layers are connected to a stitching layer. The stitching layer is connected to a recombination layer and an upsampling layer in sequence. The upsampling result is used as the output feature of the scale pooling recombination module. The output features of the deep excitation module are input into multiple adaptive average pooling layers with different output sizes to obtain pooling features of multiple scales. The pooling features of multiple scales are flattened by the corresponding flattening layers and then input into the stitching layer. They are stitched together in the spatial dimension corresponding to the channel. The stitching result is reassembled into a two-dimensional spatial structure in the reconstruction layer. When the spatial size of the reassembled feature is inconsistent with the spatial size of the input feature, it is restored to the same spatial resolution as the input feature by upsampling, thus obtaining the output features of the multi-scale pooling reconstruction module.

[0010] Preferably, the semantic feature alignment module includes a high-resolution feature projection convolutional layer and a low-resolution feature projection convolutional layer arranged in parallel. An upsampling layer is connected after the low-resolution feature projection convolutional layer. The output of the upsampling layer and the output of the high-resolution feature projection convolutional layer are jointly connected to a first stitching layer. The first stitching layer is sequentially connected to a joint semantic-spatial alignment submodule, a residual fusion alignment unit, a second stitching layer, a cross-scale semantic interaction submodule, a splitting layer, and a first output projection convolutional layer. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the first semantic feature alignment module are the high-resolution features output from the 13th layer of the neck network splicing layer and the low-resolution features output from the 11th layer of the backbone network C2PSA module, respectively. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the second semantic feature alignment module are the high-resolution features output from the 16th layer of the neck network splicing layer and the low-resolution features output from the 14th layer of the neck network C3k2 module, respectively.

[0011] Preferably, the joint semantic-spatial alignment submodule includes a semantic alignment module and a spatial alignment module set in parallel, and the outputs of the semantic alignment module and the spatial alignment module are both used as inputs to the residual fusion alignment unit in the semantic feature alignment module; The semantic alignment module includes a global semantic branch and a local semantic branch set in parallel. The outputs of the global semantic branch and the local semantic branch are connected to a third concatenation layer. The third concatenation layer is also connected to a semantic fusion layer and a weight generation layer in sequence. The inputs of the global semantic branch and the local semantic branch are features concatenated by the first concatenation layer. The global semantic branch extracts global semantic information from the input features through global average pooling and convolutional mapping, while the local semantic branch extracts local semantic information from the input features through local convolutional mapping. The global semantic information and the local semantic information are concatenated by the third concatenation layer, processed by the semantic fusion layer, and then the semantic weight generation layer generates semantic weights. The spatial alignment module includes a multi-scale spatial convolution branch, a fourth stitching layer, and a spatial fusion layer connected in sequence. The spatial fusion layer is connected in parallel with a high-resolution spatial weight generation layer and a low-resolution spatial weight generation layer. The input of the spatial alignment module is the low-resolution features processed by the low-resolution feature projection convolution layer and the upsampling layer, and the high-resolution features processed by the high-resolution feature projection convolution layer. The multi-scale spatial convolution branch uses 1×1 convolution, 3×3 convolution and 5×5 convolution to spatially model the high-resolution features and low-resolution features, respectively. The resulting multi-scale spatial features are stitched by the fourth stitching layer and processed by the spatial fusion layer. The high-resolution spatial weight and low-resolution spatial weight are generated by the high-resolution spatial weight generation layer and the low-resolution spatial weight generation layer, respectively. The residual fusion alignment unit performs residual weighted alignment processing on the high-resolution features after high-resolution feature projection convolution layer processing and the low-resolution features after low-resolution feature projection convolution layer and upsampling layer processing, respectively, based on the semantic weights, high-resolution spatial weights and low-resolution spatial weights output by the joint semantic-spatial alignment submodule, to obtain high-resolution aligned features and low-resolution aligned features. The cross-scale semantic interaction submodule includes a second multi-scale feature extraction branch, a fifth splicing layer, a scale fusion layer, a QKV projection unit, a multi-head attention unit, an attention projection unit, a first residual enhancement unit, a feature refinement unit, a second residual enhancement unit, and a second output projection convolutional layer, which are connected in sequence. High-resolution alignment features and low-resolution alignment features are concatenated by the fifth concatenation layer and then input into the second multi-scale feature extraction branch. The second multi-scale feature extraction branch uses 1×1 convolution, 3×3 convolution and 5×5 convolution to extract cross-scale multi-scale features, which are then fused by the scale fusion layer and fed into the QKV projection unit to generate query features, key features and value features. Subsequently, cross-scale semantic interaction is completed through the multi-head attention unit and attention projection unit. The output results of the attention projection unit and the scale fusion layer are processed by the first residual enhancement unit and the feature refinement unit. The output results of the feature refinement unit and the first residual enhancement unit are then input into the second residual enhancement unit for processing. Finally, the result of the second residual enhancement unit is processed by the second output projection convolution layer and output. The output of the cross-scale semantic interaction submodule is split into high-resolution interaction features and low-resolution interaction features by a splitting layer. A summation layer is also provided between the splitting layer and the first output projection convolutional layer. The summation layer sums the high-resolution projection features, high-resolution alignment features, low-resolution alignment features, high-resolution interaction features and low-resolution interaction features output by the high-resolution feature projection convolutional layer. The summation result is processed by the output projection convolutional layer and used as the output of the semantic feature alignment module.

[0012] Preferably, the joint attention module includes a projective convolutional layer connected in sequence. The projective convolutional layer is connected in parallel with a spatial attention submodule, a channel attention submodule, and a position attention submodule. The output features of the spatial attention submodule, the channel attention submodule, and the position attention submodule are concatenated by a concatenation layer, and then sequentially passed through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer. The output of the Sigmoid activation layer and the concatenated features are then multiplied element-wise and passed through an output projective convolutional layer to obtain the output features. The input features of the joint attention module are first processed through a projection convolutional layer to obtain intermediate features. These intermediate features are then input into three parallel branches: the spatial attention module, the channel attention module, and the positional attention module, to obtain spatial enhancement features, channel enhancement features, and positional enhancement features. The three enhancement features are then concatenated in a concatenation layer, and then passed through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer to generate adaptive fusion weights. These weights are then multiplied element-wise with the features concatenated in the concatenation layer and passed through the final output projection convolutional layer to obtain the output features.

[0013] Preferably, step 3 specifically involves: inputting the student classroom behavior dataset from step 1 into the improved YOLOv11 network structure for training. During training, the input resolution is set to 640×640, the batch size is set to 16, the maximum number of training epochs is set to 500, the optimizer is AdamW, and an early stopping strategy is adopted during training. When the validation set metric shows no improvement for 100 consecutive epochs, training is terminated, and the best weights are retained as the trained student classroom scene object detection model.

[0014] The beneficial effects of this invention are: This invention significantly enhances the backbone network's ability to represent complex multi-scale classroom behaviors through a multi-scale feature enhancement module, helping to improve the detection accuracy of small targets and fine-grained actions. On the other hand, the semantic feature alignment module and joint attention module effectively enhance the semantic consistency and discriminativeness of feature fusion, improving the model's robustness in dense target and occluded environments. It can stably detect student behavior in different classroom scenarios, providing reliable technical support for classroom analysis, learning status assessment, and teaching optimization in smart education environments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall structure of the YOLOv11 one-stage object detection network in the existing technology; Figure 2 This is a schematic diagram of the student classroom scene target detection model in the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of the present invention; Figure 3 This is a schematic diagram of the multi-scale feature enhancement module in the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of the present invention; Figure 4 This is a schematic diagram of the semantic feature alignment module in the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of the present invention; Figure 5 This is a schematic diagram of the joint semantic-spatial alignment submodule in the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of the present invention; Figure 6 This is a schematic diagram of the joint attention module structure in the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of the present invention; Figure 7 This is a diagram showing the comparison of visualization results of classroom scene target detection using the classroom scene target detection method based on multi-scale feature enhancement and semantic alignment of this invention. Detailed Implementation

[0016] The following detailed description is provided in conjunction with specific implementation methods.

[0017] Example 1 This invention relates to a classroom scene target detection method based on multi-scale feature enhancement and semantic alignment, which is implemented according to the following steps: Step 1: Obtain the classroom scene target detection dataset; Step 2: Improve the YOLOv11 one-stage object detection network to obtain a target detection model for student classroom scenes; Step 3: Use the classroom scene object detection dataset obtained in Step 1 to train the student classroom scene object detection model to obtain the trained student classroom scene object detection model; Step 4: Input the student classroom scene image to be detected into the trained student classroom scene target detection model for detection.

[0018] Example 2 Building upon Example 1, the classroom scene object detection dataset in step 1 utilizes the publicly available Classroom Scene Object Detection Dataset (SCB). Classroom scene images are labeled with behavior categories and target locations to form a detection sample set for training and validation. The SCB dataset comprises four classroom behavior sub-datasets: SCB-HRW, SCB-Discuss, SCB-BTH, and SCB-Teacher. These sub-datasets correspond to fine-grained student behavior, multi-person and group interaction behavior, subtle pose change behavior, and teacher-related teaching scenarios, respectively. Since each sub-dataset differs in category composition, number of images, and target bounding box distribution, the four sub-datasets are trained and validated as independent detection tasks during implementation to avoid interference from differences in category sets in the evaluation results.

[0019] Example 3 Based on Example 2, the improvement of the YOLOv11 one-stage object detection network in step 2 is as follows: like Figure 1-2 As shown, the Fast Spatial Pyramid Pooling (SPPF) module in the 10th layer of the YOLOv11 one-stage object detection network backbone is replaced with a multi-scale feature enhancement module. The output of the C3k2 module in the 9th layer of the backbone network is used as the input of the multi-scale feature enhancement module, and the output of the multi-scale feature enhancement module is used as the input of the C2PSA module in the 11th layer of the backbone network. After completing the scale feature extraction, the C3k2 module in the 9th layer of the backbone network first inputs the deep feature into the multi-scale feature enhancement module, and then sends the output of the multi-scale feature enhancement module into the subsequent C2PSA module to enhance the ability of deep features to jointly express classroom behavior small targets, local action details and global context information.

[0020] Two semantic feature alignment modules are set in the top-down feature fusion path of the neck network in the YOLOv11 one-stage object detection network. The first semantic feature alignment module is set between the 13th layer concatenation layer and the 14th layer C3k2 module of the neck network. The output of the 11th layer C2PSA module of the backbone network and the output of the 13th layer concatenation layer of the neck network are used as the input of the first semantic feature alignment module, and the output of the first semantic feature alignment module is used as the input of the 14th layer C3k2 module of the neck network. The second semantic feature alignment module is set between the 16th layer concatenation layer and the 17th layer C3k2 module of the neck network. The output of the 14th layer C3k2 module and the output of the 16th layer concatenation layer of the neck network are used as the input of the second semantic feature alignment module, and the output of the second semantic feature alignment module is used as the input of the 17th layer C3k2 module of the neck network. Both the first and second semantic feature alignment modules are used to perform semantic and spatial alignment of the high-resolution and low-resolution features to be fused. Through these two semantic feature alignment modules, the semantic inconsistency problem that exists when features of different scales are directly fused can be alleviated.

[0021] After the C3k2 module in layer 17, layer 20, and layer 23 of the neck network, a joint attention module is set. The three scale features are reweighted by the corresponding joint attention module and then input into the detection layer to achieve bounding box regression and behavior category prediction.

[0022] Preferably, such as Figure 3 As shown, the multi-scale feature enhancement module includes a first CBS module, a deep excitation module, a multi-scale pooling and recombination module, a residual connection unit, and a second CBS module connected in sequence. The output features of the C3k2 module at layer 9 of the backbone network are used as the input features of the multi-scale feature enhancement module. The input features are processed sequentially through the first CBS module, the deep activation module, and the multi-scale pooling and recombination module. At the same time, the output features of the C3k2 module at layer 9 of the backbone network are subjected to residual mapping. The residual mapping result and the output of the scale pooling and recombination module are then added element-wise in the residual connection unit. After being processed by the second CBS module, the output is used as the input of the C2PSA module at layer 11 of the backbone network. When performing residual mapping on the output features of the C3k2 module at layer 9 of the backbone network, which is also the input features of the multi-scale feature enhancement module, the residual mapping result is obtained through 1×1 convolution when the number of input channels is inconsistent with the number of output channels; and when the number of input channels is consistent with the number of output channels, the residual mapping result is obtained through identity mapping.

[0023] Preferably, the deep activation module includes a deep convolutional layer, a pointwise convolutional layer, a global average pooling layer, a multilayer perceptron, and a sigmoid activation layer connected in sequence. The output of the first CBS module is used as the input of the deep activation module. That is, the output of the first CBS module is sequentially fed into the deep convolutional layer for channel-independent spatial convolution operations, followed by pointwise convolutional layers to complete inter-channel feature mixing, then a global average pooling layer to extract channel description vectors, and then a multilayer perceptron and a sigmoid activation layer to generate channel weights. The channel weights are then multiplied elementwise with the output features of the pointwise convolutional layer, and the result of the multiplication is used as the output of the deep activation module and input into the multi-scale pooling and recombination module. The multi-scale pooling recombination module includes multiple adaptive average pooling layers with different output sizes. Each adaptive average pooling layer is connected to a flattening layer. Multiple flattening layers are connected to a stitching layer. The stitching layer is connected to a recombination layer and an upsampling layer in sequence. The upsampling result is used as the output feature of the scale pooling recombination module. The output features of the deep excitation module are input into multiple adaptive average pooling layers with different output sizes to obtain pooling features of multiple scales. The pooling features of multiple scales are flattened by the corresponding flattening layers and then input into the stitching layer. They are stitched together in the spatial dimension corresponding to the channel. The stitching result is reassembled into a two-dimensional spatial structure in the reconstruction layer. When the spatial size of the reassembled feature is inconsistent with the spatial size of the input feature, it is restored to the same spatial resolution as the input feature by upsampling, thus obtaining the output features of the multi-scale pooling reconstruction module.

[0024] like Figure 4 As shown, the semantic feature alignment module includes a high-resolution feature projection convolutional layer and a low-resolution feature projection convolutional layer arranged in parallel. The low-resolution feature projection convolutional layer is followed by an upsampling layer. The output of the upsampling layer and the output of the high-resolution feature projection convolutional layer are connected to the first stitching layer. The first stitching layer is sequentially connected to a joint semantic-spatial alignment submodule, a residual fusion alignment unit, a second stitching layer, a cross-scale semantic interaction submodule, a splitting layer, and a first output projection convolutional layer. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the first semantic feature alignment module are the high-resolution features output from the 13th layer of the neck network splicing layer and the low-resolution features output from the 11th layer of the backbone network C2PSA module, respectively. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the second semantic feature alignment module are the high-resolution features output from the 16th layer of the neck network splicing layer and the low-resolution features output from the 14th layer of the neck network C3k2 module, respectively.

[0025] like Figure 5As shown, the joint semantic-spatial alignment submodule includes a semantic alignment module and a spatial alignment module set in parallel. The outputs of the semantic alignment module and the spatial alignment module are both used as inputs to the residual fusion alignment unit in the semantic feature alignment module. The semantic alignment module includes a global semantic branch and a local semantic branch set in parallel. The outputs of the global semantic branch and the local semantic branch are connected to a third concatenation layer. The third concatenation layer is also connected to a semantic fusion layer and a weight generation layer in sequence. The inputs of the global semantic branch and the local semantic branch are features concatenated by the first concatenation layer. The global semantic branch extracts global semantic information from the input features through global average pooling and convolutional mapping, while the local semantic branch extracts local semantic information from the input features through local convolutional mapping. The global semantic information and the local semantic information are concatenated by the third concatenation layer, processed by the semantic fusion layer, and then the semantic weight generation layer generates semantic weights. The spatial alignment module includes a multi-scale spatial convolution branch, a fourth stitching layer, and a spatial fusion layer connected in sequence. The spatial fusion layer is connected in parallel with a high-resolution spatial weight generation layer and a low-resolution spatial weight generation layer. The input of the spatial alignment module is the low-resolution features processed by the low-resolution feature projection convolution layer and the upsampling layer, and the high-resolution features processed by the high-resolution feature projection convolution layer. The multi-scale spatial convolution branch uses 1×1 convolution, 3×3 convolution and 5×5 convolution to spatially model the high-resolution features and low-resolution features, respectively. The resulting multi-scale spatial features are stitched by the fourth stitching layer and processed by the spatial fusion layer. The high-resolution spatial weight and low-resolution spatial weight are generated by the high-resolution spatial weight generation layer and the low-resolution spatial weight generation layer, respectively. The residual fusion alignment unit performs residual weighted alignment processing on the high-resolution features after high-resolution feature projection convolution layer processing and the low-resolution features after low-resolution feature projection convolution layer and upsampling layer processing, respectively, based on the semantic weights, high-resolution spatial weights and low-resolution spatial weights output by the joint semantic-spatial alignment submodule, to obtain high-resolution aligned features and low-resolution aligned features. The cross-scale semantic interaction submodule includes a second multi-scale feature extraction branch, a fifth splicing layer, a scale fusion layer, a QKV projection unit, a multi-head attention unit, an attention projection unit, a first residual enhancement unit, a feature refinement unit, a second residual enhancement unit, and a second output projection convolutional layer, which are connected in sequence. High-resolution alignment features and low-resolution alignment features are concatenated by the fifth concatenation layer and then input into the second multi-scale feature extraction branch. The second multi-scale feature extraction branch uses 1×1 convolution, 3×3 convolution and 5×5 convolution to extract cross-scale multi-scale features, which are then fused by the scale fusion layer and fed into the QKV projection unit to generate query features, key features and value features. Subsequently, cross-scale semantic interaction is completed through the multi-head attention unit and attention projection unit. The output results of the attention projection unit and the scale fusion layer are processed by the first residual enhancement unit and the feature refinement unit. The output results of the feature refinement unit and the first residual enhancement unit are then input into the second residual enhancement unit for processing. Finally, the result of the second residual enhancement unit is processed by the second output projection convolution layer and output. The output of the cross-scale semantic interaction submodule is split into high-resolution interaction features and low-resolution interaction features by a splitting layer. A summation layer is also provided between the splitting layer and the first output projection convolutional layer. The summation layer sums the high-resolution projection features, high-resolution alignment features, low-resolution alignment features, high-resolution interaction features and low-resolution interaction features output by the high-resolution feature projection convolutional layer. The summation result is processed by the output projection convolutional layer and used as the output of the semantic feature alignment module.

[0026] Example 4 Based on Example 3, such as Figure 6 As shown, the joint attention module includes a series of projective convolutional layers. The projective convolutional layers are connected in parallel with spatial attention sub-modules, channel attention sub-modules, and position attention sub-modules. The output features of the spatial attention sub-module, channel attention sub-module, and position attention sub-module are concatenated by a concatenation layer, and then sequentially pass through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer. The output of the Sigmoid activation layer and the concatenated features are then multiplied element-wise and passed through an output projective convolutional layer to obtain the output features. The input features of the joint attention module are first processed through a projection convolutional layer to obtain intermediate features. These intermediate features are then input into three parallel branches: the spatial attention module, the channel attention module, and the positional attention module, to obtain spatial enhancement features, channel enhancement features, and positional enhancement features. The three enhancement features are then concatenated in a concatenation layer, and then passed through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer to generate adaptive fusion weights. These weights are then multiplied element-wise with the features concatenated in the concatenation layer and passed through the final output projection convolutional layer to obtain the output features.

[0027] The spatial attention module is used to model spatial saliency information, highlighting local areas related to classroom behavior; the channel attention module is used to model channel importance, strengthening semantic channels related to behavior category discrimination; the position attention module is used to model the relationship between different spatial positions, so as to enhance the ability to express dense targets and complex relative positional relationships. The three branches output spatial enhancement features, channel enhancement features and position enhancement features respectively.

[0028] The spatial attention module uses depthwise separable convolution, dilated convolution, and spatial pyramid pooling to model spatial attention; the channel attention module uses multi-scale pooling and channel activation mechanisms to model channel attention; and the positional attention module uses multi-scale convolutional branching, positional encoding, and a Q, K, V-based self-attention mechanism to model positional relationships.

[0029] Example 5 Based on Example 4, step 3 specifically involves: inputting the student classroom behavior dataset from step 1 into the improved YOLOv11 network structure for training. During training, the input resolution is set to 640×640, the batch size is set to 16, the maximum number of training epochs is set to 500, the optimizer is AdamW, and an early stopping strategy is adopted during training. When the validation set metric shows no improvement for 100 consecutive epochs, training is terminated, and the best weights are retained as the trained student classroom scene object detection model.

[0030] Example 6 Based on Example 5, the present invention provides a classroom scene target detection method based on multi-scale feature enhancement and semantic alignment, which is implemented according to the following steps: Step 1: Obtain the classroom scene target detection dataset; The classroom scene object detection dataset uses the publicly available Classroom Scene Object Detection Dataset (SCB). The SCB dataset comprises four classroom behavior subsets: SCB-HRW, SCB-Discuss, SCB-BTH, and SCB-Teacher. These subsets correspond to fine-grained student behavior, multi-person and group interaction behavior, subtle pose change behavior, and teacher-related teaching scenarios, respectively. Since each subset differs in category composition, number of images, and bounding box distribution, the four subsets are trained and validated as independent detection tasks during implementation to avoid interference from differences in category sets in the evaluation results.

[0031] In this embodiment, the SCB-HRW subset contains 3 target classes, with 5193 training images and 1671 validation images; the SCB-Discuss subset contains 1 target class, with 605 training images and 259 validation images; the SCB-BTH subset contains 2 target classes, with 1905 training images and 505 validation images; and the SCB-Teacher subset contains 8 target classes, with 8984 training images and 3240 validation images. These data are used to improve the network's training and detection validation. Step 2: Improve the YOLOv11 one-stage object detection network to obtain a target detection model for student classroom scenes; The specific improvements are as follows: In the backbone network, the fast spatial pyramid pooling module, located after the last C3k2 module and before the C2PSA module in the original YOLOv11 network, is replaced with a multi-scale feature enhancement module. Correspondingly, after extracting P5-scale features, the backbone network first inputs these deep features into the multi-scale feature enhancement module, and then sends the output of the multi-scale feature enhancement module into the subsequent C2PSA module. This enhances the ability of deep features to jointly express small classroom behavioral objectives, local action details, and global contextual information. The replacement relationship and insertion position are as follows: Figure 1-2 As shown.

[0032] Secondly, two semantic feature alignment modules are set in the top-down feature fusion path of the neck network. The first semantic feature alignment module is set after the P5-scale feature upsampling and concatenation with the backbone network's P4-scale feature, and before the corresponding C3k2 module, to perform semantic and spatial alignment of the P4-scale and P5-scale features to be fused. The second semantic feature alignment module is set after the processed P4-scale feature upsampling and concatenation with the backbone network's P3-scale feature, and before the corresponding C3k2 module, to perform semantic and spatial alignment of the P3-scale and P4-scale features to be fused. These two semantic feature alignment modules can alleviate the semantic inconsistency problem that exists when features of different scales are directly fused.

[0033] Furthermore, in the detection head, the features at three detection scales, P3, P4, and P5, are introduced into a joint attention module for feature refinement before entering the detection layer. Specifically, the P3 scale feature is processed by the first joint attention module, the P4 scale feature by the second joint attention module, and the P5 scale feature by the third joint attention module. The three scale features, after being reweighted by the joint attention module, are then jointly input into the detection layer to achieve bounding box regression and behavior category prediction.

[0034] Step 3: Use the classroom scene object detection dataset obtained in Step 1 to train the student classroom scene object detection model to obtain the trained student classroom scene object detection model; The enhancement process of the multi-scale feature enhancement module during training is as follows: The P5-scale features output by the backbone network are used as input features for the multi-scale feature enhancement module, denoted as X. The input feature X first undergoes convolutional mapping, batch normalization, and nonlinear activation processing in the first CBS module to obtain an intermediate feature representation. Subsequently, this intermediate feature is input into the deep activation module. The deep activation module first performs spatial convolution operations on each channel independently through a deep convolutional layer to extract local spatial structure information, and then performs cross-channel information mixing through a pointwise convolutional layer. After that, the output of the pointwise convolutional layer is subjected to global average pooling to obtain the channel description vector, and then passes through the first fully connected layer, batch normalization layer, SiLU activation layer, second fully connected layer, and Sigmoid activation layer in the multilayer perceptron to generate adaptive weights for each channel. Finally, the channel weights are multiplied element-wise with the output of the pointwise convolutional layer to obtain the feature output after channel recalibration.

[0035] The output of the deep activation module is further input into the multi-scale pooling and recombining module. The multi-scale pooling and recombining module uses three adaptive average pooling layers with output sizes of 2×2, 6×6 and 9×9 to extract statistical responses from the input features at different spatial scales. Then, the outputs of each branch are flattened and spliced ​​in the spatial dimension to form a unified multi-scale spatial encoding vector. This vector is then recombined into a two-dimensional spatial structure and upsampled to the same spatial resolution as the input features through nearest neighbor interpolation to obtain the spatial recoded features.

[0036] In the output stage of the multi-scale feature enhancement module, the spatial recoded features output by the multi-scale pooling recombination module are combined with the input features. The residual mapping results are element-wise summed within the residual connection units. When the number of input and output channels is the same, the residual mapping uses an identity mapping. When the number of input and output channels is different, 1×1 convolution is used to complete dimension matching. The features after residual summation are then input into the second CBS module for convolution mapping to obtain the enhanced backbone features, which are then sent to the subsequent C2PSA module for subsequent feature extraction and transmission. Thus, the multi-scale feature enhancement module achieves joint modeling of local details, channel responses, and multi-scale spatial context within the same feature level.

[0037] The enhancement process of the multi-scale feature enhancement module is shown in formula (A):

[0038] (A)

[0039] in, This represents the features input to the deep activation module, i.e., the input features. Feature representation after processing by the first CBS module; This indicates a depthwise convolution operation, used to perform spatial convolution operations independently on each channel; This indicates a pointwise convolution operation, used to achieve information fusion between channels; This represents a global average pooling operation used to generate channel description vectors; This represents the multilayer perceptron structure in the deep activation module, which consists of a first fully connected layer, a batch normalization layer, a SiLU activation layer, and a second fully connected layer, used to generate channel attention weights. This involves processing the Sigmoid function in the Sigmoid activation layer of the deep activation module. This indicates that the compression ratio in the multilayer perceptron of the deep excitation module is [value missing]. Two fully connected mapping operations; This indicates an element-wise multiplication operation, used to recalibrate the channel weights and feature maps. This represents the output features of the deep activation module, and also serves as the input to the multi-scale pooling recombination module; This represents the operation of the adaptive average pooling layer in the multi-scale pooling recombination module, with an output spatial size of... Adaptive average pooling operation; and These represent the splicing and flattening operations in the multi-scale pooling recombination module, respectively. Indicates an upsampling operation. This refers to the recombination operation of the recombination layer in the multi-scale pooling recombination module; Spatial recoding features output by the multi-scale pooling recombination module; express convolution; This represents the combination of batch normalization and the SiLU activation function. Finally, the spatially recoded features are... Input features The output features of the multi-scale feature enhancement module are obtained by element-wise addition through residual connections and convolutional mapping through the second CBS module. When the number of input and output channels is inconsistent, the residual branch passes through... Convolution completes channel dimension matching.

[0040] After the backbone features are enhanced, the enhanced deep features are fed into the feature fusion network for cross-scale fusion. First, the P5-scale deep features processed by C2PSA are upsampled and concatenated with the P4-scale features output from the backbone network. The concatenated result serves as the high-resolution input to the first semantic feature alignment module. At the same time, the P5 features from deeper layers serve as the low-resolution input to the first semantic feature alignment module. The semantic feature alignment module first performs 1×1 convolutional projection on the high-resolution and low-resolution features respectively, and upsamples the low-resolution features to the same spatial size as the high-resolution features to establish a unified spatial scale and channel dimension.

[0041] Subsequently, the projected high- and low-level features are input into the joint semantic-spatial alignment submodule for semantic-spatial joint alignment. The joint semantic-spatial alignment submodule constructs a cross-scale joint representation by concatenating high- and low-level features, and generates semantic weights, high-resolution spatial weights, and low-resolution spatial weights based on this joint representation and the two individual feature paths. The semantic weights are used to modulate the channel responses of the high- and low-level features, while the spatial weights are used to constrain the consistency of the two feature paths in the spatial region of interest. After this processing, the high- and low-level features are aligned at the semantic level and the spatial response level.

[0042] After joint alignment, the aligned high-resolution and low-resolution features are concatenated along the channel dimension and input into the cross-scale semantic interaction submodule for cross-scale semantic interaction. The cross-scale semantic interaction submodule first extracts multi-scale features from the concatenated features, obtaining cross-scale multi-scale features through branches of different convolutional receptive fields. Then, the outputs of each branch are concatenated and scale-fused to obtain the fused cross-scale joint features. Further, query features, key features, and value features are mapped onto the fused cross-scale joint features, and the semantic dependencies between high- and low-level features are modeled using a multi-head attention mechanism.

[0043] Based on this, the attention interaction result is projected through attention and then subjected to a first residual enhancement with the scale fusion feature. After feature refinement, it is subjected to a second residual enhancement with the first residual enhancement result. Finally, the cross-scale semantic interaction output feature is obtained through a second output projection convolutional layer. Then, the cross-scale semantic interaction output feature is split into high-resolution interaction features and low-resolution interaction features through a splitting layer. Finally, the high-resolution projection feature, high-resolution alignment feature, low-resolution alignment feature, high-resolution interaction feature, and low-resolution interaction feature are summed and fused, and then processed by a first output projection convolutional layer to obtain the output feature of the semantic feature alignment module. The alignment and fusion process of the semantic feature alignment module is shown in formula (B):

[0044] (B)

[0045] in, Indicates high-resolution features, Indicates low-resolution features. and These represent the channel projections of high- and low-level features, respectively. This indicates an upsampling alignment operation, used to adjust low-resolution features to a spatial scale consistent with high-resolution features; This represents the projected features obtained after processing by a high-resolution feature projection convolutional layer. This represents the projected features obtained by projecting low-resolution features through a convolutional layer and then performing an upsampling operation. This represents the shared semantic alignment weights generated from joint cross-scale features; and These represent the spatial alignment weights for high-level branches and low-level branches, respectively. Indicates a splicing operation; , and These represent semantic weights, high-resolution spatial weight generation functions, and low-resolution spatial weight generation functions, respectively. , , , This is the scaling factor; This represents element-wise multiplication; This represents attention-projective convolution; This indicates a feature refinement operation; This represents the second output projection convolution; This indicates the result of the attention interaction. This represents the characteristics after the first residual enhancement. This represents the features before feature refinement and the second residual enhancement. This represents the output features of cross-scale semantic interaction; This indicates a splitting operation. Represents high-resolution interactive features, This indicates low-resolution interactive features.

[0046] Will and as well as , and Both are used as inputs to the residual fusion alignment module, which processes them to obtain high-resolution alignment features and low-resolution alignment features. and ; Will and After the splicing operation, it serves as the input for the cross-scale semantic interaction submodule; High-resolution alignment features Alignment features with low resolution As input to the cross-scale semantic interaction submodule, and processed by the cross-scale semantic interaction submodule, the output is... ; Will After being split into high-resolution interactive features by the splitting layer and low-resolution interaction features ; Then as well as , , After summation, the output features of the semantic feature alignment module are obtained by processing through the first output projection convolutional layer. Specifically: ; Indicates output projection convolution; This represents the output feature of the semantic feature alignment module.

[0047] The process of the joint attention module is shown in formula (C):

[0048] (C) in, Indicates the input detection features; Indicates the input compression mapping; , and These represent the spatial attention branch, the channel attention branch, and the positional attention branch, respectively. , and These represent the feature representations after attention enhancement for three different types; Indicates feature splicing; Indicates global average pooling; Indicates to , and Features after splicing This represents the multilayer perceptron fusion weight generation function; Represents the Sigmoid function; This represents the adaptive fusion weights of the multilayer perceptron; Indicates output mapping; This represents element-wise multiplication; This represents the output characteristics of the joint attention module.

[0049] During training, the input resolution was uniformly set to 640×640, the batch size to 16, the maximum number of training epochs to 500, the AdamW optimizer was used, the learning rate was set to 0.01, and the number of data loading threads was set to 8. Automatic mixed precision and exponential moving average were enabled during training, and an early stopping strategy was employed: training was terminated when the validation set metrics showed no improvement over 100 consecutive epochs, and the optimal weights were retained as the trained student classroom behavior detection model. Each subset of datasets was trained and validated independently to obtain the detection model for the corresponding task.

[0050] Step 4: Input the student classroom scene image to be detected into the trained student classroom scene target detection model for detection.

[0051] The classroom scene image to be detected is input into the student classroom scene object detection model trained in step 3. The model outputs the target bounding box location, behavior category, and corresponding confidence score. To visually demonstrate the detection results, the predicted bounding box, behavior category label, and confidence score information are overlaid onto the original classroom image to form a visualization of the classroom scene object detection results, such as... Figure 7 As shown.

[0052] Figure 7 This presentation showcases original images from representative classroom scenarios, along with the detection results from the YOLOv11 baseline model and the detection results from the method described in this invention. The examples cover four classroom sub-scenarios: HRW, Discuss, BTH, and Teacher, reflecting the detection performance of the method in situations such as dense student targets, significant local occlusion, similar behavioral semantics, a high proportion of small targets in the back row, and complex classroom backgrounds.

[0053] pass Figure 7 The visualization comparison shows that, compared with the original YOLOv11 model, the method of this invention can more accurately locate student behavioral targets in complex classroom scenarios, more effectively distinguish semantically similar behavior categories, and reduce missed detections and false detections. Especially in scenarios with densely distributed students, severe local occlusion, and numerous small-scale targets, the method of this invention can maintain more complete target detection results and more stable detection confidence. This indicates that the multi-scale feature enhancement module helps supplement multi-scale contextual information in deep features, the semantic feature alignment module helps improve semantic consistency during cross-scale fusion, and the joint attention module further strengthens key regions and discriminative cues related to classroom behavior. The synergistic effect of these three modules improves classroom behavior detection performance. This technical solution can provide effective support for classroom behavior analysis, learning status assessment, and teaching optimization in smart education scenarios.

Claims

1. A method for classroom scene target detection based on multi-scale feature enhancement and semantic alignment, characterized in that, The specific steps are as follows: Step 1: Obtain the classroom scene target detection dataset; Step 2: Improve the YOLOv11 one-stage object detection network to obtain the object detection model for student classroom scenes; Step 3: Use the classroom scene object detection dataset obtained in Step 1 to train the student classroom scene object detection model to obtain the trained student classroom scene object detection model; Step 4: Input the student classroom scene image to be detected into the trained student classroom scene target detection model for detection.

2. The method of claim 1, wherein, In step 1, the classroom scene object detection dataset uses the publicly available classroom scene object detection dataset SCB. 3.The method of claim 1, wherein, The specific improvements to the YOLOv11 one-stage object detection network in step 2 are as follows: The Fast Spatial Pyramid Pooling (SPPF) module at layer 10 of the YOLOv11 one-stage object detection network backbone is replaced with a multi-scale feature enhancement module. The output of the C3k2 module at layer 9 of the backbone network is used as the input of the multi-scale feature enhancement module, and the output of the multi-scale feature enhancement module is used as the input of the C2PSA module at layer 11 of the backbone network. In the top-down feature fusion path of the neck network in the YOLOv11 one-stage object detection network, two semantic feature alignment modules are set. The first semantic feature alignment module is set between the 13th layer splicing layer and the 14th layer C3k2 module of the neck network. The output of the 11th layer C2PSA module of the backbone network and the output of the 13th layer splicing layer of the neck network are used as the input of the first semantic feature alignment module, and the output of the first semantic feature alignment module is used as the input of the 14th layer C3k2 module of the neck network. The second semantic feature alignment module is set between the 16th layer splicing layer and the 17th layer C3k2 module of the neck network. The output of the 14th layer C3k2 module and the output of the 16th layer splicing layer of the neck network are used as the input of the second semantic feature alignment module, and the output of the second semantic feature alignment module is used as the input of the 17th layer C3k2 module of the neck network. After the C3k2 module in layer 17, layer 20, and layer 23 of the neck network, a joint attention module is set up. The three scale features are reweighted by the corresponding joint attention module and then input into the detection layer.

4. The method of claim 3, wherein, The multi-scale feature enhancement module includes a first CBS module, a deep excitation module, a multi-scale pooling and recombination module, a residual connection unit, and a second CBS module connected in sequence. The output features of the C3k2 module at layer 9 of the backbone network are used as the input features of the multi-scale feature enhancement module. The input features are processed sequentially by the first CBS module, the deep activation module, and the multi-scale pooling and recombination module. At the same time, the output features of the C3k2 module at layer 9 of the backbone network are subjected to residual mapping. The residual mapping result and the output of the scale pooling and recombination module are then added element-wise in the residual connection unit. After being processed by the second CBS module, the output is used as the input of the C2PSA module at layer 11 of the backbone network. When performing residual mapping on the output features of the C3k2 module at layer 9 of the backbone network, which is also the input features of the multi-scale feature enhancement module, the residual mapping result is obtained through 1×1 convolution when the number of input channels is inconsistent with the number of output channels; and when the number of input channels is consistent with the number of output channels, the residual mapping result is obtained through identity mapping.

5. The method of claim 4, wherein, The deep activation module includes a deep convolutional layer, a pointwise convolutional layer, a global average pooling layer, a multilayer perceptron, and a sigmoid activation layer connected in sequence. The output of the first CBS module is used as the input of the deep activation module. That is, the output of the first CBS module is sequentially input into the deep convolutional layer for channel-independent spatial convolution operation, and then the inter-channel feature mixing is completed through the pointwise convolutional layer. Subsequently, the channel description vector is extracted through the global average pooling layer, and then the channel weights are generated through the multilayer perceptron and the sigmoid activation layer. The channel weights are then multiplied elementwise with the output features of the pointwise convolutional layer, and the result of the multiplication is used as the output of the deep activation module and input into the multi-scale pooling and recombination module. The multi-scale pooling recombination module includes multiple adaptive average pooling layers with different output sizes. Each adaptive average pooling layer is connected to a flattening layer. Multiple flattening layers are connected to a stitching layer. The stitching layer is connected to a recombination layer and an upsampling layer in sequence. The upsampling result is used as the output feature of the scale pooling recombination module. The output features of the deep excitation module are input into multiple adaptive average pooling layers with different output sizes to obtain pooling features of multiple scales. The pooling features of multiple scales are flattened by the corresponding flattening layers and then input into the stitching layer. They are stitched together in the spatial dimension corresponding to the channel. The stitching result is reassembled into a two-dimensional spatial structure in the reconstruction layer. When the spatial size of the reassembled feature is inconsistent with the spatial size of the input feature, it is restored to the same spatial resolution as the input feature by upsampling, thus obtaining the output features of the multi-scale pooling reconstruction module.

6. The method of claim 5, wherein, The semantic feature alignment module includes a high-resolution feature projection convolutional layer and a low-resolution feature projection convolutional layer arranged in parallel. The low-resolution feature projection convolutional layer is followed by an upsampling layer. The output of the upsampling layer and the output of the high-resolution feature projection convolutional layer are jointly connected to a first stitching layer. The first stitching layer is sequentially connected to a joint semantic-spatial alignment submodule, a residual fusion alignment unit, a second stitching layer, a cross-scale semantic interaction submodule, a splitting layer, and a first output projection convolutional layer. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the first semantic feature alignment module are the high-resolution features output from the 13th layer of the neck network splicing layer and the low-resolution features output from the 11th layer of the backbone network C2PSA module, respectively. The inputs to the high-resolution feature projection convolutional layer and the low-resolution feature projection convolutional layer in the second semantic feature alignment module are the high-resolution features output from the 16th layer of the neck network splicing layer and the low-resolution features output from the 14th layer of the neck network C3k2 module, respectively.

7. The method of claim 6, wherein, The joint semantic-spatial alignment submodule includes a semantic alignment module and a spatial alignment module set in parallel. The outputs of the semantic alignment module and the spatial alignment module are both used as inputs to the residual fusion alignment unit in the semantic feature alignment module. The semantic alignment module includes a global semantic branch and a local semantic branch arranged in parallel. The outputs of the global semantic branch and the local semantic branch are connected to a third concatenation layer. The third concatenation layer is further connected to a semantic fusion layer and a weight generation layer in sequence. The inputs of the global semantic branch and the local semantic branch are features concatenated by the first concatenation layer. The global semantic branch extracts global semantic information from the input features through global average pooling and convolutional mapping. The local semantic branch extracts local semantic information from the input features through local convolutional mapping. The global semantic information and the local semantic information are concatenated by the third concatenation layer, processed by the semantic fusion layer, and then semantic weights are generated by the semantic weight generation layer. The spatial alignment module includes a multi-scale spatial convolution branch, a fourth stitching layer, and a spatial fusion layer connected in sequence. The spatial fusion layer is connected in parallel with a high-resolution spatial weight generation layer and a low-resolution spatial weight generation layer. The input of the spatial alignment module is the low-resolution feature processed by the low-resolution feature projection convolution layer and the upsampling layer, and the high-resolution feature processed by the high-resolution feature projection convolution layer. The multi-scale spatial convolution branch uses 1×1 convolution, 3×3 convolution, and 5×5 convolution to spatially model the high-resolution feature and the low-resolution feature, respectively. The resulting multi-scale spatial features are stitched by the fourth stitching layer and processed by the spatial fusion layer. The high-resolution spatial weight and the low-resolution spatial weight are generated by the high-resolution spatial weight generation layer and the low-resolution spatial weight generation layer, respectively. The residual fusion alignment unit performs residual weighted alignment processing on the high-resolution features after high-resolution feature projection convolution layer processing and the low-resolution features after low-resolution feature projection convolution layer and upsampling layer processing, respectively, based on the semantic weight, high-resolution spatial weight and low-resolution spatial weight output by the joint semantic-spatial alignment submodule, to obtain high-resolution aligned features and low-resolution aligned features. The cross-scale semantic interaction submodule includes a second multi-scale feature extraction branch, a fifth splicing layer, a scale fusion layer, a QKV projection unit, a multi-head attention unit, an attention projection unit, a first residual enhancement unit, a feature refinement unit, a second residual enhancement unit, and a second output projection convolutional layer, which are connected in sequence. The high-resolution alignment features and low-resolution alignment features are concatenated by the fifth concatenation layer and then input into the second multi-scale feature extraction branch. The second multi-scale feature extraction branch uses 1×1 convolution, 3×3 convolution and 5×5 convolution to extract cross-scale multi-scale features, and after being fused by the scale fusion layer, they are sent to the QKV projection unit to generate query features, key features and value features. Then, cross-scale semantic interaction is completed through the multi-head attention unit and attention projection unit. The output results of the attention projection unit and the output results of the scale fusion layer are processed by the first residual enhancement unit and the feature refinement unit. The output results of the feature refinement unit and the output results of the first residual enhancement unit are input into the second residual enhancement unit for processing. Then, the result of the second residual enhancement unit is processed by the second output projection convolution layer and output. The output of the cross-scale semantic interaction submodule is split into high-resolution interaction features and low-resolution interaction features by a splitting layer. A summation layer is also provided between the splitting layer and the first output projection convolutional layer. The summation layer sums the high-resolution projection features, high-resolution alignment features, low-resolution alignment features, high-resolution interaction features and low-resolution interaction features output by the high-resolution feature projection convolutional layer. The summation result is processed by the output projection convolutional layer and used as the output of the semantic feature alignment module.

8. The method of claim 7, wherein, The joint attention module includes a projective convolutional layer connected in sequence. The projective convolutional layer is connected in parallel with a spatial attention submodule, a channel attention submodule, and a position attention submodule. The output features of the spatial attention submodule, the channel attention submodule, and the position attention submodule are concatenated by a concatenation layer, and then sequentially pass through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer. The output of the Sigmoid activation layer and the concatenated features are then multiplied element-wise and passed through an output projective convolutional layer to obtain the output features. The input features of the joint attention module are first processed through a projection convolutional layer to obtain intermediate features. These intermediate features are then input into three parallel branches: the spatial attention module, the channel attention module, and the positional attention module, to obtain spatial enhancement features, channel enhancement features, and positional enhancement features. The three enhancement features are then concatenated in a concatenation layer, and then passed through a global average pooling layer, a multilayer perceptron, and a Sigmoid activation layer to generate adaptive fusion weights. These weights are then multiplied element-wise with the features concatenated in the concatenation layer and passed through the final output projection convolutional layer to obtain the output features.

9. The method of claim 8, wherein, Step 3 specifically involves: inputting the student classroom behavior dataset from Step 1 into the improved YOLOv11 network structure for training. During training, the input resolution is set to 640×640, the batch size is set to 16, the maximum number of training epochs is set to 500, the optimizer is AdamW, and an early stopping strategy is adopted during training. When the validation set metric shows no improvement for 100 consecutive epochs, training is terminated, and the best weights are retained as the trained student classroom scene object detection model.

Images