Classroom behavior detection model and method based on parallel state space model and self-attention

Abstract

The invention discloses a classroom behavior detection model and method based on parallel state space model and self-attention, and belongs to the technical field of intelligent education. A convolution unit driven by a TeLU activation function is designed, and the problem of subtle behavior feature weakening is solved through nonlinear mapping of a hyperbolic tangent index linear unit. A mixed feature fusion unit module is constructed, and attitude transformation and scene change are adapted through multi-branch structure integration space and channel reconstruction convolution and dynamic feature transformation. A VCAF module is designed, features are decomposed into three types of sub-features capable of being independently processed, and through long-time timing sequence modeling of Mamba and aggregation of a convolution addition self-attention mechanism, behavior semantic confusion with highly similar visual performance is solved, redundancy of long-time static features is removed, and details of short-time dynamic features are reserved. An efficient and reliable behavior detection scheme is provided for classroom scene multi-target behavior recognition, and technical support is provided for intelligent classroom real-time behavior monitoring and teaching quality evaluation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent education, and particularly relates to a classroom behavior detection model and method based on a state space model and self-attention in parallel. BACKGROUND

[0002] Student classroom behavior is a key basis for measuring the effectiveness of teacher teaching and the quality of student learning. Teachers can more accurately grasp the participation and knowledge mastery of students by observing and analyzing their classroom performance, and then timely adjust the teaching strategy to improve the classroom learning effect. However, in the traditional classroom teaching evaluation, it depends on the subjective observation of teachers or simple scale recording, which has the limitations of low efficiency, limited coverage, strong subjectivity of data, etc. Teachers and parents can only rely on limited information resources to evaluate the teaching quality and the learning state of students, which is difficult to meet the needs of efficient and accurate collection of behavior data in the smart classroom. Therefore, how to get rid of the limitations of manual work, observe the listening state of all students comprehensively and systematically, and analyze the complete and effective classroom feedback from it has always been a key and difficult point in teaching research.

[0003] With the deep integration of artificial intelligence technology and the field of education, the smart classroom brings new possibilities for the quantitative analysis of classroom behavior. Among them, the classroom behavior recognition technology as an intelligent bridge connecting "teaching" and "learning" data, through computer vision, sensor and other sensing means, automatically captures the behaviors of students in the classroom such as listening, discussing, raising hands, etc., and tracks their behavior changes. Teachers can analyze the participation and attention of students according to their behavior in the classroom, which helps to better understand the effectiveness of teaching methods and timely adjust teaching strategies. It has important significance for improving teaching quality and classroom management.

[0004] For multi-target behavior recognition in the classroom scene, the technical route has gradually evolved from the traditional method of combining hand-crafted features such as HOG and SIFT with SVM to the end-to-end feature learning method based on CNN. However, the existing methods still have obvious limitations in complex classroom scenes:

[0005] (1) Coexistence of behavior category diversity and similarity: In actual classroom, 11 categories or more of behaviors such as writing, looking down at books, looking down at mobile phones, etc. need to be distinguished. The low head writing and the low head looking at the mobile phone are highly similar in posture outline. The amplification ability of general CNN activation functions (such as ReLU, SiLU) for subtle behavior features in the classroom is insufficient, resulting in generally low accuracy in distinguishing similar behaviors. Only relying on local differences such as pen tip movement and mobile phone screen reflection requires high granularity for feature extraction;

[0006] (2) Dynamic changes in scene interference factors: In the classroom environment, variables such as strong light on seats by the window, indoor lighting switching, student sitting posture, height differences, and background clutter can easily cause significant shifts in the visual feature distribution of the same behavior. Moreover, most models lack dynamic feature correction mechanisms and are difficult to adapt to feature shifts caused by scene variables such as lighting and sitting posture.

[0007] (3) Imbalance between global and local feature requirements: Different behaviors have different dependencies on feature scale. For behaviors such as students standing up and teachers giving guidance, global scene features such as the overall posture of the body and the relative position of the person are required. However, behaviors such as raising hands and using mobile phones require local details such as the outline of the fingers and the texture of the objects. Traditional feature extraction methods are difficult to achieve a balanced fusion of the two types of features.

[0008] In summary, recognizing behavior in the classroom environment is crucial for measuring educational progress and optimizing teaching methods in the field of smart education. However, in densely populated classroom settings, multi-target behavior recognition faces challenges such as severe behavior occlusion, complex interaction patterns, small-scale targets, and uneven target distribution. These factors often pose challenges to the detection performance of detectors. Summary of the Invention

[0009] To address the shortcomings of existing research, this invention proposes a classroom behavior detection model and method based on a state-space model and self-attention parallelism for multi-target behavior recognition in complex classroom scenarios.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A classroom behavior detection model based on a state-space model and self-attention parallelism, denoted as SC-CAMamba, is proposed. Internally, the SC-CAMamba model utilizes a feature extraction and dynamic feature transformation module (CBT, CDT), a feature transformation and reconstruction module (SC3d2), and a deep feature fusion module (VCAF). The feature extraction and dynamic feature transformation module (CBT, CDT) combines convolution operations with hyperbolic tangent exponential linear units (TeLUs). While capturing edge features of behavior through local convolution, it leverages the nonlinearity of TeLUs to enhance weak feature responses, avoiding feature loss due to gradient vanishing in the negative interval. The feature transformation and reconstruction module (SC3d2) integrates spatial and channel reconstruction convolutions with dynamic feature transformations through a multi-branch structure to adapt to pose changes and scene variations across multiple scales. The deep feature fusion module (VCAF) performs deep fusion of multi-scale features. By introducing long-term temporal modeling from Mamba and fusion through convolutional addition and self-attention, it achieves the aggregation of key information, resolving semantic confusion of behaviors with highly similar visual representations, and removing redundancy from long-term static features while preserving details from short-term dynamic features.

[0012] As a preferred embodiment of the present invention, the Feature Deep Fusion Module (VCAF) is specifically divided into three stages: a global semantic channel, a local spatial channel, and an identity preservation channel. It concatenates features of different scales output from the feature extraction network as input features, with a feature map size of X. in ∈R B×Cin×H×W Where B represents the batch number, and C represents the batch number. in Here, H and W represent the number of input channels, and H and W represent the width and height of the feature map, respectively; [The last part, "feature map width and height," appears to be a typo and can be omitted.] Figure X According to the preset channel coefficient The feature maps are divided into three sub-feature maps, including µ. , , After passing through the VCAF deep fusion layer, the feature map becomes X. out ∈R B×Cout×H×W C out This represents the number of output channels.

[0013] As a preferred embodiment of the present invention, the construction steps in the global semantic channel stage of the feature deep fusion module (VCAF) are as follows:

[0014] First, a lightweight VSS unit (VDSSBlock) is designed, comprising a three-level substructure of dynamic feature transformation, selective scan modeling, and feedforward extraction. The input features are then transformed from... Expand to ,in The number of input channels for the global branch is used, and dynamic DyTL is employed to adjust the dynamic behavior boundary of the inter-frame difference through adaptive threshold adjustment, while suppressing background noise.

[0015] Then, the selective scanning unit SS2D is introduced to model the temporal correlation of classroom behavior with linear time complexity (O(N)). This unit is designed with scanning paths in four directions to capture temporal features to adapt to the typical action directions of classroom behavior.

[0016] Finally, the temporal features after scanning are extracted through batch normalization and feedforward network (FFN). FFN adopts a lightweight structure of 1×1 convolution + TeLU activation + 1×1 convolution. It utilizes the nonlinear transformation of TeLU to enhance semantic association features and suppress irrelevant background features.

[0017] The definition formula for the VDSSBlock module is given by formulas (1) and (2):

[0018]

[0019]

[0020] in, This indicates that a split operation is performed on the feature, and the number of channels is taken. Take before Characteristics of channel number This indicates that a split operation is performed on the feature, and the number of channels is taken. After taking Characteristics of the number of channels.

[0021] As a preferred technical solution of the present invention, in the local spatial channel stage of the Feature Deep Fusion Module (VCAF), convolutional additive self-attention (CASelf_Attention) based on multiple information interactions in spatial and channel domains is adopted, and a convolutional additive token mixer is introduced to replace the query-key matrix multiplication to calculate token relevance; three feature branches are decomposed through QKV, and spatial channel attention enhancement is applied to Q and K respectively, wherein the spatial attention module generates a weight map of spatial dimensions, and filters spatial positions through multiplication to strengthen key regions such as edges and textures; a weight value is generated for each channel, and channels are filtered through multiplication to strengthen important channels and suppress redundant channels; finally, through depthwise separable convolution processing, the original classroom image is transformed into high-discrimination, low-redundancy behavioral features.

[0022] The spatial and channel domains in CASelf_Attention are defined by equations (3) and (4):

[0023]

[0024]

[0025] in, This represents the Sigmoid activation function. (·) represents the average pooling AvgPool with an average pooling of 1×1.

[0026] As a preferred technical solution of the present invention, in the identity preservation channel stage of the feature deep fusion module (VCAF), dynamic threshold convolution (CDT) is used as the core unit. Through the method of "convolution + adaptive threshold activation", noise is filtered while preserving static pose features. The definition of the identity preservation channel is shown in formula (5):

[0027]

[0028]

[0029] First, the features from the three channels are fused and refined. The output features of the three branches are first obtained by channel concatenation to obtain the fused features. Subsequently, the feature extraction unit of the hyperbolic tangent exponential linear unit is used to further enhance the cross-branch semantic association;

[0030] Finally, a 1×1 convolution is used to compress the number of concatenated channels to the target output channel Cout, and a BN layer is used to standardize the feature distribution to avoid information imbalance caused by differences in feature scale between branches.

[0031] As a preferred technical solution of the present invention, the feature transformation and reconstruction module (SC3d2) performs feature extraction and feature interaction in different network layers, as shown in formula (7):

[0032]

[0033] in, This indicates a compound operation of the function, in which the CBT operation on the right is executed first, and then the result is input into the Concat operation on the left.

[0034] SC3d2 first expands the input feature channels through CBT convolution, dividing the features into shallow information features that are directly passed and deep features that are processed in depth. The shallow features are directly passed to the next layer to preserve the background features, while the deep features are processed in depth to extract action features.

[0035] The spatial reconstruction convolutional SRU is processed as shown in equations (8) and (9), where the input features x are grouped and normalized; and through... Adjust the normalization value to balance the importance weights of each channel; then subtract the informative features from the original features to obtain x2 of non-informative features that are relatively minor; finally, merge the two features along the channel dimension to enhance the feature representation.

[0036]

[0037]

[0038] in , for The channel is split in two. for The channel is split in two. Here, g is the sigmoid function, and g is the number of groups. Learnable scaling parameters;

[0039] The Channel Reconstruction Unit (CRU) splits the features into high-channel and low-channel parts, and then uses 1×1 convolution to compress the channels to reduce the amount of computation while fusing cross-channel information. Subsequently, group convolution and point convolution are used to enhance local feature extraction and linear transformation. Finally, global average pooling is used to compress the feature map to capture global channel information, and softmax is used to highlight important channels to weight the original feature Y before adding them together for output.

[0040] The processing of spatial reconstruction convolutional SRU is shown in Equations (10) and (11);

[0041]

[0042]

[0043] Where Y represents the original feature Y for attention weighting, and GAP(·) represents global average pooling.

[0044] As a preferred technical solution of the present invention, in the feature extraction and dynamic feature transformation module (CBT, CDT), the CBT module combines convolution operation with hyperbolic tangent exponential linear unit (TeLU) for basic feature extraction of classroom behavior recognition. It captures the edge features of behavior through local convolution, and at the same time uses the nonlinear characteristics of TeLU to enhance the weak feature response. The mathematical expression of hyperbolic tangent exponential linear unit TeLU is shown in formula (12):

[0045]

[0046] When x > 0, the TeLU activation function exp(x) increases rapidly with increasing x, and tanh(e x When x ≈ 1, TeLU(x) ≈ x and strong features are preserved; when x ≈ 0, e x ≈ 1,tanh(e x When x ≈ 0.76, TeLU(x) ≈ 0.76x, which enhances the weak characteristic response through multiplication coefficients; when x > 0, e x It decays rapidly as x decreases, tanh(e x ) ≈ 0, at this time TeLU(x)≈ 0, and the noise feature is suppressed.

[0047] As a preferred technical solution of the present invention, the dynamic feature transformation unit DyTL achieves adaptive capture and optimization of different behavioral features through the extraction of global features by standard convolution, the nonlinear enhancement of TeLU and dynamic normalization of DyTL; DyTL introduces DyT (Dynamic Tanh Normalization) and hyperbolic tangent exponential linear unit TeLU, and dynamically adjusts the feature distribution through the science department threshold. The mathematical expression is shown in formula (13). When the classroom background is complex, θ automatically increases to enhance the normalization intensity and filter noise; when the background is simple, θ decreases to retain more detailed features; where θ is optimized through backpropagation and finally converges to the optimal value that fits the specific behavioral category.

[0048]

[0049] Where θ is the learnable threshold, initially set to 0.85. To minimize the value, avoid a divisor θ of 0.

[0050] This invention also proposes a method for detecting classroom behavior based on this model. After constructing the model according to the model structure, the SC-CAMamba model is obtained through training, reinforcement and validation of the dataset. After inputting the image to be detected into the SC-CAMamba model, the model analyzes and outputs the results to achieve the recognition of multi-target behaviors.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] (1) A convolutional unit (CBT) driven by the TeLU activation function is adopted. Through the nonlinear mapping of the hyperbolic tangent exponential linear unit, the weak features in classroom behavior are amplified, and the problem of weakening of subtle behavioral features is solved. A dynamic feature transformation module (CDT) is constructed to achieve adaptive correction of features through learnable dynamic parameters, thereby improving the model's ability to adapt to behavioral features under different lighting and sitting postures.

[0053] (2) Construct a hybrid feature fusion unit module (SC3d2), which integrates spatial and channel reconstruction convolutions with dynamic feature transformations through a multi-branch structure to adapt to pose changes and scene variations. Channel reconstruction convolutions are introduced to suppress redundant information, while channel recombination enhances feature interaction. The intensity of features is dynamically learned and adjusted to adapt to dynamic changes in dense scenes.

[0054] (3) Design the VCAF module, which decomposes the features into three types of sub-features that can be processed independently, and then introduces the fusion of Mamba's long-term temporal modeling and convolutional addition self-attention to achieve the aggregation of key information, solve the semantic confusion of behaviors with highly similar visual performances, and remove redundancy from long-term static features and retain details from short-term dynamic features.

[0055] (4) Construction of a dataset of classroom student learning behaviors. Based on regular natural classroom videos, a small-scale dataset of classroom learning behaviors was constructed, and the number of behaviors was increased to 11.

[0056] (5) Experimental results show that the method of this invention achieves a precision of 76.7 and an F1-score of 74.5 on the publicly available SCB-DATASET5 dataset. On the self-built classroom behavior recognition dataset, it achieves a recall of 83.3%, and mAP@50 and mAP@95 of 88.6% and 67.9%, respectively. This provides an efficient and reliable behavior detection solution for multi-target behavior recognition in classroom scenarios, and offers technical support for real-time behavior monitoring and teaching quality evaluation in smart classrooms. Attached Figure Description

[0057] Figure 1 The overall architecture of SC-CAMamba is as follows: (a) Backbone feature extraction network consisting of CBT and feature transformation SC3d2. (b) Head network consisting of key modules such as CBT, VCAF, and feature reconstruction SC3d2. (c) Feature extraction and dynamic feature transformation modules (CBT, CDT).

[0058] Figure 2 This describes the overall structure of the VCAF module.

[0059] Figure 3 For the VDSSBlock module.

[0060] Figure 4 This is a schematic diagram of an SS2D scan.

[0061] Figure 5 For (a) spatial and (b) channel domain structures.

[0062] Figure 6 The overall architecture of SC3d2 is as follows: (a) SC3d2 module selected by BDyTL and SC3k; (b) BDyTL module with multi-layer CDT convolution; (c) SC3k module with BSCC fusion.

[0063] Figure 7 The overall structure of the BSCC module is shown. (a) BSCC module with fused spatial and channel reconstruction convolution SCConv. (b) Feature processing diagram of BSCC.

[0064] Figure 8 Here are 11 examples of behaviors: (a) Students looking down, using their phones, and sleeping; (b) Students raising their hands, turning their heads, and standing up; (c) Students looking down while writing and receiving guidance from the teacher; (d) Students looking up while listening to the lesson; (e) Students discussing in groups; and (f) Students looking down while reading.

[0065] Figure 9 The experimental results show the comparison of different attention levels and Mamba fusion modules.

[0066] Figure 10 Visual comparison of different models on 11 typical behavioral datasets. Detailed Implementation

[0067] The following detailed description, in conjunction with embodiments and accompanying drawings, provides a further explanation of the classroom behavior detection model and method based on a state-space model and self-attention parallelism proposed in this invention.

[0068] Example 1

[0069] To address the challenges of multi-target behavior recognition, such as severe behavior occlusion, complex interaction patterns, small-scale targets, and uneven target distribution, this invention designs the SC-CAMamba framework. For example... Figure 1 As shown, the SC-CAMamba model internally utilizes key modules such as Dynamic Feature Transformation (CBT, CDT), Feature Transformation and Reconstruction (SC3d2), and Deep Feature Fusion (VCAF). The CBT module combines convolution operations with Hyperbolic Tangent Exponential Linear Units (TeLUs). While capturing edge features of behavior through local convolution, it leverages the non-linearity of TeLUs to enhance weak feature responses, avoiding feature loss due to gradient vanishing in the negative interval. Subsequently, SC3d2 integrates spatial and channel reconstruction convolutions with dynamic feature transformations through a multi-branch structure to adapt to pose changes and scene variations across multiple scales. Finally, VCAF performs deep fusion of multi-scale features, incorporating Mamba's long-term temporal modeling and convolutional additive self-attention to aggregate key information. This addresses semantic confusion arising from highly similar visual behaviors and removes redundancy from long-term static features while preserving details from short-term dynamic features. The three modules work synergistically to improve overall detection capabilities from feature extraction, feature transformation, and feature fusion perspectives. This allows the model to better address the issues of behavioral category diversity and similarity, as well as the imbalance between global and local feature requirements, ultimately achieving efficient prediction.

[0070] 1.1 Feature Deep Fusion Layer (VCAF)

[0071] To address the challenges of numerous irrelevant interferences, small behavioral differences, and high feature redundancy in multi-target behavior recognition in classroom scenarios, this invention designs a Deep Feature Fusion Layer (VCAF). This layer aims to resolve semantic confusion arising from highly similar visual representations, and to remove redundancy from long-term static features such as a sustained sitting posture (e.g., looking up to listen to a lesson) while preserving details from short-term dynamic features such as raising a hand. VCAF decomposes complex classroom behavioral features into three independently processable sub-features through channel segmentation, and then aggregates key information through feature extraction and fusion. Figure 2 The diagram shows the structure of the VCAF module, which concatenates features of different scales output from the feature extraction network as input features. The feature map size is X. in ∈R B×Cin×H×W (B represents the batch, C) in (where H and W are the input channels, and H and W are the width and height of the feature map, respectively). Figure X According to the preset channel coefficient The feature maps are divided into three sub-feature maps, including µ. , , After passing through the VCAF deep fusion layer, the feature map becomes X. out ∈R B ×Cout×H×W (C) out (This refers to the number of output channels). VCAF is specifically divided into three core stages: global semantic channels, local spatial channels, and identity preservation channels.

[0072] 1.1.1 Global Semantic Channel

[0073] The core objective of the global semantic channel is to model the long-term temporal correlation and global semantics of classroom behaviors. For dynamic behaviors that require cross-frame correlation, such as "raising hands," "standing up," and "group discussion," this invention designs a lightweight VSS unit (VDSSBlock), the structure of which is shown in the figure below. Figure 3 As shown, it includes a three-level substructure: dynamic feature transformation, selective scan modeling, and feedforward extraction. Specifically, the input features are first transformed from... Expand to ( (Input channel number for global branch) to achieve initial dimensionality increase and semantic decoupling of features, and adopt dynamic DyTL to replace traditional LayerNorm. Through adaptive threshold adjustment, it strengthens the dynamic behavior boundary of inter-frame differences such as hand raising when raising a hand, and suppresses background noise such as static textures of tables and chairs.

[0074] This invention then introduces the Selective Scanning Unit (SS2D) to replace the self-attention mechanism of the traditional Transformer, modeling the temporal correlation of classroom behavior with linear time complexity (O(N)). This unit uses four scanning paths (forward horizontal, reverse horizontal, forward vertical, and reverse vertical), with the scanning directions as follows: Figure 4 As shown, the forward horizontal captures the rotation from left to right when turning around, the reverse horizontal captures the head rotation of multiple people in a group discussion, the forward vertical captures the displacement from low to high when standing up, and the reverse vertical captures the displacement of the body from high to low when sitting down. The scanning path captures the above temporal features to match the typical movement directions of classroom behavior.

[0075] Finally, batch normalization and feedforward network (FFN) are used to extract the temporal features after scanning. The FFN adopts a lightweight structure of 1×1 convolution + TeLU activation + 1×1 convolution. It utilizes the nonlinear transformation of TeLU to enhance semantic association features and suppress irrelevant background features. The definition formula of the VDSSBlock module is given by formulas (1) and (2):

[0076]

[0077]

[0078] in, This indicates that a split operation is performed on the feature, and the number of channels is taken. Take before Characteristics of channel number This indicates that a split operation is performed on the feature, and the number of channels is taken. After taking Characteristics of the number of channels.

[0079] 1.1.2 Local Spatial Channel

[0080] The core objective of local spatial channels is to distinguish fine-grained classroom behaviors, such as looking down at a book versus looking down at a phone. Traditional self-attention mechanisms achieve information interaction by calculating the similarity weights of each token with all other tokens. However, this approach has a computational complexity of O(N²·d) (where N is the number of tokens and d is the token dimension). When N is large, the computational cost increases dramatically, making it difficult to deploy in resource-constrained scenarios. To address this issue, this invention employs convolutional additive self-attention (CASelf_Attention) to replace traditional spatial attention. This module, based on multiple information interactions in both spatial and channel domains, introduces a convolutional additive token mixer to replace query-key matrix multiplication for calculating token relevance. The CASelf_Attention module is as follows:Figure 2 As shown in the local spatial channel diagram, the spatial and channel domain structures are composed of Figure 5 (a) and (b) are given.

[0081] The core objective of CASelf_Attention is to replace the high-complexity multiplication weight calculation with low-complexity addition operations, reducing the complexity of token mixing to O(N·d) while ensuring that key features are not lost during cross-token information transmission. Specifically, the module decomposes three feature branches using QKV, and applies spatial channel attention enhancement to Q and K respectively. The spatial attention module generates a weight map of spatial dimensions, filters spatial locations through multiplication, and strengthens key regions such as edges and textures. The channel attention module generates a weight value for each channel, filters channels through multiplication, strengthens important channels, and suppresses redundant channels. Finally, in the core mixing module, q+k replaces the traditional self-attention multiplication, reducing the complexity from O(h²w²) to O(hw). Finally, depthwise separable convolution is used to reduce the computational load while maintaining spatial information, transforming the original classroom image into highly discriminative and low-redundancy behavioral features, providing better input for subsequent detection heads. The definitions of spatial and channel domains in CASelf_Attention are shown in formulas (3) and (4):

[0082]

[0083]

[0084] in, This represents the Sigmoid activation function. (·) represents the average pooling AvgPool with an average pooling of 1×1.

[0085] 1.1.3 Identity Retention Channel

[0086] The core objective of this branch is to preserve the static posture baseline of classroom behavior (such as the head angles for "looking up" and "looking down", and the body contours for "sitting posture"), avoiding the loss of basic features caused by multi-branch processing. Dynamic threshold convolution (CDT) is used as the core unit. Through "convolution + adaptive threshold activation", it filters out minor noise such as slight body swaying of students while preserving static posture features. The definition of the identity preservation channel is shown in formula (5):

[0087]

[0088]

[0089] As shown in formula (6), the features of the three channels are finally fused and purified. The output features of the three branches are first obtained by channel splicing to obtain the fused features. Subsequently, a feature extraction unit using a hyperbolic tangent exponential linear unit further strengthens the cross-branch semantic associations, such as the "dynamic temporal sequence" of the global branch and the "hand position" of the local branch. Then, a 1×1 convolution is used to compress the number of concatenated channels to the target output channel Cout, and a Batch Normalization (BN) layer is used to standardize the feature distribution, avoiding information imbalance caused by differences in feature scale between branches. Specifically, the BN weights of the Conv2d-BN unit are initialized to 0 to ensure balanced feature contributions from each branch in the early stages of training, avoiding a single branch dominating feature learning.

[0090] 1.2 Feature Transformation and Reconstruction Module (SC3d2)

[0091] To address the challenges of multi-scale conflicts, loss of occlusion information, and insufficient dynamic interaction modeling in complex scenarios, this invention specifically designs a feature transformation and reconstruction module (SC3d2). This module aims to extract and interact features across different network layers, flexibly utilizing a dynamic feature transformation module (Bottleneck_DyTL) and a spatial and channel feature reconstruction module (SC2k). Specifically, during feature extraction, dynamic feature transformation is used to adapt to scene changes, dynamically learning and adjusting feature strength to accommodate changes in student posture within the backbone network, such as a student transitioning from sitting to standing. During feature fusion, a channel reconstruction enhancement module suppresses redundant background noise while channel recombination enhances feature interaction, promoting interaction between different semantic features. Its core structure diagram and formulas are derived from… Figure 6 (a) and formula (7) are shown:

[0092]

[0093] in, This indicates a compound operation of the function, in which the CBT operation on the right is executed first, and then the result is input into the Concat operation on the left.

[0094] SC3d2 first expands the input feature channels using CBT convolution, dividing the features into shallow features that are directly passed and deep features that are processed. Shallow features are directly passed to the next layer to preserve background features, while the deep features undergo deep processing to extract action features. In the backbone network layers, this invention employs multiple Dynamic Feature Transform Bottleneck Layers (BDyTL) for feature extraction. The BDyTL structure is designed as follows: Figure 6As shown in (b), the BDyTL module first uses a 1×1 kernel-sized convolution to compress channels and retain key action features. Then, grouped convolutions, while ensuring computational efficiency, distinguish overlapping actions such as raising a hand and turning around, thus better capturing global semantics. Finally, the shallow background features are concatenated with the action features processed by the dynamic feature transformation bottleneck layer, and the features are fused using a 1×1 kernel-sized convolution to obtain the final output map. The processed feature map effectively fuses shallow background features and deep action features.

[0095] In the feature fusion network, deep feature processing is replaced by a feature reconstruction module (SC3k), and the SC3k structure is designed as follows: Figure 6 As shown in (c), SC3k combines a Feature Reconstruction Bottleneck Layer (BSCC) with Cross Stage Partial (CSP) to form a multi-scale feature fusion network, aiming to reduce spatial and channel redundancy between features in convolutional neural networks and promote adaptive fusion of multi-scale features. Specifically, deep features are passed through the SC3k module in parallel to CBT convolution and the Feature Reconstruction Bottleneck Layer. The Feature Reconstruction Bottleneck Layer is used to suppress spatial and channel redundancy and enhance gradient propagation while preserving the original spatial information. In the Feature Reconstruction Bottleneck Layer, this invention introduces spatial and channel reconstruction convolutions, where BSCC and SCConv... Figure 7 (a) and Figure 7 As shown in (b).

[0096] This strategy employs a dual mechanism of Spatial Reconstruction Unit (SRU) and Channel Reconstruction Unit (CRU) to suppress background noise such as book reflections and lamplight, thereby reducing redundant computation and enhancing feature representation. The SRU performs spatial feature filtering and reorganization on the input features, distinguishing between "informative features" and "non-informative features" through a gating mechanism, and then cross-reorganizing to enhance feature representation. The processing of the Spatial Reconstruction Convolutional SRU is shown in equations (8) and (9), where the input feature x is grouped and normalized to reduce batch dependency. And through... Adjust the normalization values ​​to balance the importance weights of each channel. Then, subtract the informative features from the original features to obtain x2 of the relatively less important non-informative features. Finally, merge the two parts of features along the channel dimension to enhance the feature representation.

[0097]

[0098]

[0099] in , for The channel is split in two. for The channel is split in two. Here, g is the sigmoid function, and g is the number of groups. Learnable scaling parameters.

[0100] To further reduce channel redundancy, the Channel Reconstruction Unit (CRU) enhances and fuses input features along the channel dimension. This is achieved through channel splitting, compression, transformation, and attention weighting to improve inter-channel feature interaction. Specifically, features are split into high-channel and low-channel components, then 1×1 convolutions are used for channel compression to reduce computation while fusing cross-channel information. Subsequently, grouped convolutions and point convolutions are used to enhance local feature extraction and perform linear transformations. Finally, global average pooling is used to compress the feature map and capture global channel information. Softmax is then used to highlight important channels, weighting the original feature Y before summing the results for output.

[0101] The processing of spatial reconstruction convolutional SRU is shown in Equations (10) and (11).

[0102]

[0103]

[0104] Where Y represents the original feature Y for attention weighting, and GAP(·) represents global average pooling.

[0105] SRU focuses on the "spatial dimension," optimizing the spatial distribution of features through gating and cross-recombination; CRU focuses on the "channel dimension," enhancing channel interaction through splitting, transformation, and attention weighting. Together, they achieve feature refinement in both the "spatial" and "channel" dimensions, thus better addressing issues such as multi-scale conflicts, loss of occlusion information, and insufficient dynamic interaction modeling in complex scenes.

[0106] 1.3 Feature Extraction and Dynamic Feature Transformation Modules (CBT, CDT)

[0107] To address the issues of insufficient feature response and loss of weak features in complex classroom environments, traditional convolutional methods employ a CBT module as the basic unit for feature extraction. The structure of the CBT module is shown in Figure 1(b). This module enhances the ability to capture low-contrast and weak edge features (such as head-down movements in low light and hand-raising movements at a distance) while preserving key behavioral features (such as student posture and action details).

[0108] The CBT module combines convolution operations with the hyperbolic tangent exponential linear unit (TeLU) for basic feature extraction in classroom behavior recognition. It captures edge features of behavior through local convolution and enhances weak feature responses by utilizing the nonlinear characteristics of TeLU. The mathematical expression of the hyperbolic tangent exponential linear unit TeLU is shown in formula (12):

[0109]

[0110] Traditional activation functions like ReLU lose features along the negative half-axis, while SiLU, with its gentle gradient in low-response regions, struggles to adapt to complex classroom environments with weak features, such as a student looking down at a book in backlight or raising their hand in a distant shot. In contrast, the TeLU activation function shows that when x > 0, exp(x) increases rapidly with x, and tanh(e x When x ≈ 1, TeLU(x) ≈ x and retains strong features like those of a student's face in a bright environment. When x ≈ 0, e x ≈ 1, tanh(e x When x ≈ 0.76, TeLU(x) ≈ 0.76x, which enhances weak feature responses such as the edge of a head-down movement in low light through multiplication coefficients. When x > 0, e x It decays rapidly as x decreases, tanh(e x ) ≈ 0, at which point TeLU(x) ≈ 0, and noise features such as wall shadows and desktop reflections are suppressed.

[0111] To address the challenges of feature diversity and environmental complexity in multi-target behavior recognition within complex classroom scenarios, this invention proposes a Dynamic Feature Transformation Unit (DyTL). The core mechanism utilizes a triple approach: standard convolution for global feature extraction, TeLU nonlinear enhancement, and DyTL dynamic normalization to adaptively capture and optimize different behavioral features. DyTL incorporates DyT (Dynamic Tanh Normalization) and the hyperbolic tangent exponential linear unit TeLU. It dynamically adjusts the feature distribution using a science threshold, as shown in formula (13). When the classroom background is complex, θ automatically increases to enhance normalization and filter noise. When the background is simple, θ decreases to retain more detailed features. θ is optimized through backpropagation and ultimately converges to the optimal value for a specific behavior category, such as focusing on screen highlights when using a mobile phone.

[0112]

[0113] Where θ is the learnable threshold, initially set to 0.85. To minimize the value, avoid a divisor θ of 0.

[0114] The CBT module enhances weak feature responses through the nonlinear properties of TeLU activation, providing efficient basic feature extraction capabilities for classroom behavior recognition. It performs particularly well in weak feature enhancement and adaptation to complex environments, and is a core component of advanced modules such as CDT.

[0115] The CDT module, through its collaborative design of dynamic channel interaction, weak feature enhancement, and adaptive normalization, effectively adapts to the diverse features and complex environment required in classroom behavior recognition, providing more robust input features for subsequent feature fusion and classification.

[0116] CBT focuses on feature extraction and enhancement, while CDT adds dynamic normalization to further improve robustness across different scenarios (such as adaptation to different classroom lighting and layouts). Therefore, CBT is used in shallow networks to enhance weak feature responses with nonlinear characteristics, while CDT is used in deep networks to dynamically adapt to complex feature distributions.

[0117] Example 2

[0118] This embodiment demonstrates the experimental results and analysis, covering the dataset, experimental environment and parameter settings, evaluation metrics, experimental results and analysis of the SC-CAMamba model, and compares it with other models. An ablation experiment was also conducted on the SC-CAMamba model.

[0119] In this embodiment, the dataset used in the experiment is first described in detail, and the experimental configuration and evaluation criteria for the classroom behavior dataset are carefully designed. Then, the practical application effectiveness of the SC-CAMamba architecture in multi-target behavior recognition tasks in classroom scenarios is investigated in depth. Through a series of ablation and comparative experiments, the effectiveness of each component in the architecture is systematically verified, and the performance of the SC-CAMamba architecture in terms of model complexity, accuracy, etc., is analyzed and compared with other existing methods. Furthermore, to intuitively demonstrate the superior performance of the SC-CAMamba architecture, this invention provides detailed experimental result charts and in-depth analysis.

[0120] 2.1 Dataset

[0121] This dataset originates from publicly available classroom teaching videos and has been professionally annotated to form structured data. It is specifically designed as an image detection dataset for classroom behavior recognition. The dataset contains 12,949 annotated images and aims to identify and classify various student behaviors in classroom scenarios using computer vision technology. These behaviors include 11 categories: students looking down while writing, looking down while reading, looking up while listening to the lesson, turning around, raising their hands, standing up, participating in group discussions, receiving teacher guidance, looking down, using mobile phones, and sleeping. The dataset is divided into a training set (images), a validation set (images), and a test set (images) using an 8:1:1 ratio. The final distribution of the behavior recognition samples is as follows: students writing with their heads down (4724), students reading with their heads down (6133), students listening to lectures with their heads up (8785), students turning around (3619), students raising their hands (2292), students standing up (2386), students discussing in groups (1001), teachers providing guidance (668), students looking down (2546), students using mobile phones (2433), and students sleeping (806). To better illustrate the prediction results, the labels for each column were coded accordingly. Detailed information about the dataset is shown in Table 1. Figure 8 Examples are provided for each behavior.

[0122] Table 1: Classroom Behavior Dataset Details

[0123] Behavior Coding Standard Number Students bowing to write write Students bowing to hold a pen 4724 Students bowing to read read Students bowing to hold a book 6133 Students looking up to lectures lectures Students looking up to the blackboard or teacher 8785 Students turning their heads truned Students turning their heads or looking around 3619 Students raising their hands hands Students raising one or both hands 2292 Students standing up stand Students standing in a standing position 2386 Students chatting in small groups chat Two or more students having a face-to-face discussion 1001 Teacher explaining explain Teacher bending down next to students to explain 668 Students bowing bow Difficult to determine whether it is a state of bowing to write or bowing to read 2546 Students using phones phone Students holding phones 2433 Students sleeping asleep Students sleeping face down on the table 806

[0124] 2.2 Training Environment and Parameter Settings

[0125] To avoid the impact of different experimental environments on the experimental results, all experiments were conducted in the same configuration environment. The main hardware used in this experiment was an NVIDIA GeForce RTX 3090 (24GB) and a 14-core Intel i7-14600KF processor. Since the versions of Python and CUDA do not affect the experimental results, but the version numbers need to be compatible with both the software and hardware, Python 3.11.11 was used for environment deployment, PyTorch 1.13.0 was chosen as the deep learning framework, and CUDA 11.8 was used to accelerate training. During the experiment, the image input size was set to 640×640, the batch size to 32, the initial learning rate to 0.01, the random seed to 0, and a total of 200 iterations were performed for training.

[0126] 2.3 Evaluation Indicators

[0127] To evaluate the performance of the SC-CAMamba model, precision, recall, mAP, and F1-score were selected as evaluation metrics.

[0128] Accuracy: Accuracy measures the proportion of correctly identified positive samples among all positive predictions made by the model. This metric was chosen because it effectively assesses the model's ability to minimize false positives. The accuracy calculation formula is (14):

[0129] (14)

[0130] TP stands for True Positives, representing the number of targets correctly detected by the model.

[0131] FP stands for False Positives, which represents the number of negative examples that the model incorrectly predicts as positive examples.

[0132] Recall (R): Recall measures the proportion of true positive samples correctly identified by the model, reflecting the model's sensitivity. This metric is an important indicator of the model's ability to detect all relevant targets and is key to comprehensively evaluating model performance. The formula for calculating recall is formula (15):

[0133] (15)

[0134] mAP: mAP measures the average detection accuracy of a model across multiple classes. This metric is chosen because it comprehensively evaluates the model's overall performance across multiple classes. mAP50-95 is a more comprehensive evaluation metric, calculating the mAP value from the 50% to 95% confidence threshold. mAP50-95 better evaluates the robustness and performance of the model. The calculation formula is Equation (16):

[0135] (16)

[0136] APi refers to the AP of the i-th target class. N is the number of detected images.

[0137] F1-score: The F1-score is a statistical metric used to measure the accuracy of a multi-task binary classification model. It takes into account both the accuracy and recall of the classification model. The F1-score can be seen as a weighted average of the model's accuracy and recall, ranging from 0 to 1, with a higher value indicating a better model. The calculation formula is formula (17):

[0138]

[0139] Precision refers to accuracy, and recall refers to recall rate.

[0140] 2.4 Experimental Results and Analysis

[0141] To verify the real-world detection performance of the SC-CAMamba model in classroom scenarios, a comparative experiment was conducted with existing real-time behavior detection algorithms. Based on a unified training configuration, the model's performance was systematically evaluated using core metrics such as accuracy, inference time, parameter count, and GFLOPs. A comprehensive comparative analysis was performed on various existing real-time object detection models and other object detection models incorporating Mamba. This included detection models such as YOLOv11n, YOLOv8, RT-DETR, YOLOv10, YOLOv13, and Mamba-YOLO to ensure the comprehensiveness and impartiality of the comparative experiments. As shown in Table 2, the SC-CAMamba model proposed in this invention was compared with other real-time object detection algorithms on the self-built behavior recognition dataset SCB-Dataset11.

[0142] Table 2: Performance comparison of various models on the self-built action recognition dataset SCB-Dataset11

[0143] Model Species Recall (%) mAP50 (%) mAP95 (%) GFLOPs Param (M) YOLOv8s SCB-Dataset11 80.5 85.8 65.1 8.1 3.007 YOLOv10n SCB-Dataset11 81.4 86.0 65.3 8.2 2.697 YOLOv11n SCB-Dataset11 82.6 86.7 65.6 6.3 2.583 YOLOv13n SCB-Dataset11 81.9 86.9 66.0 6.2 2.449 RT-DETR SCB-Dataset11 83.1 88.3 67.4 103.5 32.006 Mamba-YOLO SCB-Dataset11 82.6 87.8 66.9 11.8 5.986 SC-CAMamba SCB-Dataset11 83.3 88.6 67.9 8.0 3.166

[0144] Compared to all models, SC-CAMamba achieved a recall of 83.3%, which is 0.6% and 0.7% higher than the highest-ranking YOLO series models and the Mamba series models, respectively, indicating that SC-CAMamba has higher detection coverage for all targets. Compared to the YOLO series models, SC-CAMamba's average detection accuracy (mAP@50%) and mAP@95%) reached 88.6% and 67.9%, respectively, which is 1.7% and 1.9% higher than the latest YOLOv13n model, and 1.9% and 2.3% higher than the YOLOv11n model, indicating superior feature extraction and target localization capabilities. Compared to Mamba-YOLO, it is 0.8% and 1.0% higher, demonstrating stronger detection capabilities for small targets and dense scenes. While SC-CAMamba has increased parameters compared to the latest YOLOv8s, YOLOv10n, YOLOv11n, and YOLOv13n models, it still shows improved performance. Compared to RT-DETR and Mamba-YOLO models, SC-CAMamba achieves higher accuracy and parameter utilization while reducing the number of parameters. In terms of computational complexity, SC-CAMamba has a Scalar complexity of 8.0, falling between YOLOv10n and YOLOv13n, but boasts higher recall and average detection accuracy. Compared to Transformer and Mamba models like DETR and Mamba-YOLO, its computational complexity is significantly reduced, and SC-CAMamba delivers superior performance. SC-CAMamba utilizes a VCAF feature deep fusion module to address the high parameter count and computational complexity issues of traditional Transformer and Mamba models, significantly improving performance while reducing computational complexity. In summary, SC-CAMamba successfully achieves an optimal balance between speed, accuracy, and efficiency, offering both low model complexity and high detection accuracy.

[0145] 2.5 Ablation Experiment

[0146] To evaluate the effectiveness of the innovation of this invention, ablation experiments were conducted on SC-CAMamba, sequentially integrating CBT, SC3d2, and VCAF into the baseline model. Then, ablation experiments were performed on the self-built behavior recognition dataset SCB-Dataset11. Recall, mAP50, mAP75, mAP95, F1-score, inference time, and other metrics were evaluated. Table 3 shows the comparison of these metrics before and after integrating the innovative modules into SC-CAMamba. The aim is to analyze the specific impact of different modules on model performance.

[0147] This paper introduces CBT and CDT into classroom multi-object behavior recognition. A new activation function, TeLU, is integrated into the convolutional layer to replace the traditional SiLU function. Dynamic Tanh is used to replace the normalization layer, smoothing the negative interval and addressing the issue of feature loss (such as faces in shadows or occluded limbs) that traditional SiLU convolutional layers often suffer from during the negative interval phase. Particularly in complex classroom scenarios, the SC-CAMamba model incorporating CBT, despite a slight increase in parameters, achieves improvements of 0.9%, 0.9%, 0.8%, and 0.6% in Recall, F1-sore, mAP75, and mAP95, respectively. Simultaneously, the model's inference speed is reduced by 44.4%.

[0148] SC3d2 adapts to scene changes by using dynamic feature transformations during feature extraction, dynamically learning and adjusting feature strength to suit dynamic variations in dense scenes. During feature fusion, it suppresses redundant information through channel reconstruction convolutions and enhances feature interaction through channel recombination. In this study, it improved Recall, F1-sore, mAP75, and mAP95 performance by 2.0%, 1.5%, 1.5%, and 0.7%, respectively. Simultaneously, the model's inference speed was reduced by 39.6%.

[0149] VCAF, through the deep fusion of convolutional self-attention mechanism and state-space model, effectively solves the problem of balancing local and global information, thereby better capturing the detailed interactions in space. Systematically addressing these issues is a key technology for capturing information interactions and long-range dependencies between targets in classroom multi-target behavior recognition. In this study, although the number of parameters and inference time increased, they were significantly reduced compared to similar Transformer and Mamba models. Performance in Recall, F1-sore, mAP75, and mAP95 was also improved by 1.4%, 1.4%, 1.2%, and 1.2%, respectively.

[0150] The SC-CAMamba model, which integrates CBT, SC3d2, and VCAF modules, improved performance in Recall, mAP50, mAP75, and mAP95 by 2.2%, 1.9%, 2.8%, and 2.3%, respectively. Experimental results show that each component in SC-CAMamba significantly improves detection performance, and the model demonstrates excellent accuracy in classroom multi-target behavior recognition.

[0151] Table 3: Ablation experiments using the SC-CAMamba model

[0152] Exp CBT SC3d2 VCAF Recall (%) mAP50 (%) mAP75 (%) mAP95 (%) F1-score (%) Inference time (ms) GFLOPs 1 81.1 86.7 75.4 65.6 82.0 6.3 6.3 2 √ 82.0 87.2 76.2 66.2 82.9 3.5 6.4 3 √ 83.1 87.3 76.9 66.3 83.5 3.8 6.3 4 √ 82.5 87.6 76.6 66.8 83.4 6.6 7.8 5 √ √ √ 83.3 88.6 78.2 67.9 83.9 4.6 8.0

[0153] 2.6 Comparison of Experimental Results with Different Datasets

[0154] To evaluate the model's generalization ability, comparative experiments were conducted using the public dataset SCB-Dataset5. This dataset is a public dataset for classroom behavior recognition. Based on this dataset, single and scattered behaviors were integrated into a single classroom behavior recognition dataset with seven behaviors. This dataset includes 5686 images with 45578 labels, focusing on seven behaviors: raising hand, reading, writing, using a mobile phone, looking down, and lying on the table. The improved SC-CAMamba model proposed in this invention was used for training and testing. The results are shown in Table 4.

[0155] Table 4: Performance comparison of various models on the public action recognition dataset SCB-Dataset5

[0156] Model Species Precision (%) F1-score GFLOPs Param (M) YOLOv5s SCB-Dataset5 74.6 73.0 7.1 2.504 YOLOv8s SCB-Dataset5 74.8 74.3 8.1 3.007 YOLOv10n SCB-Dataset5 74.9 72.9 8.2 2.697 YOLOv11n SCB-Dataset5 74.7 73.6 6.3 2.583 YOLOv13n SCB-Dataset5 74.7 73.5 6.2 2.449 RT-DETR SCB-Dataset5 75.4 72.9 103.5 31.998 Mamba-YOLO SCB-Dataset5 74.3 72.7 13.6 5.985 SC-CAMamba SCB-Dataset5 76.7 74.5 8.0 3.165

[0157] In the SCB-Dataset5 benchmark tests, each model showed a certain performance gap compared to SC-CAMamba. In terms of accuracy, it was 1.3% higher than the second-place RT-DETR and 1.8% higher than the best mainstream YOLO series models. This indicates that in action recognition tasks, SC-CAMamba has a higher proportion of true positive predictions, effectively reducing the error of misclassifying actions as targets when they are not, which is crucial for the reliability of action recognition. In terms of F1-score, it was 1.8% higher than Mamba-YOLO and 0.9% higher than the best YOLO v11n model. Since the F1-score is the harmonic average of precision and recall, this result shows that SC-CAMamba reduced false positives without sacrificing false negatives, balancing the completeness and accuracy of action recognition. In terms of GFLOPS, it is only 1 / 13 of the high-precision model RT-DETR and far lower than Mamba-YOLO. Compared with the YOLO series, although it is higher than YOLOv13n, it has improved in both accuracy and F1-score. This shows that SC-CAMamba achieves a significant improvement in accuracy with a small increase in computation, avoiding the problem of high accuracy but explosive computational costs associated with models like RT-DETR, and is suitable for the real-time inference needs of edge devices. In terms of parameter count, it is only 1 / 10 of RT-DETR and about 46% lower than Mamba-YOLO's 5.98M. Although it is higher than the YOLO series models, the increase in parameter count is much smaller than the increase in accuracy. For example, it has 0.224M more parameters than YOLOv8s but only a 1.9% increase in accuracy.

[0158] 2.7 VCAF Channel Coefficient Experiment

[0159] Table 5: Comparison of Channel Coefficients in the SC-CAMamba Model on a Self-Built Action Recognition Dataset

[0160] Exp mAP50 (%) mAP75 (%) mAP95 (%) GFLOPs 1 1.0，0.0，0.0 87.8 77.0 67.1 10.6 2 0.0，1.0，0.0 88.1 77.5 67.2 10.9 3 0.5，0.5，0.0 87.6 76.6 66.5 8.1 4 0.4，0.4，0.2 87.6 76.8 66.9 7.6 5 0.6，0.3，0.1 86.9 75.3 65.6 8.0 6 0.3，0.6，0.1 88.6 78.2 67.9 8.0 7 0.2，0.7，0.1 87.4 76.5 66.5 8.0

[0161] While single-channel solutions relying solely on the second channel achieve similar accuracy, they incur excessive computational costs. Equal-weight fusion or other non-equal-weight solutions either lack sufficient accuracy or offer no significant efficiency advantage. The SC-CAMamba model, however, employs a coefficient allocation where the second channel is primary, the first channel secondary, and the third channel supplementary. This approach retains the core feature extraction capabilities of the second channel while enhancing accuracy at high confidence levels through the auxiliary fusion of the first and third channels, all while controlling computational load. Its GFLOPs are 8.0, significantly lower than the 10.6 and 10.9 of the first and second groups, respectively, making it less suitable for edge device deployment. Other multi-channel fusion solutions, compared to the third and fourth groups, have GFLOPs at a moderately low level, only 0.1 lower than the third group, but achieve a 1.4 higher accuracy (mAP95), resulting in a better balance between accuracy and efficiency.

[0162] 2.8 Fusion Experiment of Different Attention Levels

[0163] To verify the fusion performance of the VCAF module, the most commonly used attention modules, such as CBAM, HALO, and DLKA, as well as recently proposed attention modules, were selected to replace CASelf_Attention for comparative testing. Experimental results are as follows: Figure 9 As shown.

[0164] Experimental results show that while fusing CBAM attention significantly reduces the computational cost (GFLOPs), the model's detection performance also decreases by the same amount. In HALO and DLKA attention, while GFLOPs increase substantially, the detection performance also improves to the same extent, but not as significantly as CASelf_Attention, which achieves a substantial accuracy improvement with only a small increase in computational cost. CASelf_Attention replaces the high-complexity multiplication weight calculations of traditional attention mechanisms with low-complexity addition operations, resulting in a slight increase in computational cost, but far less than that of traditional attention and pure Mmaba mechanisms. This leads to a 2.3% improvement in mAP50 compared to CBAM, 1.6% in HALO, and 1.2% in DLKA, and a 1.8% improvement in F1-score compared to CBAM, HALO, and DLKA, respectively. This demonstrates the superiority and importance of CASelf_Attention in VCAF for multi-target behavior recognition in the classroom.

[0165] 2.9 Visualization of Detection Results

[0166] To better understand the detection results of the SC-CAMamba detection algorithm for various behavior categories in practical applications, this invention selects some representative student behavior images for visualization. Examples of images and detection results for different behavior categories are shown below. Figure 10 As shown.

[0167] The confidence scores of RT-DETR were generally between 0.75 and 0.91, with some bounding boxes showing significant fluctuations and confidence scores as low as 0.6. Furthermore, there were instances of duplicate or ambiguous category annotations. It also failed to completely cover standing detection boxes, resulting in low category-location matching. Mamba-YOLO's confidence scores were mostly between 0.83 and 0.93, showing greater stability than RT-DETR overall, but some annotations were still present at the edges, leading to missed detections. YOLOv13 and YOLOv11 exhibited uneven confidence distributions, with some detection boxes showing low confidence and a higher probability of mislabeling. SC-CAMamba, on the other hand, had higher overall confidence scores, with no category confusion in all annotations. Its category prediction accuracy and confidence stability were significantly better than other models, and its detection boxes perfectly matched the outline of standing individuals, resulting in superior joint accuracy of category and location.

[0168] As can be seen from the example detection results, the improved model SC-CAMamba can effectively detect and locate multi-target behaviors in the classroom, achieving excellent results in both behavior detection and classification.

[0169] In summary, this invention proposes a multi-target behavior recognition model (SC-CAMamba model) for complex classroom scenarios. The model utilizes key modules such as a dynamic feature transformation module (CBT, CDT), feature transformation and reconstruction (SC3d2), and a deep feature fusion module (VCAF). The CBT module combines convolution operations with hyperbolic tangent exponential linear units (TeLUs), capturing edge features of behavior through local convolution while leveraging the non-linearity of TeLUs to enhance weak feature responses, thus avoiding feature loss due to gradient vanishing in the negative interval. Subsequently, SC3d2 integrates spatial and channel reconstruction convolutions with dynamic feature transformations through a multi-branch structure to adapt to pose changes and scene variations across multiple scales. Finally, VCAF performs deep fusion of multi-scale features, aggregating key information by introducing long-term temporal modeling from Mamba and convolutional additive self-attention, resolving semantic confusion of behaviors with highly similar visual appearances, and removing redundancy from long-term static features while preserving details from short-term dynamic features.

[0170] Experimental results show that SC-CAMamba achieves excellent results on a self-built classroom behavior recognition dataset containing 11 typical teaching behaviors. This method provides an efficient and reliable behavior detection scheme for multi-target behavior recognition in classroom scenarios, and provides technical support for real-time behavior monitoring and teaching quality evaluation in smart classrooms.

Claims

1. A classroom behavior detection model based on a state-space model and parallel self-attention, denoted as SC-CAMamba model, characterized in that, The SC-CAMamba model internally uses a feature extraction and dynamic feature transformation module (CBT, CDT), a feature transformation and reconstruction module (SC3d2), and a feature deep fusion module (VCAF). The feature extraction and dynamic feature transformation module (CBT, CDT) combines convolution operations with hyperbolic tangent exponential linear units (TeLU). While capturing behavioral edge features through local convolution, it utilizes the non-linearity of TeLU to enhance weak feature responses, avoiding feature loss due to gradient vanishing in the negative interval. The Feature Transformation and Reconstruction Module (SC3d2) integrates spatial and channel reconstruction convolutions and dynamic feature transformations through a multi-branch structure to adapt to pose changes and scene variations of multi-scale features. The Feature Deep Fusion Module (VCAF) performs deep fusion of multi-scale features. By introducing long-term temporal modeling of Mamba and fusion of convolutional addition self-attention, it achieves the aggregation of key information, solves the semantic confusion of behaviors with highly similar visual representations, removes redundancy from long-term static features and preserves details of short-term dynamic features.

2. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 1, characterized in that, The Feature Deep Fusion (VCAF) module is specifically divided into three stages: a global semantic channel, a local spatial channel, and an identity preservation channel. It concatenates features of different scales output from the feature extraction network as input features, with a feature map size of X. in ∈R B×Cin×H×W Where B represents the batch number, and C represents the batch number. in H and W are the input channel numbers, and H and W are the width and height of the feature map, respectively. The feature map X is processed according to the preset channel coefficients. The feature maps are divided into three sub-feature maps, including µ. , , After passing through the VCAF deep fusion layer, the feature map becomes X. out ∈R B×Cout×H×W C out This represents the number of output channels.

3. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 2, characterized in that, In the global semantic channel stage of the Feature Deep Fusion Module (VCAF), the construction steps are as follows: First, a lightweight VSS unit (VDSSBlock) is designed, comprising a three-level substructure of dynamic feature transformation, selective scan modeling, and feedforward extraction. The input features are then transformed from... Expand to ,in The number of input channels for the global branch is used, and dynamic DyTL is employed to adjust the dynamic behavior boundary of the inter-frame difference through adaptive threshold adjustment, while suppressing background noise. Then, the selective scanning unit SS2D is introduced to model the temporal correlation of classroom behavior with linear time complexity (O(N)). This unit is designed with scanning paths in four directions to capture temporal features to adapt to the typical action directions of classroom behavior. Finally, the temporal features after scanning are extracted by batch normalization and feedforward network (FFN). FFN adopts a lightweight structure of 1×1 convolution + TeLU activation + 1×1 convolution. It utilizes the nonlinear transformation of TeLU to enhance semantic association features and suppress irrelevant background features. The definition formula for the VDSSBlock module is given by formulas (1) and (2): in, This indicates that a split operation is performed on the feature, and the number of channels is taken. Take before Characteristics of channel number This indicates that a split operation is performed on the feature, and the number of channels is taken. After taking Characteristics of the number of channels.

4. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 3, characterized in that, In the local spatial channel stage of the Feature Deep Fusion (VCAF) module, convolutional additive self-attention (CASelf_Attention) based on multiple information interactions in the spatial and channel domains is adopted. A convolutional additive token mixer is introduced to replace the query-key matrix multiplication to calculate token relevance. Three feature branches are decomposed using QKV, and spatial channel attention enhancement is applied to Q and K respectively. The spatial attention module generates a weight map of the spatial dimension, and filters spatial locations through multiplication, strengthening key regions such as edges and textures. A weight value is generated for each channel, and channels are filtered through multiplication, strengthening important channels and suppressing redundant channels. Finally, depthwise separable convolution processing transforms the original classroom image into highly discriminative, low-redundancy behavioral features. The spatial and channel domains in CASelf_Attention are defined by equations (3) and (4): in, This represents the Sigmoid activation function. (·) represents the average pooling AvgPool with an average pooling of 1×1.

5. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 4, characterized in that, In the identity preservation channel stage of the Feature Deep Fusion Module (VCAF), Dynamic Thresholding Convolution (CDT) is used as the core unit. Through "convolution + adaptive threshold activation", noise is filtered while preserving static pose features. The definition of the identity preservation channel is shown in formula (5): First, the features from the three channels are fused and refined. The output features of the three branches are first obtained by channel concatenation to obtain the fused features. Subsequently, the feature extraction unit of the hyperbolic tangent exponential linear unit is used to further enhance the cross-branch semantic association; Finally, a 1×1 convolution is used to compress the number of concatenated channels to the target output channel Cout, and a BN layer is used to standardize the feature distribution to avoid information imbalance caused by differences in feature scale between branches.

6. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 5, characterized in that, The feature transformation and reconstruction module (SC3d2) performs feature extraction and feature interaction in different network layers, as shown in formula (7): in, This indicates a compound operation of the function, in which the CBT operation on the right is executed first, and then the result is input into the Concat operation on the left. SC3d2 first expands the input feature channels through CBT convolution, dividing the features into shallow information features that are directly passed and deep features that are processed in depth. The shallow features are directly passed to the next layer to preserve the background features, while the deep features are processed in depth to extract action features. The spatial reconstruction convolutional SRU is processed as shown in equations (8) and (9), where the input features x are grouped and normalized; and through... Adjust the normalization value to balance the importance weights of each channel; then subtract the informative features from the original features to obtain x2 of non-informative features that are relatively minor; finally, merge the two features along the channel dimension to enhance the feature representation. in , for The channel is split in two. for The channel is split in two. Here, g is the sigmoid function, and g is the number of groups. Learnable scaling parameters; The Channel Reconstruction Unit (CRU) splits the features into high-channel and low-channel parts, and then uses 1×1 convolution to compress the channels to reduce the amount of computation while fusing cross-channel information. Subsequently, group convolution and point convolution are used to enhance local feature extraction and linear transformation. Finally, global average pooling is used to compress the feature map to capture global channel information, and softmax is used to highlight important channels to weight the original feature Y before adding them together for output. The processing of spatial reconstruction convolutional SRU is shown in Equations (10) and (11); Where Y represents the original feature Y for attention weighting, and GAP(·) represents global average pooling.

7. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 6, characterized in that, In the feature extraction and dynamic feature transformation modules (CBT, CDT), the CBT module combines convolution operations with the hyperbolic tangent exponential linear unit (TeLU) for basic feature extraction in classroom behavior recognition. It captures the edge features of behavior through local convolution, and at the same time uses the non-linear characteristics of TeLU to enhance the weak feature response. The mathematical expression of the hyperbolic tangent exponential linear unit TeLU is shown in formula (12): When x > 0, the TeLU activation function exp(x) increases rapidly with increasing x, and tanh(e x If ) ≈ 1, then TeLU(x) ≈ x and strong features are preserved; When x ≈ 0, e x ≈ 1,tanh(e x When x ≈ 0.76, TeLU(x) ≈ 0.76x, which enhances the weak characteristic response through multiplication coefficients; when x > 0, e x It decays rapidly as x decreases, tanh(e x ) ≈ 0, at this time TeLU(x) ≈0, and the noise feature is suppressed.

8. The classroom behavior detection model based on state-space model and self-attention parallelism as described in claim 7, characterized in that, The Dynamic Feature Transformation Unit (DyTL) achieves adaptive capture and optimization of different behavioral features through standard convolution to extract global features, nonlinear enhancement of TeLU, and dynamic normalization of DyTL. DyTL introduces DyT (Dynamic Tanh Normalization) and the hyperbolic tangent exponential linear unit TeLU, and dynamically adjusts the feature distribution through the science department threshold. The mathematical expression is shown in formula (13). When the classroom background is complex, θ automatically increases to enhance the normalization intensity and filter noise; when the background is simple, θ decreases to retain more detailed features. θ is optimized through backpropagation and eventually converges to the optimal value that fits the specific behavior category; Where θ is the learnable threshold, initially set to 0.

85. To minimize the value, avoid the divisor θ being 0.

9. A method for detecting classroom behavior based on the model described in any one of claims 1-8, characterized in that, After constructing the model according to the model structure, the SC-CAMamba model is obtained through training, reinforcement and validation of the dataset. After inputting the image to be detected into the SC-CAMamba model, the model analyzes and outputs the results to achieve the recognition of multi-target behavior.

Images