Interactive teaching behavior identification method for classroom video

Abstract

The invention discloses an interactive teaching behavior identification method for a classroom video, and the method comprises the steps: employing a spatial-temporal feature extraction module which comprises a spatial feature extraction branch and a time feature extraction branch at a network structure level, and enabling the spatial feature extraction branch to be embedded with a spatial reconstruction unit; hierarchical screening is carried out on the feature map through a channel dimension weight adaptive mechanism, and redundant information is suppressed while key space features are strengthened; the time characteristic branch constructs a time sequence focusing unit, a multi-head self-attention module is introduced to establish an inter-frame long-time-history correlation model, and dynamic time-dimension attention distribution is realized to capture a teaching interaction time sequence mode. And the refined spatial features and the long-range time features respectively extracted by the two branches are complementarily integrated through a lightweight fusion module to complete the recognition and classification of the classroom teaching interaction state. According to the method, the robustness of illumination variation and complex scenes in a classroom is remarkably enhanced, and the teaching behavior research cost of traditional methods such as questionnaire survey and interview is effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence education technology, and in particular relates to a method for identifying interactive teaching behaviors for classroom videos. Background Art

[0002] Traditional classroom teaching scenarios usually contain multimodal interaction elements: verbal communication between teachers and students, body movements, writing on the blackboard, and operating teaching aids, etc., together constitute a dynamic teaching semantic space. However, the application of existing visual behavior recognition technology in classroom teaching scenarios faces the following core challenges: (1) Complex environmental interference: The dense distribution of people, high-frequency changes in interactive subjects, and multiple types of static background targets in the classroom environment make it difficult for traditional 3D convolutional models to effectively distinguish between behavioral subjects and environmental noise, and are easily interfered by redundant features; (2) Dynamic time series modeling defects: When dealing with non-continuous interactive behaviors, traditional 3D convolutional networks lack the ability to model long-term dependencies and are therefore unable to capture the key frame correlation features distributed in the time dimension; (3) Computational efficiency bottleneck: The high parameter count design of existing behavior recognition networks cannot adapt to the real-time analysis needs of the classroom.

[0003] Traditional classrooms typically include numerous teachers and students, along with surrounding backgrounds such as blackboards, desks, walls, and windows. These elements together create a highly complex and dynamically changing scene, posing significant challenges for identifying interactive teaching behaviors. To effectively identify interactive teaching behaviors within this complex classroom environment, a method for identifying interactive teaching behaviors in classroom videos is urgently needed. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a method for identifying interactive teaching behaviors in classroom videos, which significantly enhances the robustness to lighting changes and complex scenes in the classroom. It can provide technical support for digital classroom analysis and teaching effectiveness evaluation, and effectively reduce the cost of using traditional methods such as questionnaires and interviews to study teaching behaviors.

[0005] To achieve the above objectives, the present invention provides a method for identifying interactive teaching behaviors in classroom videos, comprising:

[0006] S1. Construct a classroom interaction behavior recognition model;

[0007] S2. Obtain a video dataset of classroom interactive teaching behaviors;

[0008] S3, processing the video dataset of the classroom interactive teaching behavior to obtain a processed video dataset, and dividing the processed video dataset into a training set and a validation set in a ratio of 4:1;

[0009] S4, performing shallow feature extraction on the training set to obtain shallow features of the video data;

[0010] S5. Extracting spatiotemporal features from the shallow features of the video data to obtain temporal features and spatial features of the video data;

[0011] S6. Performing lightweight fusion processing on the temporal features and spatial features of the video data to obtain deep temporal and spatial features of the video data;

[0012] S7, using the deep spatiotemporal features of the video data through a behavior classifier to identify classroom interaction behaviors;

[0013] S8. Setting a loss function, training the classroom interaction behavior recognition model, and obtaining training results;

[0014] S9. Verify the training results using the validation set and record the model parameters with the best performance;

[0015] S10. Repeat S4-S9 until the set number of iterations is reached to obtain the optimal classroom interaction behavior recognition model.

[0016] S11. Obtain a video dataset of classroom interactive teaching behaviors to be tested, and input the optimal classroom interactive behavior recognition model to perform interactive teaching behavior recognition.

[0017] Optionally, performing shallow feature extraction on the processed video data set to obtain shallow features of the video data includes: successively inputting the input 5D tensor into a first sequence consisting of a 7×7×7 separation convolution layer, a maximum pooling layer, and a 1×1×1 base convolution layer;

[0018] The first sequence is input to the second sequence consisting of a 3×3×3 separation convolution layer and a maximum pooling layer.

[0019] Optionally, in the process of performing spatiotemporal feature extraction on the shallow features of the video data, the temporal feature extraction branch includes: downsampling the input features in spatial dimension and generating a query matrix Q, a key matrix K and a value matrix V;

[0020] Split Q, K, V into h attention heads and calculate the time dimension self-attention in parallel;

[0021] Compress the spatial dimension through global average pooling and generate the temporal weight vector A;

[0022] Multiply A by the input feature map element by element to achieve dynamic weighting of temporal features.

[0023] Optionally, the self-attention calculation is split into 8 independent attention heads, and the attention output is restored to the original dimension after cross-head feature concatenation.

[0024] Optionally, in the process of extracting spatiotemporal features from the shallow features of the video data, the spatial feature extraction branch includes: a sequentially connected 1×1×1 basic convolution layer, a 1×3×3 basic spatial convolution layer, a spatial reconstruction unit and a 3×1×1 basic temporal convolution layer. The spatial reconstruction unit suppresses spatial redundant information through channel dimension weight separation and cross-reconstruction strategy.

[0025] Optionally, the implementation process of the space reconstruction unit includes:

[0026] Perform group normalization on input features and calculate channel weights based on trainable scaling factors;

[0027] Decomposing the channel weight into a high-weight feature map and a low-weight feature map through a threshold gating strategy;

[0028] Multiplying the original feature map with the high-weight feature map and the low-weight feature map respectively to generate information-rich features and information-redundant features;

[0029] Performing cross-channel segmentation and feature summation operations on the information-rich features and the information-redundant features to generate a first fusion feature and a second fusion feature;

[0030] The first fusion feature and the second fusion feature are concatenated in the channel dimension to generate a reconstructed output feature.

[0031] Optionally, performing lightweight fusion processing on the temporal features and spatial features of the video data to obtain deep temporal and spatial features of the video data includes:

[0032] Performing local feature dimensionality reduction on the temporal features and spatial features of the video data by a first feature processing branch to obtain a first result;

[0033] Performing spatiotemporal joint feature extraction on the temporal features and spatial features of the video data through a second feature processing branch to obtain a second result;

[0034] Performing downsampling and compression processing on the temporal features and spatial features of the video data through a third feature processing branch to obtain a third result;

[0035] The first result, the second result, and the third result are combined according to the channel dimension to generate deep spatiotemporal features of the video data.

[0036] Optionally, using the deep spatiotemporal features of the video data to identify classroom interactive behaviors through a behavior classifier includes: performing a global average pooling operation on the input feature map to compress the feature tensor into a pooling vector;

[0037] The pooled vector is input into the first fully connected layer, and the hidden layer feature vector is generated after ReLU nonlinear activation;

[0038] The hidden layer features are mapped to a classification space with a preset number of categories through the second fully connected layer;

[0039] The Softmax function is applied to generate the probability distribution of behavior categories to realize interactive behavior recognition.

[0040] Technical effect of the present invention: The present invention discloses a method for identifying interactive teaching behaviors in classroom videos, which can identify interactive teaching behaviors in classroom videos to be identified, and effectively improve the recognition accuracy under the conditions of low computational complexity and small memory overhead; the present invention uses a spatiotemporal feature extraction module and a lightweight fusion module to fully explore the spatiotemporal context associations in the teaching scene while ensuring efficient use of computing resources; in the shallow network stage, the multi-branch collaborative spatiotemporal feature extraction module can capture local body movement details such as students raising their hands and teachers writing on the blackboard, and can also use the multi-head self-attention mechanism in the temporal focusing unit to establish long-range temporal associations for intermittent teaching events; in the deep network stage, the lightweight fusion module compresses the feature interaction process of spatial flow and temporal flow into an equivalent single-path calculation through structural reparameterization technology, while retaining key interactive semantics in the global features. In short, the present invention significantly enhances the robustness to lighting changes and complex scenes in the classroom, can provide technical support for digital classroom analysis and teaching effect evaluation, and effectively reduces the cost of using traditional methods such as questionnaires and interviews to study teaching behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0042] Figure 1 The figure is a flow chart of a method for identifying interactive teaching behaviors in classroom videos according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] This embodiment provides a method for identifying interactive teaching behaviors in classroom videos, including:

[0046] S1. Construct a classroom interaction behavior recognition model;

[0047] S2. Obtain a video dataset of classroom interactive teaching behaviors;

[0048] S3, processing the video dataset of the classroom interactive teaching behavior to obtain a processed video dataset, and dividing the processed video dataset into a training set and a validation set in a ratio of 4:1;

[0049] S4, performing shallow feature extraction on the training set to obtain shallow features of the video data;

[0050] S5. Extracting spatiotemporal features from the shallow features of the video data to obtain temporal features and spatial features of the video data;

[0051] S6. Performing lightweight fusion processing on the temporal features and spatial features of the video data to obtain deep temporal and spatial features of the video data;

[0052] S7, using the deep spatiotemporal features of the video data through a behavior classifier to identify classroom interaction behaviors;

[0053] S8. Setting a loss function, training the classroom interaction behavior recognition model, and obtaining training results;

[0054] S9. Verify the training results using the validation set and record the model parameters with the best performance;

[0055] S10. Repeat S4-S9 until the set number of iterations is reached to obtain the optimal classroom interaction behavior recognition model.

[0056] S11. Obtain a video dataset of classroom interactive teaching behaviors to be tested, and input the optimal classroom interactive behavior recognition model to perform interactive teaching behavior recognition.

[0057] Further, performing shallow feature extraction on the processed video data set to obtain shallow features of the video data includes: successively inputting the input 5D tensor into a first sequence consisting of a 7×7×7 separation convolution layer, a maximum pooling layer, and a 1×1×1 base convolution layer;

[0058] The first sequence is input to the second sequence consisting of a 3×3×3 separation convolution layer and a maximum pooling layer.

[0059] Furthermore, in the process of extracting spatiotemporal features from the shallow features of the video data, the temporal feature extraction branch includes: downsampling the input features in spatial dimension and generating a query matrix Q, a key matrix K and a value matrix V;

[0060] Split Q, K, V into h attention heads and calculate the time dimension self-attention in parallel;

[0061] Compress the spatial dimension through global average pooling and generate the temporal weight vector A;

[0062] Multiply A by the input feature map element by element to achieve dynamic weighting of temporal features.

[0063] Furthermore, the calculation of self-attention is split into 8 independent attention heads, and the attention output is restored to the original dimension after cross-head feature splicing.

[0064] Furthermore, in the process of extracting spatiotemporal features from the shallow features of the video data, the spatial feature extraction branch includes: a sequentially connected 1×1×1 basic convolution layer, a 1×3×3 basic spatial convolution layer, a spatial reconstruction unit and a 3×1×1 basic temporal convolution layer. The spatial reconstruction unit suppresses spatial redundant information through channel dimension weight separation and cross-reconstruction strategy.

[0065] Furthermore, the implementation process of the space reconstruction unit includes:

[0066] Perform group normalization on input features and calculate channel weights based on trainable scaling factors;

[0067] Decomposing the channel weight into a high-weight feature map and a low-weight feature map through a threshold gating strategy;

[0068] Multiplying the original feature map with the high-weight feature map and the low-weight feature map respectively to generate information-rich features and information-redundant features;

[0069] Performing cross-channel segmentation and feature summation operations on the information-rich features and the information-redundant features to generate a first fusion feature and a second fusion feature;

[0070] The first fusion feature and the second fusion feature are concatenated in the channel dimension to generate a reconstructed output feature.

[0071] Furthermore, the temporal and spatial features of the video data are subjected to lightweight fusion processing to obtain deep temporal and spatial features of the video data, including:

[0072] Performing local feature dimensionality reduction on the temporal features and spatial features of the video data by a first feature processing branch to obtain a first result;

[0073] Performing spatiotemporal joint feature extraction on the temporal features and spatial features of the video data through a second feature processing branch to obtain a second result;

[0074] Performing downsampling and compression processing on the temporal features and spatial features of the video data through a third feature processing branch to obtain a third result;

[0075] The first result, the second result, and the third result are combined according to the channel dimension to generate deep spatiotemporal features of the video data.

[0076] Furthermore, using the deep spatiotemporal features of the video data to identify classroom interactive behaviors through a behavior classifier includes: performing a global average pooling operation on the input feature map to compress the feature tensor into a pooling vector;

[0077] The pooled vector is input into the first fully connected layer, and the hidden layer feature vector is generated after ReLU nonlinear activation;

[0078] The hidden layer features are mapped to a classification space with a preset number of categories through the second fully connected layer;

[0079] The Softmax function is applied to generate the probability distribution of behavior categories to realize interactive behavior recognition.

[0080] Specifically, such as Figure 1 As shown, the implementation process of this embodiment includes the following steps:

[0081] S1: Constructing a classroom interaction behavior recognition network based on 3D convolution;

[0082] S2: Obtain a video training dataset of classroom interactive teaching behaviors;

[0083] S3: Preprocess and enhance the dataset;

[0084] S4: The shallow feature extraction module performs preliminary processing on the video data and extracts shallow features, as follows:

[0085] For a given teaching video input I, a shallow feature extraction module H is used, which consists of a 7×7×7 separation convolution layer, a 3×3×3 maximum pooling layer, a 1×1×1 base convolution layer, a 3×3×3 separation convolution layer and a 3×3×3 maximum pooling layer. SFE Perform preliminary feature extraction and dimensionality reduction on the input data to generate shallow features F′. The process is expressed as:

[0086] F′=H SFE (I);

[0087] S5: Input shallow features into the spatiotemporal feature extraction module to extract the temporal and spatial features of the video. The spatiotemporal feature extraction module includes a spatial feature extraction branch and a temporal feature extraction branch, as follows:

[0088] The first branch H b1The input features are reduced in dimension through the 1×1×1 basic convolution layer, which retains the main spatial and temporal information while extracting low-dimensional local features. b2 Mainly responsible for the task of extracting spatial information. Specifically, this branch first uses a 1×1×1 basic convolution layer for dimensionality reduction, then uses the separation convolution method to use a 1×3×3 basic spatial convolution layer to extract features in the spatial dimension, capture the local spatial information in a single frame image frame by frame, and use the spatial reconstruction unit to separate and reconstruct the feature map space, thereby suppressing redundant information in the space and enhancing the feature expression of key areas. Finally, a 3×1×1 basic temporal convolution layer is used to integrate spatial information and capture local short-term actions. The third branch H b3 It focuses on capturing contextual information in the time dimension. First, it uses a 1×1×1 base convolution layer for dimensionality reduction, then uses a 3×3×3 separation convolution layer to extract complex spatiotemporal features, and then uses the temporal focus unit to dynamically adjust the feature weights of key frames to capture long-term action sequences in the video. The fourth branch H b4 The feature map is downsampled through max pooling to retain the most significant feature information, and then further dimensionality reduction is performed through a 1×1×1 base convolution layer. Finally, the features extracted by all branches are concatenated together to capture spatiotemporal information at different scales and levels, enhancing the model's ability to handle complex dynamic scenes while maintaining reasonable computational complexity. The process is expressed as follows:

[0089] F b1 =H b1 (F′1);

[0090] F b2 =H b2 (F′2);

[0091] F b3 =H b3 (F′3);

[0092] F b4 =H b4 (F′4);

[0093] F b =Concat{F b1 ,F b2 ,F b3 ,F b4};

[0094] a) Spatial reconstruction unit:

[0095] i. For the upper layer feature map input X i, using the trainable scaling factor γ in group normalization GN to evaluate the information richness of different feature maps and separate feature maps with rich information from feature maps with less information:

[0096]

[0097] ii. Normalize the parameters of γ and obtain the relevant weight W γ The importance of being able to effectively characterize different feature maps:

[0098]

[0099] iii. Use W γ Xg is reweighted and mapped to the range of (0, 1) using the Sigmoid function. A threshold (set to 0.5 in the experiment) is then used for gating. Weights above the threshold are set to α (set to 0.8 in the experiment) and are considered to be informative weights W1. Weights below the threshold are set to (1-α) and are considered to be less informative weights W2. The specific formula is as follows:

[0100] W=Gate(σ(W γ ×GN(X g )));

[0101] iv. Input feature X i Multiply them with W1 and W2 respectively to generate weighted information-rich features X1 and information-redundant features X2. After the separation operation is completed, the reconstruction operation is further performed. Specifically, SRU uses a cross reconstruction method to split X1 and X2 into X in the channel dimension. 11 、X 12 、X 21 and X 22 Then, X 11 and X 22 Add up to get Y1, X 21 and X 12 Add them together to get Y2. Y1 and Y2 are then concatenated back to the channel dimension to get the output feature Y.

[0102] b) Timing Focus Unit:

[0103] i. For the upper layer feature map input X i , the feature representation of each video frame is evenly distributed after group normalization, and the spatial information of each frame is linearly combined through a 1×1 convolution operation:

[0104] Q=XW Q , K=XW K , V=XW V ;

[0105] Where W Q , W K and W V It is a learning matrix used to generate QKV. Then the dot product Softmax of Q and K is calculated to obtain the similarity between them and the weighted summation with V is performed to obtain the output O. i :

[0106] O i =Attention(Q,K,V)=Softmax(Q·K)·V;

[0107] ii. Further, the Multi-Head Self-Attention (MHSA) mechanism is used to split Q, K, and V into 8 heads. Attention A is calculated in parallel across multiple heads to capture temporal dependencies at different scales:

[0108] A=σ(GAP(MHSA(Q,K,V)))∈R BC×T×1×1 ;

[0109] iii. The generated temporal attention A is applied to the feature map X i The weighted operation is as follows:

[0110] X A =A·X i ;

[0111] S6: Input the spatial and temporal features of the video into the lightweight fusion module to extract the deep spatiotemporal features of the video, as follows:

[0112] The first branch F l1 and the third branch F l3 The processing ideas and methods of the first and fourth branches of the spatiotemporal feature extraction module are the same. The second branch F l2 This is a lightweight merge of the second and third branches of the spatiotemporal feature extraction module. Specifically, this branch uses a 1×1×1 base convolutional layer to reduce the dimensionality of the input features, and then uses a 3×3×3 separating convolutional layer to directly extract the spatiotemporal information of the feature map. Finally, the three branches are spliced ​​together to capture both local details and global context. The process is expressed as follows:

[0113] F l1 =H l1 (F b1 );

[0114] F l2 =H l2 (F b2 );

[0115] F l3 =Hl3 (F b3 );

[0116] F l =Concat{F l1 ,F l2 , F l3};

[0117] S7: Deep spatiotemporal features are used to identify classroom interaction behaviors through behavior classifiers. Specifically:

[0118] This module first uses a global average pooling layer to transform the three-dimensional feature map F l It is compressed into a one-dimensional vector, and then two fully connected layers are used to map the one-dimensional feature vector to the teaching behavior category space. During this period, the ReLU activation function is introduced to enhance its nonlinear expression ability. Finally, the Softmax function is applied to generate the behavior category probability distribution to realize the interactive behavior recognition Y.

[0119] S8: Set a loss function to train the classroom interaction behavior recognition network. Specifically, the loss function is a cross entropy loss function:

[0120]

[0121] S9: Use the validation dataset to verify the network training results and record the network model parameters with the best performance;

[0122] S10: Repeat S4-S9 until the set number of iterations is reached to obtain the network model parameters with optimal performance.

[0123] S11: Using the optimal performance network model to identify classroom interaction behaviors.

[0124] The present invention discloses a method for identifying interactive teaching behaviors in classroom videos. The method can effectively improve the recognition accuracy under the conditions of low computational complexity and small memory overhead by identifying interactive teaching behaviors in classroom videos. The present invention fully mines the spatiotemporal context associations in teaching scenes while ensuring efficient use of computing resources through a spatiotemporal feature extraction module and a lightweight fusion module. In the shallow network stage, the multi-branch collaborative spatiotemporal feature extraction module can capture local body movement details such as students raising their hands and teachers writing on the blackboard, and can also use the multi-head self-attention mechanism in the temporal focusing unit to establish long-range temporal associations for interval teaching events. In the deep network stage, the lightweight fusion module compresses the feature interaction process of spatial flow and temporal flow into an equivalent single-path calculation through structural reparameterization technology, while retaining key interactive semantics in the global features. In short, the present invention significantly enhances the robustness to lighting changes and complex scenes in the classroom, can provide technical support for digital classroom analysis and teaching effect evaluation, and effectively reduces the cost of using traditional methods such as questionnaires and interviews to study teaching behaviors.

[0125] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for identifying interactive teaching behaviors in classroom videos, characterized in that: include: S1. Construct a classroom interaction behavior recognition model; S2. Obtain a video dataset of classroom interactive teaching behaviors; S3, processing the video dataset of the classroom interactive teaching behavior to obtain a processed video dataset, and dividing the processed video dataset into a training set and a validation set in a ratio of 4:1; S4, performing shallow feature extraction on the training set to obtain shallow features of the video data; S5. Extracting spatiotemporal features from the shallow features of the video data to obtain temporal features and spatial features of the video data; S6. Performing lightweight fusion processing on the temporal features and spatial features of the video data to obtain deep temporal and spatial features of the video data; S7, using the deep spatiotemporal features of the video data through a behavior classifier to identify classroom interaction behaviors; S8. Setting a loss function, training the classroom interaction behavior recognition model, and obtaining training results; S9. Verify the training results using the validation set and record the model parameters with the best performance; S10. Repeat S4-S9 until the set number of iterations is reached to obtain the optimal classroom interaction behavior recognition model. S11. Obtain a video dataset of classroom interactive teaching behaviors to be tested, and input the optimal classroom interactive behavior recognition model to perform interactive teaching behavior recognition.

2. The interactive teaching behavior recognition method for classroom videos according to claim 1, characterized in that: Performing shallow feature extraction on the processed video data set to obtain shallow features of the video data includes: successively inputting the input 5D tensor into a first sequence consisting of a 7×7×7 separation convolution layer, a maximum pooling layer, and a 1×1×1 base convolution layer; The first sequence is input to the second sequence consisting of a 3×3×3 separation convolution layer and a maximum pooling layer.

3. The interactive teaching behavior recognition method for classroom videos according to claim 1, characterized in that: In the process of extracting spatiotemporal features from the shallow features of the video data, the temporal feature extraction branch includes: downsampling the input features in spatial dimension and generating a query matrix Q, a key matrix K and a value matrix V; Split Q, K, V into h attention heads and calculate the time dimension self-attention in parallel; Compress the spatial dimension through global average pooling and generate the temporal weight vector A; Multiply A by the input feature map element by element to achieve dynamic weighting of temporal features.

4. The interactive teaching behavior recognition method for classroom videos according to claim 3, characterized in that: The calculation of self-attention is split into 8 independent attention heads, and the attention output is restored to the original dimension after cross-head feature splicing.

5. The interactive teaching behavior recognition method for classroom videos according to claim 1, characterized in that: In the process of extracting spatiotemporal features from the shallow features of the video data, the spatial feature extraction branch includes: a sequentially connected 1×1×1 basic convolution layer, a 1×3×3 basic spatial convolution layer, a spatial reconstruction unit and a 3×1×1 basic temporal convolution layer. The spatial reconstruction unit suppresses spatial redundant information through channel dimension weight separation and cross-reconstruction strategy.

6. The interactive teaching behavior recognition method for classroom videos according to claim 5, characterized in that: The implementation process of the space reconstruction unit includes: Perform group normalization on input features and calculate channel weights based on trainable scaling factors; Decomposing the channel weight into a high-weight feature map and a low-weight feature map through a threshold gating strategy; Multiplying the original feature map with the high-weight feature map and the low-weight feature map respectively to generate information-rich features and information-redundant features; Performing cross-channel segmentation and feature summation operations on the information-rich features and the information-redundant features to generate a first fusion feature and a second fusion feature; The first fusion feature and the second fusion feature are concatenated in the channel dimension to generate a reconstructed output feature.

7. The interactive teaching behavior recognition method for classroom videos according to claim 1, characterized in that: Lightweight fusion processing of the temporal and spatial features of the video data to obtain deep temporal and spatial features of the video data includes: Performing local feature dimensionality reduction on the temporal features and spatial features of the video data by a first feature processing branch to obtain a first result; Performing spatiotemporal joint feature extraction on the temporal features and spatial features of the video data through a second feature processing branch to obtain a second result; Performing downsampling and compression processing on the temporal features and spatial features of the video data through a third feature processing branch to obtain a third result; The first result, the second result, and the third result are combined according to the channel dimension to generate deep spatiotemporal features of the video data.

8. The interactive teaching behavior recognition method for classroom videos according to claim 1, characterized in that: Using the deep spatiotemporal features of the video data to identify classroom interactive behaviors through a behavior classifier includes: performing a global average pooling operation on the input feature map to compress the feature tensor into a pooling vector; The pooled vector is input into the first fully connected layer, and the hidden layer feature vector is generated after ReLU nonlinear activation; The hidden layer features are mapped to a classification space with a preset number of categories through the second fully connected layer; The Softmax function is applied to generate the probability distribution of behavior categories to realize interactive behavior recognition.

Images