Classroom behavior recognition method and device based on multi-scale dilated residual attention perception

Abstract

The present invention relates to the technical field of classroom behavior recognition, and in particular to a classroom behavior recognition method and device based on multi-scale dilated residual attention perception, which is as follows: collect classroom behavior images to construct a data set, annotate and enhance the data set, and then divide the training set and the verification set; construct a backbone network, input the image data in the training set into the backbone network features; construct a hybrid attention module, input the obtained feature input module to obtain features with attention perception; input the features to the part to obtain feature maps, and; input the feature maps, and to multiple decoupled prediction heads CBPH modules of yolov5 for multi-scale prediction, and finally locate the position of students in the classroom image, and identify the students' classroom behavior categories. The present invention can solve the problem of low recognition accuracy caused by inconsistent target scales in classroom scenes, and more effectively extract the discriminative features of classroom behaviors.

Description

Technical Field

[0001] The present invention relates to the technical field of classroom behavior recognition, and in particular to a classroom behavior recognition method and device based on multi-scale dilated residual attention perception. Background Art

[0002] Classroom student behavior analysis is an important means of evaluating classroom teaching quality, and it is also an effective way for teachers to obtain teaching feedback and improve teaching level. Traditional teaching quality evaluation is mainly based on manual observation methods.

[0003] It is time-consuming and labor-intensive, and cannot get timely teaching feedback, which affects the progress and quality of teachers' teaching. With the development of artificial intelligence technology, researchers use behavior recognition technology to analyze student behavior in classroom videos and judge students' classroom status and participation. The process of classroom behavior recognition mainly includes feature extraction and behavior classification, among which feature extraction is the most important link and is related to the performance of classroom behavior recognition. Feature extraction includes traditional manual methods and deep learning methods. The former extracts classroom behavior features based on manually designed algorithms, and the latter uses convolutional neural networks to automatically extract relevant features from input data, which not only saves complex manual design links, but also improves the accuracy of classroom behavior recognition. Student behavior detection is mainly completed by video action recognition, posture estimation and target detection methods. Although some methods have achieved remarkable results, behavior recognition methods in real classroom scenes still face some challenges. On the one hand, students' targets are often dense, resulting in occlusion between each other. This occlusion will cause the loss of behavior information, seriously affect the judgment ability of the model, may lead to incorrect behavior classification, and affect the analysis of the entire classroom atmosphere and learning status. On the other hand, the position of students usually leads to different scale differences. Students in the front row are usually closer to the camera and have higher image resolution, while students in the back row are farther away and have significantly lower resolution. The behavior of students in the back row may be misjudged due to blurred or low-resolution images, affecting the accuracy of overall behavior recognition.

[0004] Therefore, the present invention provides a classroom behavior recognition method and device based on multi-scale dilated residual attention perception to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a classroom behavior recognition method and device based on multi-scale dilated residual attention perception, which can solve the problem of low recognition accuracy caused by inconsistent target scales in classroom scenes and more effectively extract the discriminative features of classroom behaviors.

[0006] The technical solution of the present invention to solve the technical problem is a classroom behavior recognition method based on multi-scale dilated residual attention perception, which is as follows:

[0007] a) Collect classroom behavior images to build a dataset , then the dataset Divide into training set and validation set , and then for the training set and validation set The image data in the training set is annotated and enhanced to obtain the training set and validation set The annotation information corresponding to the classroom behavior images and the training set after data enhancement and validation set ;

[0008] b) Build Backbone network, The backbone network consists of the first downsampling layer, four Conv_C3 modules and The spatial pyramid pooling module uses the data-enhanced training set The image data in is input to In the backbone network, the characteristics of each stage are obtained, namely and ,feature Input to the SPPF module to get the yolov5 backbone network output ;

[0009] c) Build Hybrid Attention Module, The module includes grouping layers,

[0010] Channel Attention Module and position note , convolutional layer, the two modules are specifically divided into four parallel branches, and the features obtained are enter The module obtains attention-aware features ;

[0011] d) The backbone network also includes Neck part, Some of the structures used are feature pyramids and path aggregation networks , The part includes two up-sampling parts and two down-sampling parts. Input to Partial feature map , and ;

[0012] e) The feature map , and The images are input into multiple decoupled prediction head (CBPH) modules of yolov5 for multi-scale prediction, which ultimately locates the students’ positions in classroom images and identifies the students’ classroom behavior categories.

[0013] Step a) is as follows:

[0014] a-1) The dataset Divide into training sets in a ratio of 3:1 and validation set , , , ,in, Representation dataset The number of classroom behavior images in Representation dataset Middle classroom behavior images, Represents the training set The number of classroom behavior images in Represents the training set Middle classroom behavior images, , Represents the validation set The number of classroom behavior images in Represents the validation set Middle classroom behavior images, , ;

[0015] a-2) Use the labeling software LabelImg to label the training set and validation set The classroom behavior images in are annotated to obtain training sets and validation set The annotation information corresponding to the classroom behavior images in the training set includes the size, position and category of the target. The annotation information corresponding to the classroom behavior image in is , validation set The annotation information corresponding to the classroom behavior image is ,in, Represents the training set Middle The annotation information corresponding to each classroom behavior image, Represents the validation set Middle The annotation information corresponding to each classroom behavior image;

[0016] a-3) Yes and The classroom behavior images in the image are randomly affine transformed and horizontally flipped to obtain the training set after data enhancement. and validation set , Represents the training set after data enhancement Middle classroom behavior images, Represents the validation set after data augmentation Middle Image of classroom behavior.

[0017] Step b) is as follows:

[0018] b-1) The training set after data enhancement Classroom behavior images in Input to the first downsampling layer to get the initial feature map , the first downsampling layer includes convolutional layer, regularization layer and Activation function layer, where the convolution kernel size of the convolution layer is 6×6, the stride is 2, and the padding is 2;

[0019] b-2) Input feature T into the first Conv_C3 module to obtain feature , and then the feature Input to the second Conv_C3 module to get the features , and then the feature Input to the third Conv_C3 module to get the features ,Each Conv_C3 module consists of a convolutional layer and a C3 module;

[0020] b-3) The characteristics Input to the fourth Conv_C3 module to get the feature map ;

[0021] b-4) The feature map Input to The module performs multi-scale fusion and extraction to obtain features .

[0022] Step c) is as follows:

[0023] c-1) The four branches are used , , , express;

[0024] First branch Including global average pooling layer , two convolutional layers, and a reweighting layer - , where the convolution kernel size of the global average pooling layer is the input feature map of , stride 1, padding 0, the first convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is Function, the second convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is function, Indicates height, Indicates width;

[0025] Second branch Including grouping layer , ;

[0026] The third branch Including grouping layer , horizontal average pooling layer , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the horizontal average pooling layer is the input feature map , step size is 1, padding is 0, the convolution kernel size of the convolution layer is 1, step size is 1, and padding is 0;

[0027] The fourth branch Including grouping layer , vertical average pooling , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the vertical average pooling layer is the input feature map , stride is 1, padding is 0, and the convolution kernel size of the convolution layer is 1×1, stride is 1, and padding is 0.

[0028] c-2) The feature map Input to Modules:

[0029] c-2-1) Feature map Enter the first branch ,go through The global average pooling layer Get Features ,feature Input to The first convolutional layer Get Features ,feature Input to The second convolutional layer Get Features ,feature Input to The reweighted layer obtains the features containing attention weights ;

[0030] c-2-2) Feature map Input to , and Common grouping layer , feature map It is divided into g groups with the number of channels, and then g groups of feature maps are obtained , the g groups of feature maps Import to adjustment shape layer , reshape it to the original feature map Same shape, get features ;

[0031] c-2-3) Set g groups of feature maps Enter into the third branch Horizontal average pooling layer and the fourth branch Vertical average pooling layer , and obtain the characteristics and Features ,feature and Features Input to and Common stitching layer Concatenate in the second dimension to get the features ,feature Input to and Common convolutional layers Get Features , then the feature According to the original feature map in the second dimension of and Separate and obtain two feature maps and , the feature Input to The group normalization layer Get Features , the feature Input to The activation function layer Get features with attention weights , feature map Input to The group normalization layer Get Features ,feature Input to The activation function layer Get features with attention weights ;

[0032] c-2-4) The characteristics , , and Input to the reweighting layer First, the feature and Multiply to get a more comprehensive attention weight, and feature Multiply them together to get features with more comprehensive attention weights , and then and Multiply to further adjust the feature's attention weight and get the feature ;feature Input to the last convolutional layer Get the final output features .

[0033] Step d) is as follows:

[0034] d-1) The upsampling part includes convolutional layer, upsampling layer and Module, where the convolution kernel size of the convolution layer is 1×1, the stride is 1, the padding is 0, and the upsampling layer uses bilinear interpolation;

[0035] The downsampling part includes the convolutional layer and Module, where the convolution kernel size of the convolution layer is 3×3, the stride is 2, and the padding is 1;

[0036] The module includes two convolutional layers, a separation layer, indivual Multi-scale dilated convolution modules, splicing layers, settings , where each convolutional layer is followed by a regularization layer and Activation function, the convolution kernel size of the two convolutional layers is 1×1, the stride is 1, and the padding is 0;

[0037] The MDCM module consists of a three-layer structure and a residual connection. The first layer is composed of two parallel convolutional layers, each of which has a regularization layer and a ReLU activation function. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second convolutional layer is 1×1, the step size is 1, and the padding is 0. The second layer is composed of three parallel convolutional layers, each of which has a regularization layer and Activation function, where the convolution kernel size of the first convolution layer is 3×3, the step size is 1, the padding is 1, and the dilation rate is 1; the convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 3, and the dilation rate is 3; the convolution kernel size of the third convolution layer is 3×3, the step size is 1, the padding is 5, and the dilation rate is 5; the third layer structure consists of a convolution layer, a regularization layer, and an activation function layer, where the convolution kernel size of the convolution layer is 1×1, the step size is 1, the padding is 0, and the activation function is Activation function.

[0038] d-2) The characteristics Input to the convolution layer of the first upsampling part to get the features ,feature Input to the upsampling layer of the first upsampling part to get the features ,feature Features output from the backbone network Splicing to get features ;

[0039] feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module Divide it into two parts from the channel dimension and get the features and Features ,feature and Features The number of channels becomes the characteristic one-half of;

[0040] feature Input to The parallel convolutional layer of the first layer of the module structure obtains two different features, features and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. ,feature and Features , the feature ,feature and Features Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , and then the feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0041] Then the features Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, feature and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. , and , the feature , and Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , the feature and Features Add them together to get the second Output characteristics of the module ;

[0042] The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ;

[0043] d-3) Characteristics Input to the second upsampling part of Neck, feature Input to the convolution layer of the first upsampling part to get the features ,feature Input to the upsampling layer of the first upsampling part to get the features ,feature Features obtained with the backbone network Splicing to get features ,feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of

[0044] feature Input to The parallel convolutional layers of the first layer of the module structure obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the first Output characteristics of the module ;

[0045] feature Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the second Output characteristics of the module ;

[0046] The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ;

[0047] d-4) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of

[0048] feature Input to the parallel convolutional layer of the first layer of the MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0049] feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ;

[0050] feature , and Concatenate on the channel dimension to get features ,feature Input to the second convolution layer of the C2f_MDCM module to get the features ;

[0051] d-5) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of

[0052] feature Input to the parallel convolutional layer of the first layer of the MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0053] feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ;

[0054] feature , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature .

[0055] Step e) is as follows:

[0056] e-1) The features obtained from the Neck part of the yolov5 structure , and Input to the decoupled prediction head CBPH module, the CBPH module includes an inverted residual block and two parallel CBS modules. The inverted residual block consists of three convolutional layers. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second and third convolutional layers are both 1×1, the step size is 1, and the padding is 0. The CBS module includes a convolutional layer, a batch normalization layer, and a SiLU activation function layer, where the convolution kernel size is 1×1, the step size is 1, and the padding is 0;

[0057] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the small-scale target position coordinates and the behavior category to which the student belongs respectively;

[0058] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the target position and the behavior category to which the student belongs respectively;

[0059] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the large target location and the behavior category to which the student belongs;

[0060] e-2) Among them, the loss function of YOLOv5 includes classification loss , confidence loss and positioning loss , the total loss is ,in, , , They represent the balance coefficients corresponding to the three loss functions respectively. The classification loss and confidence loss use BCE loss binary cross entropy loss, and the positioning loss uses CIoU loss complete intersection-over-union loss.

[0061] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a classroom behavior recognition method based on multi-scale dilated residual attention perception.

[0062] The present invention also provides a computer device, comprising the following modules:

[0063] Memory: used to store computer programs / instructions;

[0064] Processor: used to execute computer programs and instructions to implement a classroom behavior recognition method based on multi-scale dilated residual attention perception.

[0065] The effects provided in the summary of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects:

[0066] A classroom behavior recognition method based on multi-scale dilated residual attention perception uses dilation rates and residual connections of different sizes to achieve multi-scale feature extraction, which can capture global and local behavior features to solve the problem of low recognition accuracy caused by inconsistent target scales in classroom scenarios; by adding the RFCA attention module proposed by the present invention to the original network structure, the model can focus on behavior features that are important for behavior recognition from different angles of channel and position, reduce the interference of irrelevant features, and more effectively extract the discriminative features of classroom behavior. In addition, the decoupling head proposed by the present invention is used to separate the classification task and the regression task to prevent mutual interference between different tasks, thereby further improving the performance of the network;

[0067] By improving the original YOLOv5 network, the present invention can solve the problems of severe occlusion and inconsistent change scale in classroom scenes, thereby more accurately locating the positions of students in classroom images and further analyzing the categories of students' classroom behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0069] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0070] In order to clearly illustrate the technical features of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0071] Example 1

[0072] like Figure 1 As shown, the present invention provides a classroom behavior recognition method based on multi-scale dilated residual attention perception, which is as follows:

[0073] a) Collect classroom behavior images to build a dataset , then the dataset Divide into training set and validation set , and then for the training set and validation set The image data in the training set is annotated and enhanced to obtain the training set and validation set The annotation information corresponding to the classroom behavior images and the training set after data enhancement and validation set ;

[0074] b) Build Backbone network, The backbone network consists of the first downsampling layer, four Conv_C3 modules and The spatial pyramid pooling module uses the data-enhanced training set The image data in is input to In the backbone network, the characteristics of each stage are obtained, namely and ,feature Input to the SPPF module to get the yolov5 backbone network output ;

[0075] c) Build Hybrid Attention Module, The module includes grouping layers,

[0076] Channel Attention Module and position note , convolutional layer, the two modules are specifically divided into four parallel branches, and the features obtained are enter The module obtains attention-aware features ;

[0077] d) The backbone network also includes Neck part, Some of the structures used are feature pyramids and path aggregation networks , The part includes two up-sampling parts and two down-sampling parts. Input to Partial feature map , and ;

[0078] e) The feature map , and The images are input into multiple decoupled prediction head (CBPH) modules of yolov5 for multi-scale prediction, which ultimately locates the students’ positions in classroom images and identifies the students’ classroom behavior categories.

[0079] Step a) is as follows:

[0080] a-1) The dataset Divide into training sets in a ratio of 3:1 and validation set , , , ,in, Representation dataset The number of classroom behavior images in Representation dataset Middle classroom behavior images, Represents the training set The number of classroom behavior images in Represents the training set Middle classroom behavior images, , Represents the validation set The number of classroom behavior images in Represents the validation set Middle classroom behavior images, , ;

[0081] a-2) Use the labeling software LabelImg to label the training set and validation set The classroom behavior images in are annotated to obtain training sets and validation set The annotation information corresponding to the classroom behavior images in the training set includes the size, position and category of the target. The annotation information corresponding to the classroom behavior image in is , validation set The annotation information corresponding to the classroom behavior image is ,in, Represents the training set Middle The annotation information corresponding to each classroom behavior image, Represents the validation set Middle The annotation information corresponding to each classroom behavior image;

[0082] a-3) Yes and The classroom behavior images in the image are randomly affine transformed and horizontally flipped to obtain the training set after data enhancement. and validation set , Represents the training set after data enhancement Middle classroom behavior images, Represents the validation set after data augmentation Middle Image of classroom behavior.

[0083] Step b) is as follows:

[0084] b-1) The training set after data enhancement Classroom behavior images in Input to the first downsampling layer to get the initial feature map , the first downsampling layer includes convolutional layer, regularization layer and Activation function layer, where the convolution kernel size of the convolution layer is 6×6, the stride is 2, and the padding is 2;

[0085] b-2) Input feature T into the first Conv_C3 module to obtain feature , and then the feature Input to the second Conv_C3 module to get the features , and then the feature Input to the third Conv_C3 module to get the features ,Each Conv_C3 module consists of a convolutional layer and a C3 module;

[0086] b-3) The characteristics Input to the fourth Conv_C3 module to get the feature map ;

[0087] b-4) The feature map Input to The module performs multi-scale fusion and extraction to obtain features .

[0088] Step c) is as follows:

[0089] c-1) The four branches are used , , , express;

[0090] First branch Including global average pooling layer , two convolutional layers, and a reweighting layer - , where the convolution kernel size of the global average pooling layer is the input feature map of , stride 1, padding 0, the first convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is Function, the second convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is function, Indicates height, Indicates width;

[0091] Second branch Including grouping layer , ;

[0092] The third branch Including grouping layer , horizontal average pooling layer , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the horizontal average pooling layer is the input feature map , step size is 1, padding is 0, the convolution kernel size of the convolution layer is 1, step size is 1, and padding is 0;

[0093] The fourth branch Including grouping layer , vertical average pooling , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the vertical average pooling layer is the input feature map , stride 1, padding 0, the convolution kernel size of the convolution layer is 1×1, stride 1, and padding 0.

[0094] c-2) The feature map Input to Modules:

[0095] c-2-1) Feature map Enter the first branch ,go through The global average pooling layer Get Features ,feature Input to The first convolutional layer Get Features ,feature Input to The second convolutional layer Get Features ,feature Input to The reweighted layer obtains the features containing attention weights ;

[0096] c-2-2) Feature map Input to , and Common grouping layer , feature map It is divided into g groups with the number of channels, and then g groups of feature maps are obtained , the g groups of feature maps Import to adjustment shape layer , reshape it to the original feature map Same shape, get features ;

[0097] c-2-3) Set g groups of feature maps Enter into the third branch Horizontal average pooling layer and the fourth branch Vertical average pooling layer , respectively get the characteristics and Features ,feature and Features Input to and Common stitching layer Concatenate in the second dimension to get the features ,feature Input to and Shared convolutional layers Get Features , then the feature According to the original feature map in the second dimension of and Separate and obtain two feature maps and , the feature Input to The group normalization layer Get Features , the feature Input to The activation function layer Get features with attention weights , feature map Input to The group normalization layer Get Features ,feature Input to The activation function layer Get features with attention weights ;

[0098] c-2-4) Features , , and Input to the reweighting layer First, the feature and Multiply to get a more comprehensive attention weight, and feature Multiply them together to get features with more comprehensive attention weights , and then and Multiply to further adjust the feature's attention weight and get the feature ;feature Input to the last convolutional layer Get the final output features .

[0099] Step d) is as follows:

[0100] d-1) The upsampling part includes convolutional layer, upsampling layer and Module, where the convolution kernel size of the convolution layer is 1×1, the stride is 1, the padding is 0, and the upsampling layer uses bilinear interpolation;

[0101] The downsampling part includes the convolutional layer and Module, where the convolution kernel size of the convolution layer is 3×3, the stride is 2, and the padding is 1;

[0102] The module includes two convolutional layers, a separation layer, indivual Multi-scale dilated convolution modules, splicing layers, settings , where each convolutional layer is followed by a regularization layer and Activation function, the convolution kernel size of the two convolutional layers is 1×1, the stride is 1, and the padding is 0;

[0103] The MDCM module consists of a three-layer structure and a residual connection. The first layer is composed of two parallel convolutional layers, each of which has a regularization layer and a ReLU activation function. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second convolutional layer is 1×1, the step size is 1, and the padding is 0. The second layer is composed of three parallel convolutional layers, each of which has a regularization layer and Activation function, where the convolution kernel size of the first convolution layer is 3×3, the step size is 1, the padding is 1, and the dilation rate is 1; the convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 3, and the dilation rate is 3; the convolution kernel size of the third convolution layer is 3×3, the step size is 1, the padding is 5, and the dilation rate is 5; the third layer structure consists of a convolution layer, a regularization layer, and an activation function layer, where the convolution kernel size of the convolution layer is 1×1, the step size is 1, the padding is 0, and the activation function is Activation function.

[0104] d-2) The characteristics Input to the convolution layer of the first upsampling part to get the features ,feature Input to the upsampling layer of the first upsampling part to get the features ,feature Features output from the backbone network Splicing to get features ;

[0105] feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module Divide it into two parts from the channel dimension and get the features and Features ,feature and Features The number of channels becomes the characteristic one-half of;

[0106] feature Input to The parallel convolutional layer of the first layer of the module structure obtains two different features, features and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. ,feature and Features , the feature ,feature and Features Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , and then the feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0107] Then the features Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, feature and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. , and , the feature , and Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , the feature and Features Add them together to get the second Output characteristics of the module ;

[0108] The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ;

[0109] d-3) Characteristics Input to the second upsampling part of Neck, feature Input to the convolution layer of the first upsampling part to get the features ,feature Input to the upsampling layer of the first upsampling part to get the features ,feature Features obtained with the backbone network Splicing to get features ,feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of;

[0110] feature Input to The parallel convolutional layers of the first layer of the module structure obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the first Output characteristics of the module ;

[0111] feature Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the second Output characteristics of the module ;

[0112] The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ;

[0113] d-4) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of;

[0114] feature Input to the parallel convolutional layer of the first layer of the MDCM module, two different features are obtained, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0115] feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ;

[0116] feature , and Concatenate on the channel dimension to get features ,feature Input to the second convolution layer of the C2f_MDCM module to get the features ;

[0117] d-5) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of;

[0118] feature Input to the parallel convolutional layer of the first layer of the MDCM module, two different features are obtained, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ;

[0119] feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ;

[0120] feature , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature .

[0121] Step e) is as follows:

[0122] e-1) The features obtained from the Neck part of the yolov5 structure , and Input to the decoupled prediction head CBPH module, the CBPH module includes an inverted residual block and two parallel CBS modules. The inverted residual block consists of three convolutional layers. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second and third convolutional layers are both 1×1, the step size is 1, and the padding is 0. The CBS module includes a convolutional layer, a batch normalization layer, and a SiLU activation function layer, where the convolution kernel size is 1×1, the step size is 1, and the padding is 0;

[0123] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the small-scale target position coordinates and the behavior category to which the student belongs respectively;

[0124] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the target position and the behavior category to which the student belongs respectively;

[0125] The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the large target location and the behavior category to which the student belongs;

[0126] e-2) Among them, the loss function of YOLOv5 includes classification loss , confidence loss and positioning loss , the total loss is ,in, , , They represent the balance coefficients corresponding to the three loss functions respectively. The classification loss and confidence loss use BCE loss binary cross entropy loss, and the positioning loss uses CIoU loss complete intersection-over-union loss.

[0127] Example 2

[0128] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a classroom behavior recognition method based on multi-scale dilated residual attention perception.

[0129] Example 3

[0130] A computer device, comprising the following modules:

[0131] Memory: used to store computer programs / instructions;

[0132] Processor: used to execute computer programs and instructions to implement a classroom behavior recognition method based on multi-scale dilated residual attention perception.

[0133] Example 4

[0134] As shown in Table 1, the present invention and other existing methods are compared on the data set STBD-08. The existing methods include the following target detection algorithms: YOLOv5, Faster R-CNN, TPH-YOLOv5, YOLOX and YOLOv7. The evaluation indicators are mean average precision mAP (%), average precision of small targets (%), the average accuracy of the target (%), average precision of large targets (%) and latency (ms). According to the experimental results, the present invention is superior to the existing methods in detection accuracy, and is nearly one point higher than the second-ranked YOLOX model, and 4.1% higher than the baseline model YOLOv5 model. The performance of the present invention in small target accuracy, medium target accuracy and large target accuracy is improved compared with the existing methods. Low latency is crucial for classroom behavior recognition tasks. The latency of the present invention is 31.2ms, second only to 30.3ms of YOLOv5. Therefore, the present invention further improves the performance of student classroom behavior recognition, and can also better handle classroom behavior recognition problems under situations with varying scales and severe occlusion.

[0135] Table 1 Comparison of experimental results between the method in the present invention and the existing method

[0136]

[0137] Example 5

[0138] The present invention relates to computer vision. In the practical application of the classroom behavior recognition method based on multi-scale dilated residual attention perception in the present invention, an experiment is conducted through a classroom behavior recognition device based on multi-scale dilated residual attention perception: a classroom image captured by a camera in the classroom is input into the device of the present invention, wherein the image contains a plurality of students, who are respectively sitting in different seats and exhibit various classroom behaviors, such as raising hands, listening to a class, writing, turning the body, discussing, and the like; first, the classroom image is subjected to data preprocessing such as random affine transformation and horizontal flipping to obtain a processed image, and then the processed image is input into a network architecture proposed by the present invention to locate the position of each student in the classroom image, and at the same time identify the behavior category to which the student belongs, and the system will frame a boundary box on each student to display the specific positions of the students, and will also give the classroom behavior category of the student in the upper left corner of the boundary box;

[0139] The system will capture classroom images from time to time and analyze each student's classroom behavior at that time.

[0140] Record each student's classroom behavior at different times and generate a detailed behavior log. Analyzing this log report can obtain the behavioral data of each student in this class, and analyze which students have a higher degree of classroom participation and which students may have problems such as inattention. If the system detects that a student turns his body for a long time or talks for a long time, the system will remind the teacher that the student may be distracted, thereby helping the teacher to grasp the classroom status of each student, give timely reminders, and improve the teaching effect of the entire class.

[0141] In summary, the method of the present invention can help teachers understand the classroom status of each student, provide teachers with timely teaching feedback, and improve teaching quality.

[0142] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A classroom behavior recognition method based on multi-scale dilated residual attention perception, characterized by: The following steps are involved: a) Collect classroom behavior images to build a dataset , then the dataset Divide into training set and validation set , and then for the training set and validation set The image data in the training set is annotated and enhanced to obtain the training set and validation set The annotation information corresponding to the classroom behavior images and the training set after data enhancement and validation set ; b) Build Backbone network, The backbone network consists of the first downsampling layer, four Conv_C3 modules and The spatial pyramid pooling module converts the training set The image data in is input to In the backbone network, get the features output by the first Conv_C3 module , the features output by the second Conv_C3 module , the features output by the third Conv_C3 module And the feature map output by the fourth Conv_C3 module , and then the feature Input to SPPF module, output features ; c) Build Hybrid Attention Module, The module includes grouping layer groups and channel attention module and position attention module , convolutional layer, the two modules are specifically divided into four parallel branches, and the features obtained are enter The module obtains attention-aware features ; Step c) is as follows: c-1) The four branches are used , , , express; First branch Including global average pooling layer , two convolutional layers, and a reweighting layer - , where the convolution kernel size of the global average pooling layer is the input feature map of , stride 1, padding 0, the first convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is Function, the second convolutional layer The convolution kernel size is 1×1, the step size is 1, the padding is 0, and the activation function is function, Indicates height, Indicates width; Second branch Including grouping layer , ; The third branch Including grouping layer , horizontal average pooling layer , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the horizontal average pooling layer is the input feature map , step size is 1, padding is 0, the convolution kernel size of the convolution layer is 1, step size is 1, and padding is 0; The fourth branch Including grouping layer , vertical average pooling , concatenation layer and convolution layer , group normalization layer , Activation function layer, where the convolution kernel size of the vertical average pooling layer is the input feature map , the step size is 1, the padding is 0, the convolution kernel size of the convolution layer is 1×1, the step size is 1, and the padding is 0; c-2) The feature map Input to Modules: c-2-1) Feature map Enter the first branch ,go through The global average pooling layer Get Features ,feature Input to The first convolutional layer Get Features ,feature Input to The second convolutional layer Get Features ,feature Input to The reweighted layer obtains the features containing attention weights ; c-2-2) Feature map Input to , and Common grouping layer , feature map It is divided into g groups with the number of channels, and then g groups of feature maps are obtained , the g groups of feature maps Import to adjustment shape layer , reshape it to the original feature map Same shape, get features ; c-2-3) Set g groups of feature maps Enter into the third branch Horizontal average pooling layer and the fourth branch Vertical average pooling layer , respectively get the characteristics and Features ,feature and Features Input to and Common stitching layer Concatenate in the second dimension to get the features ,feature Input to and Shared convolutional layers Get Features , then the feature In the second dimension, according to the original feature map of and Separate and obtain two feature maps and , the feature Input to The group normalization layer Get Features , the feature Input to The activation function layer Get features with attention weights , feature map Input to The group normalization layer Get Features ,feature Input to The activation function layer Get features with attention weights ; c-2-4) The characteristics , , and Input to the reweighting layer First, the feature and Multiply to get a more comprehensive attention weight, and feature Multiply them together to get features with more comprehensive attention weights , and then and Multiply to further adjust the feature's attention weight and get the feature ;feature Input to the last convolutional layer Get the final output features ; d) The backbone network also includes Neck part, Some of the structures used are feature pyramids and path aggregation networks , The part includes two up-sampling parts and two down-sampling parts. Input to Partial feature map , and ; e) The feature map , and The input is sent to multiple decoupled prediction head CBPH modules of yolov5 for multi-scale prediction, and finally the position of students in the classroom image is located and the category of students' classroom behavior is identified; Step e) is as follows: e-1) The features obtained from the Neck part of the yolov5 structure , and Input to the decoupled prediction head module CBPH. The CBPH module includes an inverted residual block and two parallel CBS modules. The inverted residual block consists of three convolutional layers. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second and third convolutional layers are both 1×1, the step size is 1, and the padding is 0. The CBS module includes a convolutional layer, a batch normalization layer, and a SiLU activation function layer. The convolution kernel size is 1×1, the step size is 1, and the padding is 0. The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the small-scale target position coordinates and the behavior category to which the student belongs respectively; The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the target position and the behavior category to which the student belongs respectively; The characteristics Input to the inverted residual block to get the output , and then the feature Input to two parallel convolutional layers to obtain tensors and , and They represent the coordinates of the large target location and the behavior category to which the student belongs; e-2) Among them, the loss function of YOLOv5 includes classification loss , confidence loss and positioning loss , the total loss is ,in, , , They represent the balance coefficients corresponding to the three loss functions respectively. The classification loss and confidence loss use BCE loss binary cross entropy loss, and the positioning loss uses CIoU loss complete intersection-over-union loss.

2. The classroom behavior recognition method based on multi-scale dilated residual attention perception according to claim 1 is characterized in that: Step a) is as follows: a-1) The dataset Divide into training sets in a ratio of 3:1 and validation set , , , ,in, Representation dataset The number of classroom behavior images in Representation dataset Middle classroom behavior images, Represents the training set The number of classroom behavior images in Represents the training set Middle classroom behavior images, , Represents the validation set The number of classroom behavior images in Represents the validation set Middle classroom behavior images, , ; a-2) Use the labeling software LabelImg to label the training set and validation set The classroom behavior images in are annotated to obtain training sets and validation set The annotation information corresponding to the classroom behavior images in the training set includes the size, position and category of the target. The annotation information corresponding to the classroom behavior image in is , validation set The annotation information corresponding to the classroom behavior image is ,in, Represents the training set Middle The annotation information corresponding to each classroom behavior image, Represents the validation set Middle The annotation information corresponding to each classroom behavior image; a-3) Yes and The classroom behavior images in the image are randomly affine transformed and horizontally flipped to obtain the training set after data enhancement. and validation set , Represents the training set after data enhancement Middle classroom behavior images, Represents the validation set after data augmentation Middle Image of classroom behavior.

3. The classroom behavior recognition method based on multi-scale dilated residual attention perception according to claim 2 is characterized in that: Step b) is as follows: b-1) The training set after data enhancement Classroom behavior images in Input to the first downsampling layer to get the initial feature map , the first downsampling layer includes convolutional layer, regularization layer and Activation function layer, where the convolution kernel size of the convolution layer is 6×6, the stride is 2, and the padding is 2; b-2) Input feature T into the first Conv_C3 module to obtain feature , and then the feature Input to the second Conv_C3 module to get the features , and then the feature Input to the third Conv_C3 module to get the features ,Each Conv_C3 module consists of a convolutional layer and a C3 module; b-3) The characteristics Input to the fourth Conv_C3 module to get the feature map ; b-4) The feature map Input to The module performs multi-scale fusion and extraction to obtain features .

4. The classroom behavior recognition method based on multi-scale dilated residual attention perception according to claim 3 is characterized in that: Step d) is as follows: d-1) The upsampling part includes convolutional layer, upsampling layer and Module, where the convolution kernel size of the convolution layer is 1×1, the stride is 1, the padding is 0, and the upsampling layer uses bilinear interpolation; The downsampling part includes the convolutional layer and Module, where the convolution kernel size of the convolution layer is 3×3, the stride is 2, and the padding is 1; The module includes two convolutional layers, a separation layer, indivual Multi-scale dilated convolution modules, splicing layers, settings , where each convolutional layer is followed by a regularization layer and Activation function, the convolution kernel size of the two convolutional layers is 1×1, the stride is 1, and the padding is 0; The MDCM module consists of a three-layer structure and a residual connection. The first layer is composed of two parallel convolutional layers, each of which has a regularization layer and a ReLU activation function. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of the second convolutional layer is 1×1, the step size is 1, and the padding is 0. The second layer is composed of three parallel convolutional layers, each of which has a regularization layer and Activation function, where the convolution kernel size of the first convolution layer is 3×3, the step size is 1, the padding is 1, and the dilation rate is 1; the convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 3, and the dilation rate is 3; the convolution kernel size of the third convolution layer is 3×3, the step size is 1, the padding is 5, and the dilation rate is 5; the third layer structure consists of a convolution layer, a regularization layer, and an activation function layer, where the convolution kernel size of the convolution layer is 1×1, the step size is 1, the padding is 0, and the activation function is Activation function.

5. The classroom behavior recognition method based on multi-scale dilated residual attention perception according to claim 4 is characterized by: d-2) The characteristics Input to the convolution layer of the first upsampling part to get the features ,feature Input to the upsampling layer of the first upsampling part to get the features ,feature Features output from the backbone network Splicing to get features ; feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module Divide it into two parts from the channel dimension and get the features and Features ,feature and Features The number of channels becomes the characteristic one-half of; feature Input to The parallel convolutional layer of the first layer of the module structure obtains two different features, features and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. ,feature and Features , the feature ,feature and Features Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , and then the feature and Features Add them together to get the output characteristics of the first MDCM module ; Then the features Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, feature and Features , the feature and Features Concatenate on the channel dimension to get features , the feature Input to The three parallel convolutional layers in the second layer of the module obtain three different features. , and , the feature , and Concatenate on the channel dimension to get features , the feature Input to The convolutional layer of the third layer of the module structure obtains the features , the feature and Features Add them together to get the second Output characteristics of the module ; The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ; d-3) Characteristics Input to the second upsampling part of Neck, feature Input to the convolution layer of the second upsampling part to get the features ,feature Input to the upsampling layer of the second upsampling part to get the features ,feature Features obtained with the backbone network Splicing to get features ,feature Input to The first convolutional layer of the module obtains the features ,feature Input to The separation layer of the module From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of feature Input to The parallel convolutional layers of the first layer of the module structure obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the first Output characteristics of the module ; feature Enter the second The parallel convolutional layer of the first layer of the module obtains two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to The three parallel convolutional layers in the second layer of the module structure obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to The convolutional layer of the third layer of the module structure obtains the features ,feature and Features Add them together to get the second Output characteristics of the module ; The characteristics , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature ; d-4) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of feature Input to the parallel convolutional layer of the first layer of the MDCM module, two different features are obtained, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ; feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ; feature , and Concatenate on the channel dimension to get features ,feature Input to the second convolution layer of the C2f_MDCM module to get the features ; d-5) Characteristics Input to the convolution layer of the first downsampling part in Neck to get the feature ,feature and Features Fusion to obtain features , the feature is input to the first convolutional layer of the C2f_MDCM module to obtain the feature ,feature Input to the separation layer of C2f_MDCM module, and transform the features From the channel dimension, it is divided into two parts, that is, the characteristics are obtained and Features ,feature and Features The number of channels is the same as the original feature one-half of; feature Input to the parallel convolutional layer of the first layer of the MDCM module, two different features are obtained, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the first MDCM module ; feature Input to the parallel convolutional layer of the first layer structure of the second MDCM module, and obtain two different features, namely and ,feature and Concatenate on the channel dimension to get features ,feature Input to the three parallel convolutional layers of the second layer structure of the MDCM module, and obtain three different features, namely , and ,feature , and Concatenate on the channel dimension to get features ,feature Input to the convolutional layer of the third layer structure of the MDCM module to obtain the feature ,feature and Features Add them together to get the output characteristics of the second MDCM module ; feature , and Concatenate on the channel dimension to get features ,feature Input to The second convolutional layer of the module obtains the feature .

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a classroom behavior recognition method based on multi-scale dilated residual attention perception is implemented as described in any one of claims 1-5.

7. A computer device, characterized in that: Includes the following modules: Memory, for storing computer programs / instructions; A processor, used to execute computer programs and instructions to implement a classroom behavior recognition method based on multi-scale dilated residual attention perception as described in any one of claims 1-5.

Images