A terrestrial mammal action recognition method based on spatiotemporal channel attention mechanism

Abstract

The present invention discloses a method for identifying terrestrial mammal actions based on a spatiotemporal channel attention mechanism, and belongs to the technical field of deep learning action recognition. The present invention first sets an action recognition model, which includes a 3D backbone network for extracting spatiotemporal features of a video, a 2D backbone network for extracting spatial features of a key frame of a video, and a channel fusion module for fusing spatiotemporal features and spatial features, and its output is input into an attention mechanism module after a first convolution block, and the attention feature map output by the attention mechanism module is sent to a classification layer through a second convolution block, and the classification layer is used to perform action recognition prediction on the classification feature map, and obtain action recognition prediction results, including: terrestrial mammal action category, confidence of a prediction box, and location information of a prediction box; based on the collected training data set, the action recognition model is deep-learned and trained, and when a preset convergence condition is met, an action recognition model for a target animal is obtained; the total loss function used when the action recognition model is deep-learned and trained is a weighted sum of a prediction box regression function, a classification function, and a confidence function. Based on the existing spatiotemporal action recognition model based on deep learning, the present invention optimizes the backbone network architecture, loss function composition, etc., which to a certain extent overcomes the problems of small sample actions and difficulty in recognizing easily confused actions in the original model, enhances the model's generalization ability and the recognition accuracy of terrestrial mammals, and improves detection performance.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning action recognition, and in particular to a terrestrial mammal action recognition method based on a spatiotemporal channel attention mechanism. Background Art

[0002] As an important technology in the field of computer vision, video action recognition is gradually showing its wide application value in many fields, especially in the emerging direction of animal action recognition. Animal action recognition is not only crucial for analyzing animal behavior and understanding animal habits, but also can promote the rational formulation of ecological protection strategies. At present, terrestrial mammals such as giant pandas, golden monkeys, and Asian elephants are the key protected animals in my country. However, the scope and efficiency of traditional animal behavior observation and data collection methods are relatively limited, which emphasizes the need for advanced technologies such as deep learning models to achieve the purpose of effectively and accurately identifying animal behavior from video data.

[0003] Currently, there are three mainstream models for video action recognition:

[0004] The first is a dual-stream network, which inputs a single key frame and multi-frame optical flow into two branch networks respectively, capturing temporal and spatial information respectively, and performing a weighted average of the scores obtained by the two networks to obtain the final recognition result. Its disadvantage is that the calculation of optical flow information is relatively complex and time-consuming, which increases time and computing costs.

[0005] The second type, 3D CNN (convolutional neural network), expands the 2D convolutional network to 3D (such as n×n to n×n×n), and extracts video motion information through multiple 3D convolution modules. The disadvantage is high computational complexity and long training and inference time.

[0006] The third type, the Transformer network, completely relies on the attention mechanism, does not use the CNN network, segments and embeds each frame of the video, encodes the position of the sub-image, and adds global encoding, encoding in the spatial and temporal dimensions in sequence to achieve long sequence modeling. Its disadvantage is that it may be affected by memory limitations and efficiency when processing long videos, and real-time monitoring is difficult.

[0007] The common disadvantages of the above methods are mainly reflected in:

[0008] (1) The feature extraction method for video action information is relatively simple, which has a certain impact on the accuracy of action recognition.

[0009] (2) Existing motion recognition methods are mainly focused on recognizing human motions, and there are very few methods for animal motion recognition.

[0010] Therefore, developing a simple and efficient model to quickly and accurately identify animal movements, thereby promoting behavioral monitoring and protection of terrestrial mammals, is an urgent problem that needs to be solved by technicians in this field. Summary of the invention

[0011] The present application provides a terrestrial mammal action recognition method based on a spatiotemporal channel attention mechanism, aiming to solve the problems of scarcity of animal action recognition models on the market, low animal action recognition accuracy, and low efficiency.

[0012] The technical solution adopted by the present invention is:

[0013] A method for terrestrial mammal action recognition based on spatiotemporal channel attention mechanism, the method comprising the following steps:

[0014] Setting up the action recognition model

[0015] The action recognition model includes: a 3D backbone network, a 2D backbone network, a channel fusion module, a first convolution block, an attention mechanism module, a second convolution block and a classification layer;

[0016] Among them, the 3D backbone network takes multiple frames of video as input and is used to extract the spatiotemporal features of the input video;

[0017] The 2D backbone network takes the key frames of the video as input and is used to extract the spatial features of the current input key frames;

[0018] The channel fusion module is used to perform channel dimension feature fusion on spatiotemporal features and spatial features, and then the feature fusion is passed through the first convolution block to obtain the final fusion feature map;

[0019] The attention mechanism module is used to perform self-attention calculation on the final fusion feature map to obtain the attention feature map, and then the attention feature map is passed through the second convolution block to obtain the classification feature map of the input classification layer;

[0020] The classification layer is used to perform action recognition prediction on the classification feature map and obtain action recognition prediction results, including: terrestrial mammal action category, confidence of the prediction box (target box) and location information of the prediction box;

[0021] Action recognition model training

[0022] The action recognition model is trained through deep learning based on the collected training data set. When the preset convergence condition is met (the number of training times reaches the upper limit or the set loss function converges), the action recognition model for the target animal is obtained.

[0023] The total loss function used in deep learning training of the action recognition model is the weighted sum of the prediction box regression function, classification function and confidence function.

[0024] Furthermore, the prediction box regression function is specifically set as:

[0025]

[0026] Among them, L reg represents the prediction box regression function, IoU is the intersection over union ratio between the real box and the prediction box, is the Euclidean distance between the center point of the real box and the predicted box, b and Represent the true box and the predicted box respectively;

[0027] The calculation formula of parameter D is:

[0028] First distance assistance Second distance assistance w and h are the width and height of the real frame respectively. are the width and height of the prediction box respectively;

[0029] parameter parameter

[0030] Furthermore, the expression of the total loss function used in deep learning training of the action recognition model is:

[0031] L=I obj λ obj L conf +I noobj λ noobj L conf +I obj λ cls L cls +I obj λ reg L reg

[0032] Among them, L represents the total loss function, L conf is the confidence function, L cls is the classification function, I obj Represents the mask function corresponding to the positive sample. If it is identified as a positive sample, then I obj =1, otherwise I obj =0,I noobj Represents the mask function corresponding to the negative sample. If it is identified as a negative sample, then I noobj =1, otherwise I noobj =0;λ obj , noobj , cls , reg They are the weight of the confidence function of the positive sample, the weight of the confidence function of the negative sample, the weight of the classification function, and the weight of the prediction box regression function.

[0033] Furthermore, the 3D backbone network can adopt a ResNeXt network.

[0034] Furthermore, the 2D backbone network sequentially includes a cascaded 2D convolution block, a Layer Norm normalization layer, a first stacking module of a ConvNeXt module stack, and a plurality of second stacking modules consisting of a stack of cascaded downsampling layers and ConvNeXt modules as basic unit modules;

[0035] Among them, the 2D convolution block is a cascade of 2D convolution layers and activation functions;

[0036] The ConvNeXt module consists of a cascaded depth-wise separable convolution block, a 2D convolution block, a Layer Norm normalization, a Leaky RELU activation function, an anchor and scaling block, and a Drop Path block.

[0037] Furthermore, the downsampling layer is a cascade of Layer Norm normalization layers and 2D convolution blocks.

[0038] Furthermore, the number of ConvNeXt modules in the first stacking module is 3, the number of the second stacking modules is 3, and the corresponding number of cycles of the basic unit modules are set to 3, 27, and 3 respectively.

[0039] Furthermore, the channel fusion module is a cascade of convolutional layers with kernel sizes of 1×1 and 3×3 respectively.

[0040] Furthermore, the attention mechanism module is specifically:

[0041] The final fusion feature map is transformed into two tensors of dimensions C3×H'W' and H'W'×3, and the tensors C3×H'W' and H'W'×3 are matrix multiplied to obtain the attention score of dimension C3×C3. The attention score is normalized using the softmax function, and the tensor of dimension C3×H'W' is multiplied by the normalized attention score to obtain the attention feature map of dimension C3×H'×W'.

[0042] Among them, C3 represents the number of channels of the final fusion feature map, and H' and W' are the channel height and width of the final fusion feature map respectively.

[0043] Furthermore, the first convolution block is a cascade of convolutional layers with convolution kernel sizes of 1×1 and 3×3 respectively.

[0044] Furthermore, the second convolution block includes in sequence: a convolution layer with a convolution kernel size of 3×3, an overfitting layer (ie, a Dropout layer), and a convolution layer with a convolution kernel size of 1×1.

[0045] The technical solution provided by the present invention brings at least the following beneficial effects:

[0046] The present invention can input animal video frames and perform efficient end-to-end prediction to obtain the animal's position and action recognition results. Based on the existing spatiotemporal action recognition model based on deep learning, the present invention optimizes the backbone network architecture, loss function composition, etc., and overcomes the problems of small sample actions and easily confused action recognition difficulties in the original model to a certain extent, enhances the model's generalization ability and the recognition accuracy of terrestrial mammals, and improves the detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of a network architecture of a terrestrial mammal action recognition method based on a spatiotemporal channel attention mechanism provided by an embodiment of the present invention;

[0049] Figure 2 This is a diagram of the 2D backbone ConvNeXt network architecture used in an embodiment of the present invention.

[0050] Figure 3 This is a structural diagram of the downsampling module used in an embodiment of the present invention.

[0051] Figure 4 This is a structural diagram of the ConvNeXt module used in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described in detail and completely in conjunction with the drawings in the implementation of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present invention.

[0053] Video action recognition technology is an important branch of computer vision. It automatically detects and classifies actions by analyzing action information in video sequences. With the development of deep learning, video action recognition technology has made significant progress and is widely used in many fields such as action analysis, video surveillance, and health monitoring.

[0054] Animal action recognition refers to the detection and analysis of animal behavior patterns in videos. This technology is important for understanding animal habits, emotional states, health conditions, and interactions with the environment. Animal action recognition can be applied in many fields, including but not limited to wildlife protection, livestock management, animal behavior research, and biodiversity monitoring.

[0055] The embodiments of the present invention focus on the existing problems of land mammal protection in the field of animal identification. At present, land mammals such as giant pandas, red pandas, golden monkeys, Asian elephants, and Chinese serows are the key protected endangered animals in my country. However, the scope and efficiency of traditional animal behavior observation and data collection methods are relatively limited, which emphasizes the need for advanced technologies such as deep learning models to achieve the purpose of effectively and accurately identifying animal behavior from video data.

[0056] In order to solve the problems in the prior art of the scarcity of deep learning recognition methods for animal actions, the low efficiency and accuracy of the existing algorithms for animal action recognition, the embodiments of the present invention provide a terrestrial mammal action recognition method based on the spatiotemporal channel attention mechanism. The method is based on the YOWO (You Only Watch Once) model, and is improved in terms of network architecture, module design, loss function composition, loss function weight, etc., so as to achieve real-time end-to-end prediction after the animal video frame is input, and obtain the precise position and action recognition results of the animal.

[0057] The following is a further explanation of a terrestrial mammal action recognition method based on a spatiotemporal channel attention mechanism provided by an embodiment of the present invention in conjunction with specific embodiments.

[0058] like Figure 1 As shown, a method for recognizing terrestrial mammal actions based on a spatiotemporal channel attention mechanism provided by an embodiment of the present invention includes:

[0059] Step 1: Build a 3D backbone network model to extract the spatiotemporal features of video frames:

[0060] Specifically, in this embodiment, ResNeXt is used as the 3D backbone network model.

[0061] The model defines three different depths of ResNeXt: 3D-ResNeXt-50, 3D-ResNeXt-101, and 3D-ResNeXt-152, each with a different number of layers and parameters. The pre-trained weights of each model are obtained by pre-training the Kinetics dataset; in the embodiment of the present invention, the 3D-ResNeXt-101 model and the ResNeXt-101-Kinetics pre-trained weight file are used.

[0062] The backbone network used uses an initial convolutional layer, a batch normalization layer, and a ReLU activation function, followed by multiple layers constructed by the Make Layer function, each of which includes multiple ResNeXtBottleneck modules. Depending on whether the dimension or depth needs to be changed, it is decided whether to use downsampling. In this embodiment, the number of cycles [3, 4, 23, 3] is used in the four stages of ResNeXt. The downsampling function is used to downsample between each stage of ResNeXt to reduce the dimension of the feature map. After using average pooling, zero padding is used to match the desired number of output channels. The ResNeXtBottleneck module has a two-fold expansion ratio and contains three convolutional layers, of which the second convolutional layer is a grouped convolution, the number of groups of which is determined by the cardinalit parameter, and the downsampling rate parameter is used to downsample the residual branch when necessary.

[0063] The video multi-frame image is used as input, and its dimension is 3×D×H×W, where “3” is the image RGB2 channel, “D” is the number of video frames, “H” is the height of the image, and “W” is the width of the image.

[0064] In actual application scenarios, ResNeXt backbone networks of different sizes can be selected according to needs. In addition, the method provided in the embodiment of the present invention also includes the ResNet and ShuffleNetv2 backbone networks of the original model, and the actual application effect is inferior to the ResNeXt model.

[0065] Step 2: Build a 2D backbone network model to extract the spatial features of each image frame of the video frame:

[0066] The embodiment of the present invention uses ConvNeXt as the 2D backbone network model, see Figure 2The 2D backbone network model used in this embodiment includes cascaded 2D convolution blocks (including cascaded 2D convolution layers and activation functions), Layer Norm normalization layers, a first stacking module of ConvNeXt module stacking, and a plurality of second stacking modules composed of cascaded downsampling layers and ConvNeXt modules as basic unit modules. In this embodiment of the present invention, the number of cycles of the ConvNeXt module in the first stacking module is 3, the number of second stacking modules is 3, and the number of cycles of the corresponding basic unit modules is set to 3, 27, and 3, respectively. That is, the 2D backbone network model of the embodiment of the present invention consists of a trunk and four stages, each stage is a stacking structure of corresponding modules, and the selected channel number scale is: the number of output channels of the Layer Norm normalization layer and the first stacking module is 128, and the number of output channels of the three second stacking modules is set to: 256, 512, and 1024, respectively.

[0067] The trunk of the 2D backbone network model uses a convolution layer to increase the number of input channels to the dimension of the first stage, and then normalizes it through LayerNorm. Starting from the second stage, each stage uses a downsampling layer and a ConvNeXt module to increase the number of channels stage by stage.

[0068] like Figure 3 As shown in Figure 3, the downsampling layer includes a cascaded Layer Norm normalization layer and a 2D convolution block for normalized features and spatial feature aggregation.

[0069] like Figure 4 As shown in the figure, the ConvNeXt module includes cascaded depthwise convolution blocks, 2D convolution blocks, Layer Norm normalization, Leaky RELU activation function, anchor & scaling (LayerScale) blocks, and Drop Path blocks; among them, the depthwise convolution block (Depthwise Convolution) and the LayerNorm normalization layer help to fully extract features and improve the stability of model training; the Leaky RELU activation function introduces nonlinear activation to enhance the expressiveness of the model, and the DropPath regularization technology prevents overfitting by randomly discarding certain paths. Layer Scale (Anchor & Scaling) scales the output of each block through learnable parameters to adjust the dynamic range of the features.

[0070] The current frame of the video is used as input, with a dimension of 3×H×W. In actual application scenarios, ConvNeXt backbone networks of different sizes can be selected according to needs. In addition, when implementing the method proposed in the embodiment of the present invention, the 2D backbone network model can also use the YOLOv2 and YOLOv3 backbone networks of the original model, which have fewer parameters than the ConvNeXt backbone network, but the recognition effect is poor.

[0071] Construct a channel fusion and attention mechanism module (CFAM):

[0072] The CFAM includes channel fusion and attention mechanism modules. Through CFAM, the model can obtain powerful spatial features from the 2D branch, obtain time series information from the 3D branch, and use the attention mechanism to focus on the key areas or time periods where the action occurs.

[0073] The channel fusion module integrates the feature maps of the two branches in the channel dimension. After integration, the output dimension of the 2D backbone network is C1×H'×W', and the output dimension of the 3D backbone network is C2×H'×W'. After channel fusion, the feature map of dimension (C1+C2)×H'×W' is synthesized. Subsequently, convolutions with kernel sizes of 1×1 and 3×3 are performed in sequence to obtain a feature map of dimension C3×H'×W'. Among them, C3=C1+C2.

[0074] The attention mechanism module adaptively focuses on the most important part of the input features. The feature maps obtained by channel fusion are transformed into dimensions C3×H'W' and H'W'×3, and matrix multiplication is performed to obtain an attention score of dimension C3×C3. Then, the softmax function is used for normalization, and the tensor of dimension C3×H'W' is multiplied with the normalized tensor to obtain a feature map of C3×H'×W'.

[0075] The feature map output by CFAM is passed through convolution layers with 3 convolution kernels and 1 convolution kernel, and finally through a classification layer (such as at least one fully connected layer and a classification layer consisting of a fully connected layer with a softmax function) to obtain the final prediction result. The Dropout method is used between the convolution layers with 3 convolution kernels and 1 convolution kernel to randomly set some neurons in the network to zero to prevent overfitting.

[0076] In order to further verify the performance of the method proposed in the embodiment of the present invention, the action recognition model (3D backbone network, 2D backbone network, channel fusion module, first convolutional layer, attention mechanism module, second convolutional layer and classification layer) constructed in the embodiment of the present invention is used for the recognition of giant panda behavior, and the action recognition model is trained and evaluated based on an AVA format giant panda behavior dataset. The dataset includes 1.5K videos, 1.35M video frames and 48K bounding boxes, covering various daily activities of giant pandas. In the dataset, the behavior of giant pandas is systematically classified into 22 different behavior types, including:

[0077] Continuous action: continued standing, continued bipedalstanding, continued sitting, continued prone, continued lying, walking, climbing, rubbing, marking, curling up, excretion.

[0078] Estrus-specific behaviors: licking the chest and abdomen, licking the vulva, rubbing the yin, pregnancy grasping,

[0079] Overall activities: Drinking, Eating, Locomotive, Resting, Grooming,

[0080] In the embodiment of the present invention, the loss function used in the training action recognition model is composed of three loss functions: prediction box regression function, classification function, and confidence function. The specific settings of each function are as follows:

[0081] The prediction box regression function uses a new function R-CIoU proposed in an embodiment of the present invention, and its formula is as follows:

[0082]

[0083] in, represents the prediction box regression function, b and They represent the true bounding box (true box) and the predicted bounding box (predicted box), respectively. IoU is the ratio of the intersection and union areas of two bounding boxes, that is, the intersection-over-union ratio of the true bounding box and the predicted bounding box. is the Euclidean distance between the center points of the two bounding boxes; the calculation formula of parameter D is as follows:

[0084]

[0085] Among them, the distance auxiliary amount R, The calculation formula is as follows:

[0086]

[0087]

[0088] Among them, w and h are the width and height of the real bounding box respectively. They refer to the width and height of the predicted bounding box respectively. The calculation formula of parameter d is:

[0089] In order to guide the loss function to converge to the center point, the embodiment of the present invention introduces a center distance loss in the regression process to accelerate the network convergence. Based on the introduction of the center distance loss, no matter what the angle parameter of the bounding box after rotation is, the ratio of the center point distance to the diameter of the circumscribed circle remains unchanged.

[0090] The confidence function uses the Huber loss function, which is expressed as:

[0091]

[0092] in, represents the confidence function, x and They represent the true result (confidence of the true box) and the predicted result (prediction confidence of the predicted box), respectively. δ is a preset hyperparameter. By adjusting the value of δ, accuracy and robustness can be balanced. As a possible implementation, δ is set to 1 in the embodiment of the present invention.

[0093] The classification function is the AVA Focal loss function:

[0094]

[0095] Among them, L cls (p1, p0) is the classification function, p0 is the predicted probability of positive samples, p1 is the predicted probability of negative samples, the superscript i of p1 represents the negative sample number, the superscript j of p0 represents the positive sample number, positive represents the number of negative samples, negative represents the number of positive samples, α is the weight coefficient used to balance positive and negative samples, γ is the index used to adjust the model's attention to difficult and easy samples, and their values ​​are all preset values. Preferably, in this embodiment, α is set to 1 and γ is set to 0.5.

[0096] The three loss functions are weighted and multiplied by the mask function I, and the total loss function of the action recognition model during training is obtained by adding them together:

[0097]

[0098] Among them, I obj Represents the mask function corresponding to the positive sample. If it is identified as a positive sample, then I obj =1, otherwise I obj =0,I noobj Represents the mask function corresponding to the negative sample. If it is identified as a negative sample, then I noobj =1, otherwise I noobj =0;λ obj , noobj , cls , reg They are the weight of the confidence function of the positive sample, the weight of the confidence function of the negative sample, the weight of the classification function, and the weight of the prediction box regression function.

[0099] Indicator function (I obj and I noobj ) is used to distinguish positive and negative samples and determine which samples’ losses should be calculated. In actual application scenarios, the performance of the loss function is improved by adjusting the weights of different loss functions. In this embodiment of the present invention, the weight coefficients in the total loss function are set to: obj Set to 10,λ noobj Set to 1,λ cls Set to 1,λ reg Set to 1.

[0100] In the process of optimizing parameters, the AdamW optimizer was used with a weight decay of 0.0001, a momentum of 0.9, and 10 training cycles. The initial learning rate was 0.0001, and the first 500 iterations of the first stage were used as a warm-up phase, increasing the learning rate linearly from 0 to 0.0001, with a warm-up factor of 6.6667×10 -4 .

[0101] The input frame size is fixed at 224×224 pixels, and the trained model is evaluated using the mAP metric. Each action with a confidence level exceeding 0.15 is visualized.

[0102] The comparison between the improved model and the original model is shown in Table 1. The number of parameters and floating-point operations has increased, the mAP has increased from 61.5% to 66.5%, and the recognition speed has dropped from 34 frames / second to 30 frames / second, which is still within the range of real-time monitoring speed.

[0103] Table 1

[0104]

[0105] After training with the panda dataset, the test evaluation results of the method proposed in the embodiment of the present invention are shown in Table 2. Among them, mAP1 is the test result of the original model, and mAP2 is the test result of the improved model. It can be seen from the table that, under the premise of maintaining a high recognition rate for easy-to-recognize actions, the mAP of actions that are difficult to recognize, such as "continuous standing", "rubbing", and "rubbing the genitals", has been greatly improved. It can be seen that the method provided by the present application can more accurately recognize the action behavior of animals while being efficient.

[0106] Table 2

[0107]

[0108]

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0110] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for terrestrial mammal action recognition based on spatiotemporal channel attention mechanism, characterized in that: Used to understand an animal's behavior, emotional state, health, and interactions with its environment, including the following steps: Setting up the action recognition model The action recognition model includes: a 3D backbone network, a 2D backbone network, a channel fusion module, a first convolution block, an attention mechanism module, a second convolution block and a classification layer; Among them, the 3D backbone network takes multiple frames of video as input and is used to extract the spatiotemporal features of the input video; The 2D backbone network takes the key frame of the video as input and is used to extract the spatial features of the key frame currently input; the 2D backbone network sequentially includes a cascaded 2D convolution block, a Layer Norm normalization layer, a first stacking module of a ConvNeXt module stack, and a second stacking module composed of a plurality of cascaded downsampling layers and ConvNeXt modules as basic unit modules; wherein the 2D convolution block is a cascaded 2D convolution layer and an activation function; the ConvNeXt module sequentially includes a cascaded depth-separable convolution block, a 2D convolution block, a Layer Norm normalization, a Leaky RELU activation function, an anchor and scaling block, and a Drop Path block; The channel fusion module is used to perform channel dimension feature fusion on spatiotemporal features and spatial features, and then the feature fusion is passed through the first convolution block to obtain the final fusion feature map; The attention mechanism module is used to perform self-attention calculation on the final fusion feature map to obtain the attention feature map, and then the attention feature map is passed through the second convolution block to obtain the classification feature map of the input classification layer; The classification layer is used to perform action recognition prediction on the classification feature map and obtain action recognition prediction results, including: terrestrial mammal action categories, confidence of the prediction box and location information of the prediction box; among which, terrestrial mammal action categories include continuous action, estrus-specific behavior and overall activity; Action recognition model training The action recognition model is trained through deep learning based on the collected training data set, and when the preset convergence conditions are met, the action recognition model for the target animal is obtained; The total loss function used in deep learning training of the action recognition model is the weighted sum of the prediction box regression function, classification function and confidence function; The expression of the total loss function used in deep learning training of the action recognition model is: L=I obj l obj L conf +I noobj l noobj L conf +I obj l cls L cls +I obj l reg L reg Among them, L represents the total loss function, L reg Represents the prediction box regression function, L conf is the confidence function, L cls is the classification function, I obj Represents the mask function corresponding to the positive sample. If it is identified as a positive sample, then I obj =1, otherwise I obj =0,I noobj Represents the mask function corresponding to the negative sample. If it is identified as a negative sample, then I noobj =1, otherwise I noobj =0;λ obj , noobj , cls , reg They are the weight of the confidence function of the positive sample, the weight of the confidence function of the negative sample, the weight of the classification function, and the weight of the prediction box regression function; Among them, the prediction box regression function is specifically set as: Among them, L reg represents the prediction box regression function, IoU is the intersection over union ratio between the real box and the prediction box, is the Euclidean distance between the center point of the real box and the predicted box, b and Represent the true box and the predicted box respectively; The calculation formula of parameter D is: First distance assistance Second distance assistance w and h are the width and height of the real frame respectively. are the width and height of the prediction box respectively; parameter parameter Among them, the expression of the confidence function is: Among them, x and They represent the confidence of the true box and the predicted confidence of the predicted box respectively. The preset hyperparameter δ is used to balance accuracy and robustness.

2. The method according to claim 1, characterized in that The downsampling layer is a cascade of Layer Norm normalization layers and 2D convolution blocks.

3. The method according to claim 1, characterized in that The number of ConvNeXt modules in the first stacking module is 3, and the number of ConvNeXt modules in the second stacking module is 3, and the number of cycles of the corresponding basic unit modules are set to 3, 27, and 3 respectively.

4. The method according to claim 1, characterized in that The channel fusion module is a cascade of convolutional layers with kernel sizes of 1×1 and 3×3 respectively.

5. The method according to claim 1, characterized in that The attention mechanism module is specifically: The final fusion feature map is transformed into two tensors of dimensions C3×H'W' and H'W'×3, and the tensors C3×H'W' and H'W'×3 are matrix multiplied to obtain the attention score of dimension C3×C3. The attention score is normalized using the softmax function, and the tensor of dimension C3×H'W' is multiplied by the normalized attention score to obtain the attention feature map of dimension C3×H'×W'. Among them, C3 represents the number of channels of the final fusion feature map, and H' and W' are the channel height and width of the final fusion feature map respectively.

6. The method according to claim 1, characterized in that The first convolutional block is a cascade of convolutional layers with kernel sizes of 1×1 and 3×3 respectively.

7. The method according to claim 1, characterized in that Furthermore, the second convolution block sequentially includes: a convolution layer with a convolution kernel size of 3×3, an overfitting layer, and a convolution layer with a convolution kernel size of 1×1.

Images