Brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism

Abstract

The invention discloses a brain-like small sample behavior recognition algorithm based on a motion-guided attention mechanism, and relates to the technical field of small sample behavior recognition of deep learning tasks. The brain-like induction reasoning mechanism algorithm module based on the attention mechanism simulates a human brain recognition mechanism and mainly comprises a self-attention motion feature guiding module, a video-level global perception module and a cross attention motion feature guiding module. The self-attention motion feature guiding module is used for capturing key features of a motion-related region between each video frame; the video-level sensing module continuously pays attention to key action related region features of the whole video; the cross attention motion feature guiding module compares key features of motion related areas among different videos and establishes association among the motion related areas; compared with some existing classic methods, the constructed model can significantly improve the detection precision of small sample behavior recognition, and an optimal result is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of small sample behavior recognition for deep learning tasks, and in particular to a brain-like small sample behavior recognition algorithm based on a motion-guided attention mechanism. Background Art

[0002] In recent years, the rapid development of deep learning has significantly improved the performance of behavior recognition models. Although existing behavior recognition methods have achieved good results, they are often unable to effectively distinguish between different categories of videos when samples are limited, resulting in low accuracy. This is because these methods rely on a large amount of labeled data to learn the subtle differences between different behaviors. When there are insufficient data samples, the model has difficulty capturing these differences, which affects its recognition ability and generalization performance.

[0003] Therefore, for this small sample behavior recognition environment, some related methods use advanced technologies such as meta-learning, transfer learning, and data augmentation to effectively learn and distinguish different categories of behaviors with only a small amount of labeled data. The typical model structure includes a feature extractor, a spatiotemporal enhancement module, and a metric classification function. However, there is currently no method that considers how to use the attention mechanism to design a spatiotemporal enhancement module from the perspective of the inductive reasoning mechanism of the human brain. Therefore, it is urgent to design a reasonable small sample behavior recognition network framework for these problems to improve the accuracy of small sample behavior recognition. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned defects of the prior art and provide a brain-like inductive reasoning mechanism algorithm based on the attention mechanism, which is suitable for small sample behavior recognition. This method draws on the inductive reasoning mechanism of the human brain when recognizing behavior. First, the area closely related to the action in a single video is determined, and the attention is gradually extended from short-range continuous actions to long-range continuous actions of the entire video. Finally, the differences or connections between motion-related areas of different videos are compared at the task level. This is not only a brand-new method, but also a novel small sample behavior recognition framework-"Video toTask".

[0005] The purpose of the present invention can be achieved by the following technical means:

[0006] A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism, comprising the following steps:

[0007] 1) Preprocessing the input data, wherein the input data includes support set videos and query set videos, and the preprocessing includes random cropping, horizontal flipping, and color jittering;

[0008] 2) The preprocessed input data is passed through a feature extractor to extract spatial features;

[0009] 3) The feature map with spatial scale extracted by the feature extractor is passed through the self-attention motion feature guidance module (S-MGA) to capture the correlation between the key features of the action-related areas in each video frame;

[0010] 4) The video-level perception module receives the output from the self-attention motion feature guidance module (S-MGA) and continuously pays attention to and associates the key action-related regional features of the entire video, providing a more comprehensive understanding of the differences and connections between different videos for subsequent comparison;

[0011] 5) Operate the video features of the entire task output by the video perception module through the Cross-Attention Motion Feature Guidance Module (C-MGA), compare the motion-related regions of different videos in the task, and explicitly optimize the association between motion-related regions in a given task, potentially increasing inter-class variance while reducing intra-class variance;

[0012] 6) Output discriminative task-specific features and combine them with existing metric functions for small-sample behavior recognition and classification to achieve higher detection accuracy.

[0013] In step 2), the feature extractor adopts ResNet.

[0014] In step 3), the self-attention motion feature guidance module (S-MGA) includes: using bidirectional and multi-scale motion extracted by temporal difference operation to associate motion-related regions between adjacent frames in a video; S-MGA uses bidirectional and multi-scale motion features to learn a motion-related self-correlation score matrix, which explicitly represents the degree of correlation between adjacent frames at the patch level, thereby guiding the association of motion-related region features in the input video.

[0015] In step 3), the feature map with spatial scale extracted by the feature extractor is passed through a self-attention motion feature guidance module (S-MGA) to capture the association between key features of the action-related area between each video frame, including the following steps:

[0016] 3.1) Convolutional dimension reduction operation, which is used to reduce the dimension of input video features to reduce the amount of calculation and parameters of subsequent operations;

[0017] 3.2) Temporal difference feature extraction operation, which is used to extract the corresponding temporal difference features from the feature operation after dimensionality reduction. The temporal difference features are intended to simulate the form of optical flow and contain rich motion features;

[0018] 3.3) Motion feature extraction operation, used to operate on the temporal difference feature and extract bidirectional multi-scale motion features;

[0019] 3.4) Calculate the autocorrelation score matrix operation, which is used to operate the motion features. The calculation method of the autocorrelation score matrix is ​​as follows:

[0020]

[0021] Among them, μ represents the softmax function, D is the dimension of the frame-level feature, M(F i ,F i+1 ) is through F i With F i+1 Motion features extracted by frame difference, frame-level features F i SA i For explicit expression F i The degree of correlation between the motion-related regions in the adjacent frames at the patch level; each video frame generates a corresponding autocorrelation score matrix, which explicitly indicates the degree of correlation between adjacent frames at the patch level, thereby guiding the correlation of motion-related region features in the video;

[0022] 3.5) Output a single associated video operation for the self-association score matrix and deep convolution enhanced video feature operation, the self-association score matrix SA i The calculation method of motion-related region feature association in the guided video is as follows:

[0023] AF i =λ·SA i Φ3(F) i +F i ,i=1K L

[0024] Among them, λ is a learnable parameter, L is the number of sampled frames in the video, and Φ3 is a 3D deep convolution using a 3×1×1 convolution kernel to enhance the modeling ability of motion-related regions with small spatial changes in the temporal dimension, so as to clearly highlight the motion-related region features in the original frame-level features that have high correlation with the motion-related regions in the adjacent frames at the block level.

[0025] In step 4), the video-level perception module includes: utilizing the idea of ​​a multi-layer perceptron to efficiently fuse the temporal dynamic features and channel features in the video. This method pays more attention to the interaction between dynamic features in the temporal dimension and the channel dimension, thereby achieving continuous attention to action-related areas in the entire video.

[0026] In step 4), the video-level perception module receives the output from the self-attention motion feature guidance module (S-MGA) and continuously pays attention to and associates the key action-related regional features of the entire video, specifically including the following steps:

[0027] 4.1) Temporal MLP operation is used to operate the video-level features output by S-MGA. It pays more attention to the interaction of dynamic features in the time dimension and adopts the calculation method:

[0028] G i,*,*,* =W2·σ(W1·F i,*,*,* )+F i,*,*,* i=1K L

[0029] Among them, σ() is the ReLU nonlinear function, W1 and W2 are L×L matrices;

[0030] 4.2) Channel MLP operation is used to operate on the features output by the time series MLP. It pays more attention to the interaction of dynamic features in the channel dimension and adopts the calculation method:

[0031] H *,j,*,* =W4·σ(W3·G *,j,*,* )+G *,j,*,* i=1K D

[0032] Among them, W3 and W4 are D×D matrices;

[0033] The video-level perception module effectively expands the receptive field along the time dimension in the feature space and enhances the global context perception in time, thereby achieving continuous attention and association of key action-related regional features in the entire video, providing a more comprehensive understanding of the video data.

[0034] In step 5), the cross-attention motion feature guidance module (C-MGA) includes: associating motion-related regions of different videos in the same task, C-MGA uses the same method as S-MGA to extract bidirectional and multi-scale motion features of the entire task; for a query video containing L frames, C-MGA calculates L motion-related cross-correlation score matrices to represent the degree of correlation between the query video and all supporting videos in the entire task at the patch level, thereby guiding the association of motion-related region features between different videos. Through this association, the relationship between motion-related regions within the task can be directly learned, which promotes the same type of motion-related regions to obtain higher scores, while different types of motion-related regions obtain lower scores.

[0035] In step 5), the video features of the entire task output by the video perception module are operated by the cross-attention motion feature guidance module (C-MGA), and the specific steps include:

[0036] 5.1) Convolutional dimension reduction operation, which is used to reduce the dimension of the input support set video features and query set video features to reduce the amount of calculation and parameters of subsequent operations;

[0037] 5.2) Temporal difference feature extraction operation, which is used to operate the reduced-dimensional support set video features and query set video features to extract the corresponding temporal difference features. The temporal difference features are designed to simulate the form of optical flow and contain rich motion features. It should be noted that S-MGA extracts the temporal difference features of a single video, while C-MGA extracts the temporal difference features of all support set and query set videos in a task;

[0038] 5.3) Motion feature extraction operation, which is used to operate the support set video temporal difference features and the query set temporal difference features to extract the bidirectional multi-scale motion features of the entire task;

[0039] 5.4) Calculate the cross-correlation attention score matrix to operate on the motion features of the entire task, using the calculation method:

[0040]

[0041] Among them, μ represents the softmax function, MQ,m represents the motion features of the mth video in the query set Q, and MS represents the motion features of the entire support set S;

[0042] 5.5) Output the associated support set video and query set video feature operations, which are used to operate the cross-correlation score matrix and the video feature operations enhanced by deep convolution. The calculation method of the cross-correlation score matrix guiding the correlation of features of different video motion-related regions is as follows:

[0043] A Q,m,i =λ1·CA Q,m,i Φ3(F Q,m ) i +λ2·Φ3(F Q,m ) i +F Q,m,i ,

[0044] A S,n,i =λ2·Φ3(F S,n ) i +F S,n,i

[0045] Among them, the variable n ranges from 1 to N×K, indicating the nth video in the support set, Φ3 is the 3D depth convolution, and F x,x,x Representing frame-level features, this approach maximizes the differences between motion-related regions of different categories and minimizes the differences between the same category, ensuring a more accurate task-specific attention mechanism. Among the selected several categories with similar sub-actions, the C-MGA module accurately focuses on the motion-related regions within the same category, thereby producing more discriminative task-specific features.

[0046] Compared with the prior art, the present invention has the following advantages and effects:

[0047] 1. The present invention proposes a brain-like inductive reasoning mechanism algorithm based on the attention mechanism. This process is similar to the process of small sample recognition by humans. First, the area closely related to the action in a single video is determined, and the attention is gradually extended from short-range continuous actions to long-range continuous actions in the entire video. Finally, the differences or connections between motion-related areas in different videos are compared at the task level. This is not only a brand-new method, but also a novel small sample behavior recognition framework - "Video to Task".

[0048] 2. The method proposed in the present invention includes three newly designed modules: self-attention motion feature guidance module (S-MGA), video-level perception module, and cross-attention motion feature guidance module (C-MGA), which solve two limitations of previous small sample behavior recognition methods: 1) They mainly use frame-level representations that do not contain spatial information in temporal and spatial relationship modeling, resulting in the neglect of important temporal and spatial information related to motion features in the frame; 2) In the process of generating category prototypes using task-specific methods, it is impossible to fully utilize the temporal dynamics and spatial relationships in the task at the same time. Therefore, the correlation between motion-related areas in the video frame in the task is not considered, resulting in unsatisfactory results.

[0049] 3. The present invention proposes a new task-level spatiotemporal enhancement method, which can be combined with any existing metric function to realize small sample behavior recognition and classification. Compared with previous methods, this method achieves the best detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the overall framework and each module of a model designed by a brain-like inductive reasoning mechanism algorithm based on an attention mechanism disclosed in the present invention;

[0051] Figure 2 It is a specific network structure diagram of a self-attention motion feature guidance module designed by a brain-like inductive reasoning mechanism algorithm based on an attention mechanism disclosed in the present invention;

[0052] Figure 3 It is a specific network structure diagram of a video-level perception module designed by a brain-like inductive reasoning mechanism algorithm based on an attention mechanism disclosed in the present invention;

[0053] Figure 4 It is a specific network structure diagram of a cross-attention motion feature guidance module designed by a brain-like inductive reasoning mechanism algorithm based on the attention mechanism disclosed in the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following embodiments will further illustrate the present invention in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. On the contrary, the present invention encompasses any substitutions, modifications, equivalent methods and schemes made on the essence and scope of the present invention as defined by the claims. Further, in order to enable the public to have a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. For those skilled in the art, the present invention can be fully understood without the description of these details.

[0055] The brain-like inductive reasoning mechanism algorithm based on the attention mechanism disclosed in the present invention is suitable for small sample behavior recognition, and the small sample behavior recognition is defined as follows:

[0056] The task of few-shot action recognition aims to identify new classes using only a small number of labeled samples. This task usually consists of a training set D train ={(vi,yi),yi∈C train} for training small sample models, and a test set D test ={(vi,yi),yi∈C test} for testing, where A scenario training model is used, in which each scenario contains a train We randomly select K videos from each category of N action categories to form a support set video S. Then, we select the video from D train The query set video Q samples are selected from the remaining videos of the N action categories in the support set. In each scenario, it is expected that the video in the query set can be classified into one of the N categories with the help of the support set. For the inference process, the test set D is calculated. test The average accuracy of recognition performance for all scenarios above.

[0057] Figure 1 The overall framework diagram and each module diagram of the model designed by the brain-like inductive reasoning mechanism algorithm based on the attention mechanism are given, mainly including:

[0058] The self-attention motion guidance module (S-MGA) is used to capture the correlation between key features of action-related regions in each video frame;

[0059] A video-level perception module that continuously focuses on and associates key action-related regional features throughout the video;

[0060] The Cross-Attention Motion Guided Module (C-MGA) is used to compare motion-related regions across different videos in a task, explicitly optimizing the association between motion-related regions in a given task, potentially increasing inter-class variance while reducing intra-class variance.

[0061] The overall structure of the present invention is:

[0062] First, the input data is preprocessed, including the following operations: random cropping, horizontal flipping, and color jittering.

[0063] For the composition of input data: support set video S∈R NK×T×3×224×224 , query set video Q∈R M ×T×3×224×224 , where T represents the number of frames sampled for each video, and M represents the number of query set videos in a task.

[0064] The preprocessed input data will pass through the feature extractor to extract spatial features: the feature extractor usually selects an existing model, such as ResNet, to generate a feature map ∈ R with a spatial scale D×H×W , where D represents the number of channels of the feature map, H and W represent the spatial size of the feature map, which will be used as the input of the subsequent algorithm.

[0065] The feature map with spatial scale extracted by the feature extractor will pass through the self-attention motion feature guidance module (S-MGA) to capture the association between the key features of the action-related areas between each video frame, including: inputting a single video feature, performing convolution operations on the feature to reduce the dimension of the feature, reduce the amount of calculation, and maintain key information; performing temporal difference feature extraction on the video features after convolution dimensionality reduction to calculate the time difference between video frames to capture motion information; extracting key motion features related to the action based on the results of temporal difference; calculating the self-correlation score matrix between frames or features within the video to highlight the importance of key features. Input a single video feature and use 3D deep convolution to extract the features of the input video, outputting the key features related to the action between each video frame, which are used in subsequent modules such as video-level perception or cross-attention motion feature guidance;

[0066] The specific network structure of the self-attention motion feature guidance module is shown in the figure Figure 2 As shown, the specific operations are as follows:

[0067] The feature map is downsampled by r1=8 times through a 1×1 convolution to reduce the amount of computation and parameters.

[0068] The temporal difference feature extraction operation is used to extract the temporal difference feature. The specific operation is to use the feature of the previous frame to smooth the spatial misalignment through a 2D deep convolution with a convolution kernel size of 3×1, and then subtract the feature of the next frame to generate the forward temporal difference feature of each frame, which is expressed as follows:

[0069] C(F i ,F i+1 )=F i-Φ2(F i+1 )

[0070] Among them, C(Fi,Fi+1) represents the aligned temporal difference between frame Fi and frame Fi+1, Φ2 is a 2D depth convolution, and the extraction operation of reverse temporal difference features is similar.

[0071] The motion feature extraction operation is used to extract the motion feature from the time series difference feature. The specific operation can be expressed by the following formula:

[0072]

[0073] In practice, N = 4 corresponds to CNNs of different spatial scales. j The number of each scale extracts motion information from different receptive fields, following a three-branch implementation: (1) short connections, (2) 3×3 convolutions, and (3) average pooling, 3×3 convolutions and bilinear upsampling, (4) bilinear upsampling, 3×3 convolutions and average pooling. Conv is a channel convolution for aggregating multi-scale motion features. Bidirectional and multi-scale motion features contain rich motion cues between frames, so these features can be used to capture the association of precise motion-related regions in each video.

[0074] The frame-level feature F is obtained through the self-attention mechanism of the bidirectional motion feature. i The motion-related autocorrelation score matrix SA i , to clearly express F i The degree of correlation between the motion-related regions and adjacent frames at the patch level is calculated as follows:

[0075]

[0076] Among them, μ represents the softmax function, D is the dimension of frame-level features, M(Fi,Fi+1) is the motion feature extracted by the difference between Fi and Fi+1 frames, and each video frame generates a corresponding autocorrelation score matrix, which explicitly represents the correlation degree between adjacent frames at the patch level, thereby guiding the association of motion-related regional features in the video.

[0077] Output a single correlation video operation for operating on the autocorrelation score matrix and deep convolution enhanced video features, using the calculation method:

[0078] AF i =λ·SA i Φ3(F) i +F i ,i=1K L

[0079] Wherein, λ is a learnable parameter, L is the number of sampled frames in the video, and Φ3 is a 3D deep convolution using a 3×1×1 convolution kernel to enhance the modeling capability of motion-related regions with small spatial variations in the temporal dimension, so as to clearly highlight the motion-related region features in the original frame-level features that have a high correlation with the motion-related regions in the adjacent frames at the block level. In this way, the present invention clearly highlights the motion-related region features in the original frame-level features F i Features with higher correlation with motion-related regions at the patch level in adjacent frames.

[0080] The video-level perception module receives the output from the self-attention motion feature guidance module (S-MGA), operates on the output of the self-attention motion feature guidance module, obtains continuous attention to the areas closely related to motion in a single video, and provides a more comprehensive understanding of the differences and connections between different videos in the subsequent comparison. The specific network structure of the video-level perception module is shown in the figure below. Figure 3 As shown, the specific operations are as follows:

[0081] The module inputs the video frame F i ∈R T×D×H×W .

[0082] The temporal MLP operation is used to operate on the video-level features output by S-MGA. It pays more attention to the interaction of dynamic features in the time dimension and adopts the calculation method:

[0083] G i,*,*,* =W2·σ(W1·F i,*,*,* )+F i,*,*,* i=1K L

[0084] Among them, σ() is the ReLU nonlinear function, W1 and W2 are L×L matrices;

[0085] The channel MLP operation is used to operate on the features output by the time series MLP. It pays more attention to the interaction of dynamic features in the channel dimension and adopts the calculation method:

[0086] H *,j,*,* =W4·σ(W3·G *,j,*,* )+G *,j,*,* i=1K D

[0087] Among them, W3 and W4 are D×D matrices;

[0088] The video-level perception module effectively expands the receptive field along the time dimension in the feature space and enhances the global context perception in time, thereby achieving continuous attention and association of key action-related regional features in the entire video, providing a more comprehensive understanding of the video data.

[0089] The video features of the entire task output by the video perception module are operated through the cross-attention motion feature guidance module (C-MGA), including: first, the C-MGA module receives video data from the query video set (set video Q) as input, and at the same time, the C-MGA module also receives video data from the support video set (set video S), which are used to compare and associate with the query video; 3D deep convolution is used to perform preliminary feature extraction on the input query video and support video respectively, generate respective video features, and perform convolution dimensionality reduction; temporal difference operation is performed on the extracted video features to calculate the time difference between video frames to capture motion information; based on the results of temporal difference, key motion features related to the action are extracted; by calculating the cross-correlation score matrix, the cross-attention mechanism is applied to compare the motion features between the query video and the support video; after the above processing, the C-MGA module outputs the processed feature representations, which can be used for subsequent classification, recognition or other tasks.

[0090] The specific network structure of the cross-attention motion feature guidance module is shown in the figure Figure 4 As shown, specifically, C-MGA uses convolution to compress the feature dimension by a ratio of r2, and then uses the same method as S-MGA to extract bidirectional and multi-scale motion features for the entire task. Subsequently, for a query video containing T frames, C-MGA calculates T motion-related cross-correlation score matrices, indicating the degree of correlation between the query video and all support videos in the entire task at the patch level. This enables the present invention to explicitly optimize the correlation between motion-related regions in a given specific task, potentially increasing the difference between categories while reducing the difference within categories.

[0091] For each query video, C-MGA generates T unique motion-related cross-correlation score matrices, each corresponding to a frame in the video. Then, for the i-th frame (i∈[1,T]) of the m-th (m∈[1,|Q|]) video in the query set Q, its motion-related cross-correlation score matrix is ​​calculated as follows:

[0092]

[0093] Among them, μ represents the softmax function, M Q,m represents the motion feature of the mth video in the query set Q, M S represents the motion characteristics of the entire support set S; M Q,m ∈R L×HW×D / r2 , M S ∈R L×NKHW×D / r2 , so CA Q,m,i ∈R HW×NKHW .

[0094] Output the associated support set video and query set video feature operations, which are used to operate the cross-correlation score matrix and deep convolution enhanced video feature operations, using the calculation method:

[0095] A Q,m,i =λ1·CA Q,m,i Φ3(F Q,m ) i +λ2·Φ3(F Q,m ) i +F Q,m,i ,

[0096] A S,n,i =λ2·Φ3(F S,n ) i +F S,n,i

[0097] Among them, the variable n ranges from 1 to N×K, indicating the nth video in the support set, Φ3 is the 3D deep convolution, and Fx,x,x represents the frame-level features. This approach maximizes the differences between motion-related regions of different categories and minimizes the differences between the same categories, ensuring a more accurate task-specific attention mechanism. Among the selected several categories with similar sub-actions, the C-MGA module accurately focuses on the motion-related regions within the same category, thereby producing more discriminative task-specific features.

[0098] Afterwards, the output discriminative task-specific features are combined with existing metric functions for few-shot behavior recognition classification, thus achieving higher detection accuracy.

[0099] The model of the present invention outperforms a large number of existing works in both 1-shot and 5-shot settings on all datasets. On classic small-sample behavior recognition datasets such as UCF101, HMDB51, Kinetics, SSv2-Small, and SSv2-Full, the 1-shot accuracy can reach: 88.0%, 63.5%, 75.0%, 42.8%, and 56.7%. The 5-shot accuracy can reach: 97.4%, 79.3%, 88.0%, 61.8%, and 71.1%. The effectiveness of the brain-like inductive reasoning mechanism algorithm based on the attention mechanism of the present invention is verified.

[0100] The above embodiments are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope of the patent coverage of the present invention. Any changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism, characterized by The following steps are involved: 1) Preprocessing the input data, wherein the input data includes support set videos and query set videos, and the preprocessing includes random cropping, horizontal flipping, and color jittering; 2) The preprocessed input data is passed through a feature extractor to extract spatial features; 3) The feature map with spatial scale extracted by the feature extractor is passed through the self-attention motion feature guidance module S-MGA to capture the correlation between the key features of the action-related areas in each video frame; 4) The video-level perception module receives the output from the self-attention motion feature guidance module S-MGA, and continuously pays attention to and associates the key action-related regional features of the entire video, providing a more comprehensive understanding of the differences and connections between different videos for subsequent comparison; 5) The cross-attention motion feature guidance module (C-MGA) operates on the video features of the entire task output by the video perception module, compares the motion-related regions of different videos in the task, and explicitly optimizes the association between motion-related regions in a given task, potentially increasing inter-class differences while reducing intra-class differences; 6) Output discriminative task-specific features and combine them with existing metric functions for small-sample behavior recognition and classification to achieve higher detection accuracy.

2. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 2), the feature extractor adopts ResNet.

3. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 3), the self-attention motion feature guidance module S-MGA includes: using bidirectional and multi-scale motion extracted by temporal difference operation to associate motion-related regions between adjacent frames in a video; S-MGA uses bidirectional and multi-scale motion features to learn a motion-related self-correlation scoring matrix, which explicitly represents the degree of correlation between adjacent frames at the patch level, thereby guiding the association of motion-related region features in the input video.

4. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 3), the feature map with spatial scale extracted by the feature extractor is passed through a self-attention motion feature guidance module S-MGA to capture the association between key features of the action-related area between each video frame, including the following steps: 3.1) Convolutional dimension reduction operation, which is used to reduce the dimension of input video features to reduce the amount of calculation and parameters of subsequent operations; 3.2) Temporal difference feature extraction operation, which is used to extract the corresponding temporal difference features from the feature operation after dimensionality reduction. The temporal difference features are intended to simulate the form of optical flow and contain rich motion features; 3.3) Motion feature extraction operation, used to operate on the temporal difference feature and extract bidirectional multi-scale motion features; 3.4) Calculate the autocorrelation score matrix operation, which is used to operate the motion features. The calculation method of the autocorrelation score matrix is ​​as follows: Among them, μ represents the softmax function, D is the dimension of the frame-level feature, M(F i ,F i+1 ) is through F i With F i+1 Motion features extracted by frame difference, frame-level features F i SA i For explicit expression F i The degree of correlation between the motion-related regions in the adjacent frames at the patch level; each video frame generates a corresponding autocorrelation score matrix, which explicitly indicates the degree of correlation between adjacent frames at the patch level, thereby guiding the correlation of motion-related region features in the video; 3.5) Output a single associated video operation for the self-association score matrix and deep convolution enhanced video feature operation, the self-association score matrix SA i The calculation method of motion-related region feature association in the guided video is as follows: AF i =λ·SA i Φ3(F) i +F i ,i=1KL Among them, λ is a learnable parameter, L is the number of sampled frames in the video, and Φ3 is a 3D deep convolution using a 3×1×1 convolution kernel to enhance the modeling ability of motion-related regions with small spatial changes in the temporal dimension, so as to clearly highlight the motion-related region features in the original frame-level features that have high correlation with the motion-related regions in the adjacent frames at the block level.

5. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 4), the video-level perception module includes: utilizing the idea of ​​a multi-layer perceptron to efficiently fuse the temporal dynamic features and channel features in the video. This method pays more attention to the interaction between dynamic features in the temporal dimension and the channel dimension, thereby achieving continuous attention to action-related areas in the entire video.

6. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 4), the video-level perception module receives the output from the self-attention motion feature guidance module S-MGA, and continuously pays attention to and associates the key action-related regional features of the entire video, specifically including the following steps: 4.1) Temporal MLP operation is used to operate the video-level features output by S-MGA. It pays more attention to the interaction of dynamic features in the time dimension and adopts the calculation method: G i,*,*,* =W2·σ(W1·F i,*,*,* )+F i,*,*,* i=1KL Among them, σ() is the ReLU nonlinear function, W1 and W2 are L×L matrices; 4.2) Channel MLP operation is used to operate on the features output by the time series MLP. It pays more attention to the interaction of dynamic features in the channel dimension and adopts the calculation method: H *,j,*,* =W4·σ(W3·G *,j,*,* )+G *,j,*,* i=1KD Among them, W3 and W4 are D×D matrices; The video-level perception module effectively expands the receptive field along the time dimension in the feature space and enhances the global context perception in time, thereby achieving continuous attention and association of key action-related regional features in the entire video, providing a more comprehensive understanding of the video data.

7. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 5), the cross-attention motion feature guidance module C-MGA includes: associating motion-related regions of different videos in the same task, C-MGA uses the same method as S-MGA to extract bidirectional and multi-scale motion features of the entire task; for a query video containing L frames, C-MGA calculates L motion-related cross-correlation score matrices to represent the degree of correlation between the query video and all supporting videos in the entire task at the patch level, thereby guiding the association of motion-related region features between different videos. Through this association, the relationship between motion-related regions within the task can be directly learned, promoting similar motion-related regions to obtain higher scores, while different motion-related regions obtain lower scores.

8. A brain-like small sample behavior recognition algorithm based on motion-guided attention mechanism as claimed in claim 1, characterized in that In step 5), the cross-attention motion feature guidance module C-MGA operates the video features of the entire task output by the video perception module, and the specific steps include: 5.1) Convolutional dimension reduction operation, which is used to reduce the dimension of the input support set video features and query set video features to reduce the amount of calculation and parameters of subsequent operations; 5.2) Temporal difference feature extraction operation, which is used to operate the reduced-dimensional support set video features and query set video features to extract the corresponding temporal difference features. The temporal difference features are designed to simulate the form of optical flow and contain rich motion features. It should be noted that S-MGA extracts the temporal difference features of a single video, while C-MGA extracts the temporal difference features of all support set and query set videos in a task; 5.3) Motion feature extraction operation, which is used to operate the support set video temporal difference features and the query set temporal difference features to extract the bidirectional multi-scale motion features of the entire task; 5.4) Calculate the cross-correlation attention score matrix to operate on the motion features of the entire task, using the calculation method: Among them, μ represents the softmax function, MQ,m represents the motion features of the mth video in the query set Q, and MS represents the motion features of the entire support set S; 5.5) Output the associated support set video and query set video feature operations, which are used to operate the cross-correlation score matrix and the video feature operations enhanced by deep convolution. The calculation method of the cross-correlation score matrix guiding the correlation of features of different video motion-related regions is as follows: A Q,m,i =λ1·CA Q,m,i Φ3(F Q,m ) i +λ2·Φ3(F Q,m ) i +F Q,m,i , A S,n,i =λ2·Φ3(F S,n ) i +F S,n,i Among them, the variable n ranges from 1 to N×K, indicating the nth video in the support set, Φ3 is the 3D depth convolution, and F x,x,x Representing frame-level features, this method maximizes the differences between motion-related regions of different categories and minimizes the differences between the same category, ensuring a more accurate task-specific attention mechanism. Among the selected several categories with similar sub-actions, the C-MGA module accurately focuses on the motion-related regions within the same category to produce more discriminative task-specific features.

Images