Lightweight motion recognition method based on OfficientGCN improvement

Abstract

The invention discloses an improved lightweight motion recognition method based on an OfficientGCN, which is applied to the technical field of human motion recognition, and comprises the following steps: improving a lightweight graph convolution network model based on progressive cross-scale time convolution, a double-branch parallel time graph convolution architecture and a mixed multi-dimensional attention module, and obtaining a lightweight motion recognition result; carrying out ablation and contrast experiments on the improved lightweight graph convolutional network model by utilizing the data set to obtain a human body action recognition model; obtaining an input video, extracting human body key skeleton points from the input video, and performing preprocessing to obtain an information stream; and performing extraction, fusion, reinforcement and aggregation processing on the information flow by using the human body action recognition model, and determining the human body action category based on the human body action category probability. According to the method, the adaptability of the model to complex actions is improved; the system is more convenient to deploy in a mobile device and an embedded system; the method has a wide application prospect in the fields of man-machine interaction, video monitoring and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human action recognition, in particular, to an improved lightweight action recognition method based on EfficientGCN. BACKGROUND

[0002] Human action recognition has become an important research direction in the field of artificial intelligence. Its application range is extremely wide, covering video monitoring, human-computer interaction, computer animation, and medical care systems and many other fields. This technology provides accurate recognition of human actions for related devices, and plays an important role in the continuous development and progress of artificial intelligence technology. However, the existing human action recognition algorithm still faces many challenges in terms of accuracy and real-time performance.

[0003] The current mainstream human action recognition method is mainly based on graph convolutional neural network, such as EfficientGCN, HD-GCN (Hierarchical Decomposition Graph Convolutional Network) and the like. Among them, EfficientGCN (full name: Efficient Graph Convolutional Network) balances performance and efficiency, but still needs to be further improved in recognition accuracy.

[0004] In addition, the attention mechanism designed by the existing method uses a serial processing method, which processes spatial, temporal and channel features in a fixed order, which may cause the attention learned later to be affected by the attention learned earlier, resulting in the loss of cross-dimensional feature collaborative relationship, and thus the key features for distinguishing actions cannot be extracted. At the same time, most algorithms lack an effective fusion mechanism for multi-scale time features, making it difficult to effectively recognize complex actions with multiple time sequences. These problems limit the application effect of human action recognition technology in practical scenarios such as human-computer interaction and game action modeling.

[0005] In view of the problems in the related art, no effective solution has been proposed so far. SUMMARY

[0006] In order to overcome the above problems, the present application aims to provide an improved lightweight action recognition method based on EfficientGCN, which aims to solve the above problems.

[0007] To this end, the specific technical solutions adopted by the present application are as follows: An improved lightweight action recognition method based on EfficientGCN, the method comprising the following steps: S1, based on progressive cross-scale time convolution, double-branch parallel time graph convolution architecture and mixed multi-dimensional attention module, the light-weight graph convolution network model is improved, and the improved light-weight graph convolution network model is ablated and compared with the data set, and a human action recognition model is obtained; S2, obtaining an input video, extracting human key skeleton points from the input video, and preprocessing to obtain an information stream; S3, using the human action recognition model, extracting, fusing, strengthening and aggregating the information stream, and determining the human action category based on the human action category probability.

[0008] Further, based on the progressive cross-scale time convolution layer, the double-branch parallel time graph convolution architecture and the mixed multi-dimensional attention module, the light-weight graph convolution network model is improved, including: The light-weight graph convolution network model uses the progressive cross-scale time convolution layer to replace the original time processing layer, which is used to reduce the parameter quantity of the light-weight graph convolution network model while improving the recognition performance of the light-weight graph convolution network model; The light-weight graph convolution network model adopts the double-branch parallel time graph convolution architecture to capture the time dependence of long and short range actions; The light-weight graph convolution network model adopts the mixed multi-dimensional attention module to aggregate the spatio-temporal features in different channels, so that the channel response of the key time sequence and the channel feature of the important joint are synergistically strengthened.

[0009] Further, the progressive cross-scale time convolution layer combines multi-scale feature residual connection and channel segmentation strategy, constructs a time sequence receptive field through a multi-scale time convolution kernel, and realizes feature extraction of actions changing over time; The progressive cross-scale time convolution layer divides the channel number on average, so that the parameter quantity of the light-weight graph convolution network model is reduced, realizing the light-weight design.

[0010] Further, the double-branch parallel time graph convolution layer architecture includes a spatial graph convolution layer, a plurality of progressive cross-scale time convolution layers and a cross-time adaptive fusion module; The spatial graph convolution layer is used to extract the spatial information of the skeleton to obtain the spatial features; The plurality of progressive cross-scale time convolution layers are used to perform time sequence processing on the spatial features using convolution kernels of different sizes to capture the time dependence of long and short range actions, and obtain the time sequence features of the short time branch and the time sequence features of the long time branch, respectively; The cross-time adaptive fusion module is used to fuse the time sequence features of the short time branch and the time sequence features of the long time branch to obtain the fused time sequence features.

[0011] Further, the short-time branch timing feature and the long-time branch timing feature are fused to obtain a fused timing feature, including the following steps: The short-time branch timing feature and the long-time branch timing feature are added to obtain a preliminary fused feature; The preliminary fused feature is globally averaged along the time dimension to obtain a timing-level global information; The timing-level global information is divided into two independent features along the channel dimension, and the two independent features are spliced using convolution to obtain a feature of double-branch differential information; The feature of double-branch differential information is normalized using a Softmax function to obtain a timing attention weight; The timing attention weight and the timing features of the two heterogeneous time convolution branches are weighted and fused to obtain a fused timing feature.

[0012] Further, the expression of the fused timing feature is: ; ; ; In the formula, W T1 represents a short-time branch weight; W T2 represents a long-time branch weight; A T1 represents a short-time branch timing attention weight; A T2 represents a long-time branch timing attention weight; T S represents a short-time branch timing feature; T L represents a long-time branch timing feature; W T represents a fused timing feature.

[0013] Further, aggregating spatiotemporal features in different channels to synergistically strengthen the channel response of a key timing and the channel feature of an important node includes: The input feature is divided into several branches in parallel to capture three-dimensional dependency relationships of channel time, time space, and channel space, respectively, to obtain several attention maps; The several attention maps are aggregated by attention weights to synergistically strengthen the channel response of a key timing and the channel feature of an important node.

[0014] Further, the input feature is divided into several branches in parallel to capture three-dimensional dependency relationships of channel time, time space, and channel space, respectively, to obtain several attention maps, including the following steps: The input features are divided into a channel-time branch, a time-space branch, and a channel-space branch in parallel; The channel-time branch reshapes the dimension of the input features, extracts time features in combination with a feature processing layer, processes the time features based on a function, and obtains a channel-time attention map; The time-space branch performs average pooling on the input features in the time frame dimension and the joint dimension, respectively obtains time features and space features, merges the time features and the space features, extracts space-time features in combination with a feature processing layer, processes the space-time features based on a function, and obtains a time-space attention map; The channel-space branch reshapes the dimension of the input features, extracts space features in combination with a feature processing layer, processes the space features based on a function, and obtains a channel-space attention map.

[0015] Further, the input video is obtained, the human key skeleton points of the input video are extracted, and pre-processing is performed to obtain information flow including the following steps: S21, obtaining an input video, extracting the input video to obtain human key skeleton points; S22, storing the human key skeleton point coordinates in the input sequence by frame to obtain human skeleton point data; S23, pre-processing the human skeleton point data to obtain a plurality of information flows, wherein the information flows include joint flow, skeleton flow, and velocity flow.

[0016] Further, the human action recognition model is used to extract, fuse, strengthen, and aggregate the information flow, and determine the human action category based on the human action category probability, including the following steps: S31, using the basic block in the human action recognition model to normalize the joint flow, the skeleton flow, and the velocity flow respectively; S32, based on the double-branch parallel time graph convolution architecture in the human action recognition model, extracting the normalized joint flow, the skeleton flow, and the velocity flow respectively to obtain a plurality of fused time sequence features; S33, using the mixed multi-dimensional attention module in the human action recognition model to aggregate and strengthen the plurality of fused time sequence features to obtain a plurality of optimized time sequence features; S34, concatenating the plurality of optimized time sequence features in the channel dimension, and aggregating the concatenated plurality of optimized time sequence features based on the mainstream branch in the human action recognition model to obtain action features; S35, based on the feature extraction network and the global average pooling layer, aggregating the action features in the time and space dimensions to obtain a channel-level statistical vector; S36. Using a fully connected layer, a linear transformation is performed on the channel-level statistical vector, and the Softmax function is used to process the linearly transformed channel-level statistical vector to obtain the probability of human action categories. S37. Determine the category of human action based on the probability of human action category.

[0017] Compared with the prior art, this application has the following beneficial effects: 1. This invention designs a progressive cross-scale temporal convolution, which gradually expands the temporal perception range while preserving local action details through a hierarchical multi-scale feature transfer mechanism; 2. This invention proposes a dual-branch parallel temporal graph convolution block, which effectively captures the composite features of local micro-actions and global actions by fusing different convolution kernel temporal branches; 3. This invention designs a hybrid multi-dimensional attention module, which achieves efficient extraction of multi-dimensional features through cross-dimensional parallel interaction.

[0018] 2. This invention improves accuracy while maintaining the original model's lightweight design; its innovative modular design enhances the model's adaptability to complex actions; and its lightweight nature makes it easier to deploy in mobile devices and embedded systems. This invention has broad application prospects in fields such as human-computer interaction and video surveillance. Attached Figure Description

[0019] The above-mentioned features, characteristics, and advantages of the present invention, as well as their implementation methods, will become clearer and more readily understood in conjunction with the following description of the embodiments, which are illustrated in detail with reference to the accompanying drawings. Schematic diagrams are shown here: Figure 1 This is a flowchart of a lightweight action recognition method based on EfficientGCN, according to an embodiment of the present invention. Figure 2 This is an overall structural diagram of the human motion recognition model in a lightweight motion recognition method based on EfficientGCN according to an embodiment of the present invention. Figure 3 This is a diagram of the progressive cross-scale temporal convolution structure in a lightweight action recognition method based on EfficientGCN according to an embodiment of the present invention. Figure 4 This is a structural diagram of a cross-temporal adaptive fusion module in a lightweight action recognition method based on EfficientGCN according to an embodiment of the present invention. Figure 5 This is a hybrid multi-dimensional attention structure diagram in a lightweight action recognition method based on EfficientGCN according to an embodiment of the present invention; Figure 6This is one of the action visualizations of "throwing" in a lightweight action recognition method based on EfficientGCN according to an embodiment of the present invention; Figure 7 This is the second visualization of the "throwing" action in a lightweight action recognition method based on EfficientGCN according to an embodiment of the present invention. Figure 8 This is the third visualization of the "throwing" action in a lightweight action recognition method based on EfficientGCN, according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0021] According to an embodiment of the present invention, a lightweight action recognition method based on EfficientGCN is provided.

[0022] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-6 As shown, the lightweight action recognition method based on EfficientGCN, according to an embodiment of the present invention, includes the following steps: S1. Based on progressive cross-scale temporal convolution, dual-branch parallel temporal graph convolution architecture and hybrid multi-dimensional attention module, the lightweight graph convolutional network model is improved. The improved lightweight graph convolutional network model is then subjected to ablation and comparison experiments using a dataset to obtain a human action recognition model.

[0023] It needs to be explained that, for example Figure 2The diagram shown (Three Branch represents three branches; Main Stream represents the main branch; Class Score represents the class probability) illustrates the overall structure of the human action recognition model. This invention designs a progressive multi-scale temporal convolution (PCST-Conv), which gradually expands the temporal perception range while preserving local action details through a hierarchical multi-scale feature transfer mechanism. Secondly, this invention proposes a dual-branch parallel temporal graph convolutional block (DBPT-Block), which effectively captures composite features of local micro-actions and global actions by fusing temporal branches of different convolutional kernels. Finally, this invention designs a hybrid multidimensional attention module (HMDA), which achieves efficient extraction of multidimensional features through cross-dimensional parallel interaction. Ablation and comparison experiments are conducted on the officially provided NTU-RGB+D 60 and NTU-RGB+D 120 datasets to verify that the proposed improved algorithm can further enhance the overall performance of the method in terms of detection accuracy and parameter quantity. These improvements enable the model to outperform the original lightweight graph convolutional network model on the NTU-RGB+D 60 and NTU-RGB+D 120 datasets. Among them, NTU-RGB+D 60 and NTU-RGB+D 120 are two extremely important benchmark datasets in the field of large-scale, multimodal, skeleton-based human action recognition.

[0024] Preferably, improvements to the lightweight graph convolutional network model based on progressive cross-scale temporal convolutional layers, a dual-branch parallel temporal graph convolutional architecture, and a hybrid multi-dimensional attention module include: The lightweight graph convolutional network model replaces the original temporal processing layer with an incremental cross-scale temporal convolutional layer, which reduces the number of parameters in the lightweight graph convolutional network model while also improving the recognition performance of the lightweight graph convolutional network model. The lightweight graph convolutional network model employs a dual-branch parallel temporal graph convolutional architecture to capture the temporal dependencies between long and short-range actions; The lightweight graph convolutional network model employs a hybrid multi-dimensional attention module to aggregate spatiotemporal features from different channels, thereby synergistically enhancing the channel responses of key time sequences and the channel features of important nodes.

[0025] It's important to explain that EfficientGCN is a lightweight graph convolutional network architecture that balances performance and efficiency, renowned for its sophisticated model compression and exceptional inference speed. Its network structure primarily consists of three main branches and a main branch. Each branch comprises three parts: an input feature normalization module, a feature extraction module, and an attention module. First, the input data is normalized to ensure consistent distribution across multiple information streams. Next, the feature extraction module models spatial and temporal information. Then, the attention module adjusts the level of attention given to different features, finally outputting the prediction result.

[0026] This invention employs a novel progressive cross-scale temporal convolution to replace the temporal processing layer in the original network (i.e., the lightweight graph convolutional network model), which reduces the number of model parameters while improving the model's recognition performance. This invention uses a dual-branch parallel temporal graph convolutional architecture to effectively capture the temporal dependencies between long and short-range actions. This invention uses a hybrid multi-dimensional attention module to aggregate spatiotemporal features from different channels and further optimize feature representation.

[0027] Preferably, the progressive cross-scale temporal convolutional layer combines multi-scale feature residual connections and channel segmentation strategies, and constructs a temporal receptive field through multi-scale temporal convolutional kernels to achieve feature extraction of actions that change over time; The progressive cross-scale temporal convolutional layer averages the number of channels, which reduces the number of parameters in the lightweight graph convolutional network model, thus achieving a lightweight design.

[0028] It needs to be explained that, for example Figure 3 The diagram shows the progressive cross-scale temporal convolution structure. This invention replaces the original temporal processing layer of EfficientGCN with progressive cross-scale temporal convolution (PCST-Conv). This module innovatively combines multi-scale feature residual connections with a channel segmentation strategy, constructing a temporal receptive field through multi-scale temporal convolution kernels to extract features that change over time. Simultaneously, the average division of the number of channels reduces the number of parameters in the model, achieving a lightweight design.

[0029] Preferably, the dual-branch parallel temporal graph convolutional layer architecture includes a spatial graph convolutional layer, several progressive cross-scale temporal convolutional layers, and a cross-temporal adaptive fusion module; Spatial graph convolutional layers are used to extract spatial information of the skeleton and obtain spatial features; Several progressively multi-scale temporal convolutional layers are used to perform temporal processing on spatial features using convolutional kernels of different sizes, capturing the temporal dependencies of long and short-range actions, and obtaining the temporal features of short-time branches and long-time branches respectively. The cross-time adaptive fusion module is used to fuse the time-series features of short-time branches and long-time branches to obtain fused time-series features.

[0030] Preferably, fusing the temporal features of short-term branches and long-term branches to obtain fused temporal features includes the following steps: The temporal features of the short-term branch and the long-term branch are added together to obtain the preliminary fusion features; The features after initial fusion are subjected to global average pooling along the time dimension to obtain temporal-level global information; The temporal-level global information is divided into two independent features along the channel dimension, and the two independent features are concatenated using convolution to obtain the features of dual-branch differential information. The features of the bi-branch differential information are normalized using the Softmax function to obtain the temporal attention weights; The temporal attention weights are weighted and fused with the temporal features of the two heterogeneous temporal convolutional branches to obtain fused temporal features.

[0031] Preferably, the expression for fusing time-series features is: ; ; ; In the formula, W T1 Indicates the short-term branch weight; W T2 Indicates the weight of long-term branches; A T1 Represents the temporal attention weights of short-term branches; A T2 Represents the temporal attention weights of long-term branches; T S The temporal characteristics of short-time branches; T L Represents the temporal characteristics of long-term branches; W T This indicates the fusion time sequence characteristics.

[0032] It needs to be explained that the dual-branch parallel temporal graph convolutional layer structure (DBPT-Block), such as Figure 2 As shown in (b), this module consists of a simplified spatial graph convolution, two parallel temporal convolutional layers, and a cross-temporal adaptive fusion module (CTAF). First, the normalized data is fed into the graph convolutional layer to extract the spatial information of the skeleton. Then, the extracted spatial features are fed in parallel into two heterogeneous temporal convolutional branches. One branch uses two smaller convolutional kernels to capture short-range action dependency information, denoted as... T S The other branch uses two larger convolutional kernels to obtain long-range action dependency information, represented as...T L Then, the two different timing information are sent to the CTAF module.

[0033] like Figure 4 The diagram shows the structure of the cross-temporal adaptive fusion module. In the fusion module, the features from both branches are first aggregated through an addition operation. Then, the preliminarily fused features are subjected to global average pooling along the time dimension to obtain temporal-level global information. S T The formula is expressed as follows: ; In the formula, S T Represents time-level global information; GAP Indicates global average pooling; T S The temporal characteristics of short-time branches; T L This represents the temporal characteristics of long-term branches.

[0034] Next, S T Segmented into two independent feature representations along the channel dimension S T1 and S T2 The data are then fed into two 1x1 convolutions and concatenated to restore the original number of channels, resulting in a feature representation of the dual-branch differential information. H T This provides crucial input for subsequent attention mechanisms. The specific process is represented by the following formula:

[0035] ; ; In the formula, H T1 This represents the differentiated information features of short-term branches; H T2 This represents the differentiated information features of long-term branches; H T This indicates the difference between the two branches; Cat This indicates concatenation of channel dimensions.

[0036] Then to H T Temporal attention weights are generated by performing Softmax normalization along the time dimension. A T Each feature is weighted and fused with the original bi-branch features to achieve temporal adaptive fusion of the bi-branch features. The formula is as follows: ; ; ; ; In the formula, W T1 Indicates the short-term branch weight; W T2 Indicates the weight of long-term branches; A T1 Represents the temporal attention weights of short-term branches; A T2 Represents the temporal attention weights of long-term branches; T S The temporal characteristics of short-time branches; T L Represents the temporal characteristics of long-term branches; W T This indicates the fusion of temporal features; Softmax indicates the processing by the Softmax function; H T This indicates the difference between the two branches; A T This represents the temporal attention weights.

[0037] Preferably, aggregating spatiotemporal features from different channels to synergistically enhance the channel responses of key time series and the channel features of important nodes includes: The input features are divided into several branches in parallel to capture the three-dimensional dependencies of channel time, time space and channel space, respectively, resulting in several attention maps; By aggregating several attention maps using attention weights, the channel responses of key time series and the channel features of important key points are synergistically enhanced.

[0038] Preferably, the input features are divided into several branches in parallel to capture the three-dimensional dependencies of channel time, time space, and channel space, respectively, resulting in several attention maps, including the following steps: The input features are divided into channel-time branch, time-space branch and channel-space branch in parallel. The channel time branch reshapes the input feature dimensions and extracts time features in conjunction with the feature processing layer. The time features are then processed based on a function to obtain the channel time attention map. The temporal-spatial branch performs average pooling on the input features in the temporal frame dimension and the joint dimension to obtain temporal features and spatial features respectively. The temporal and spatial features are merged and combined with the feature processing layer to extract the spatiotemporal features. The spatiotemporal features are processed based on the function to obtain the temporal-spatial attention map. The channel space branch reshapes the dimensions of the input features and extracts spatial features in conjunction with the feature processing layer. The spatial features are then processed based on a function to obtain the channel space attention map.

[0039] It needs to be explained that, for example Figure 5 The diagram shows the Hybrid Multidimensional Attention (HMDA) architecture. The HMDA module constructs a parallel cross-dimensional interaction mechanism to synergistically enhance the channel responses of key temporal sequences and the channel features of important nodes. First, the input features are divided into three branches in parallel. The upper and lower branches are responsible for capturing the cross-dimensional interactions between channels and spatiotemporal relationships, while the middle branch models spatiotemporal dependencies. In the upper and lower branches, for the input feature FϵR... BxCxTxV A reshaping operation is used to establish a connection between the spatiotemporal dimension and any dimension of the channel, namely F. CT ϵR BxVxTxC and F VC ϵR BxTxVxC (i.e., channel temporal branch and channel spatial branch). These two branches then extract spatial and spatiotemporal features through feature processing layers, and then use functions to obtain the corresponding attention maps. The feature FϵR is input into the intermediate branch (i.e., the temporal-spatial branch). BxCxTxV First, average pooling is performed on both the temporal frame and joint dimensions. Then, these two features are merged and fed into a convolutional layer to obtain spatiotemporal features. Subsequently, a function is used to generate a spatiotemporal feature attention map, which is then multiplied element-wise with the original features to output the spatiotemporal attention. Finally, the attention weights from the three branches are aggregated to obtain the final output. Therefore, Hybrid Multidimensional Attention (HMDA) captures the dependencies between the channel, temporal, and spatial dimensions. Simultaneously, through a parallel cross-dimensional interaction strategy, it effectively avoids the information attenuation problem in traditional sequential attention mechanisms.

[0040] S2. Acquire the input video, extract key human skeletal points from the input video, and preprocess them to obtain an information stream, thereby providing a standardized and highly discriminative input for the human action recognition model, improving the accuracy and efficiency of subsequent action recognition.

[0041] Preferably, acquiring the input video, extracting key human skeletal points from the input video, and performing preprocessing to obtain the information stream includes the following steps: S21. Obtain the input video, extract the key skeletal points of the human body from the input video; S22. Store the coordinates of key human skeletal points frame by frame into the input sequence to obtain human skeletal point data; S23. Preprocess the human skeletal point data to obtain several information streams, including joint stream, bone stream and velocity stream.

[0042] It needs to be explained that human skeletal point recognition extracts key human points from the input video, then stores the processed skeletal point coordinates frame by frame into the input sequence; and preprocesses the data to form three different information streams, namely joint stream, skeletal stream and velocity stream.

[0043] S3. Using a human motion recognition model, extract, fuse, enhance, and aggregate the information flow, and determine the human motion category based on the probability of the human motion category.

[0044] It needs to be explained that, for example Figure 6 , Figure 7 and Figure 8 The image shown (where Frame represents a video frame) is a visualization of the "throwing" action.

[0045] Preferably, the process of extracting, fusing, enhancing, and aggregating information streams using a human motion recognition model, and determining the human motion category based on the probability of the human motion category, includes the following steps: S31. Using the basic blocks in the human motion recognition model, the joint flow, skeletal flow and velocity flow are standardized respectively. S32. Based on the dual-branch parallel temporal graph convolutional architecture in the human motion recognition model, the normalized joint flow, skeletal flow and velocity flow are extracted respectively to obtain several fused temporal features. S33. Using the hybrid multi-dimensional attention module in the human action recognition model, several fused temporal features are aggregated and enhanced to obtain several optimized temporal features. S34. Several optimized temporal features are concatenated along the channel dimension, and the concatenated optimized temporal features are aggregated based on the mainstream branches in the human action recognition model to obtain action features. S35. Based on the feature extraction network and the global average pooling layer, the action features are aggregated in terms of time and space dimensions to obtain channel-level statistical vectors. S36. Using a fully connected layer, a linear transformation is performed on the channel-level statistical vector, and the Softmax function is used to process the linearly transformed channel-level statistical vector to obtain the probability of human action categories. S37. Determine the category of human action based on the probability of human action category.

[0046] It should be explained that the specific steps of the human motion recognition model in extracting information streams to obtain human motion categories are as follows: Step 1, as follows Figure 2 As shown in (a), the three information streams are fed into the base block for normalization processing to prevent gradient explosion and ensure the consistency of the distribution of multiple information streams.

[0047] Step 2 involves feeding the normalized data into a dual-branch parallel temporal graph convolutional block. This module first performs simplified graph convolution based on the connection methods between various skeletal points to extract spatial information of the human skeleton. Based on this, the obtained feature information is fed in parallel into a progressive cross-scale temporal convolutional module to extract information from different temporal dimensions. Then, a cross-temporal adaptive fusion module is used to fuse the information from different temporal dimensions, obtaining composite features of local micro-movements and global actions.

[0048] Step 3: The extracted spatiotemporal features are fed into the hybrid multidimensional attention module. This module applies adaptive weights in the spatial dimension (focusing on key joints), temporal dimension (focusing on important frames), and channel dimension (filtering important feature maps) through parallel computation. This dynamically enhances salient features that are sensitive to discriminative action patterns and suppresses irrelevant or noisy information, thereby providing more discriminative and robust optimized feature representations for subsequent classification or recognition tasks.

[0049] Step 4: Concatenate the features extracted from the three branches along the channel dimension and feed them into the main branch.

[0050] The mainstream branch, responsible for the core task of fusing heterogeneous information from multiple sources, is designed with a significantly larger number of channels (i.e., wider layers) to provide ample feature representation capacity. This branch performs deep integration, nonlinear transformation, and information extraction on the concatenated high-dimensional features. Through this design, the mainstream branch can effectively aggregate complementary spatiotemporal contextual information from different branches, eliminate feature redundancy and inconsistency, and ultimately output a highly fused, discriminative, and dimensionally uniform feature representation, providing a powerful input for subsequent classifiers.

[0051] Step 5: Connect a global average pooling layer at the end of the feature extraction network to perform feature aggregation along the time and space dimensions to generate a channel-level statistical vector; then connect a fully connected layer to linearly transform the vector to a dimension that matches the number of action categories, and finally output the classification probability through the Softmax function.

[0052] Step 6: Determine the category of human movement based on the probability of the human movement category.

[0053] In summary, by utilizing the above-mentioned technical solutions of this invention, we have designed a progressive cross-scale temporal convolution, which, through a hierarchical multi-scale feature transfer mechanism, gradually expands the temporal perception range while preserving local action details. Secondly, we propose a dual-branch parallel temporal graph convolution block, which effectively captures the composite features of local micro-actions and global actions by fusing temporal branches of different convolutional kernels. Finally, we design a hybrid multi-dimensional attention module, which achieves efficient extraction of multi-dimensional features through cross-dimensional parallel interaction. This invention improves accuracy while maintaining the original model's lightweight nature; its innovative module design enhances the model's adaptability to complex actions; and its lightweight nature makes it easier to deploy on mobile devices and embedded systems. This invention has broad application prospects in fields such as human-computer interaction and video surveillance. It can not only be applied to remote assessment and training guidance in medical institutions and rehabilitation centers, but also to real-time detection of abnormal behavior in security monitoring locations to ensure public safety, and to provide accurate motion technology analysis for athletes and fitness enthusiasts in sports training and fitness scenarios; this invention has broad market prospects.

[0054] Although the present invention has been disclosed above with reference to preferred embodiments, the embodiments are merely examples for illustrative purposes and are not intended to limit the present invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the present invention. The scope of protection claimed by the present invention should be determined by the claims.

Claims

1. A lightweight action recognition method based on an improved EfficientGCN, characterized in that, The method includes the following steps: S1. Based on progressive cross-scale temporal convolution, dual-branch parallel temporal graph convolution architecture and hybrid multi-dimensional attention module, the lightweight graph convolutional network model is improved, and the improved lightweight graph convolutional network model is subjected to ablation and comparison experiments using dataset to obtain a human action recognition model. S2. Acquire the input video, extract key human skeletal points from the input video, and perform preprocessing to obtain an information stream; S3. Using a human motion recognition model, extract, fuse, enhance, and aggregate the information flow, and determine the human motion category based on the probability of the human motion category.

2. The lightweight action recognition method based on EfficientGCN improvement according to claim 1, characterized in that, The improvements to the lightweight graph convolutional network model based on progressive cross-scale temporal convolutional layers, a dual-branch parallel temporal graph convolutional architecture, and a hybrid multi-dimensional attention module include: The lightweight graph convolutional network model replaces the original temporal processing layer with an incremental cross-scale temporal convolutional layer, which reduces the number of parameters in the lightweight graph convolutional network model while also improving the recognition performance of the lightweight graph convolutional network model. The lightweight graph convolutional network model employs a dual-branch parallel temporal graph convolutional architecture to capture the temporal dependencies between long and short-range actions; The lightweight graph convolutional network model employs a hybrid multi-dimensional attention module to aggregate spatiotemporal features from different channels, thereby synergistically enhancing the channel responses of key time sequences and the channel features of important nodes.

3. The lightweight action recognition method based on EfficientGCN improvement according to claim 2, characterized in that, The progressive cross-scale temporal convolutional layer combines multi-scale feature residual connections and channel segmentation strategies, and constructs a temporal receptive field through multi-scale temporal convolutional kernels to achieve feature extraction of actions that change over time. The progressive cross-scale temporal convolutional layer divides the number of channels by average, which reduces the number of parameters in the lightweight graph convolutional network model and achieves lightweight design.

4. The lightweight action recognition method based on EfficientGCN improvement according to claim 2, characterized in that, The dual-branch parallel temporal graph convolutional layer architecture includes a spatial graph convolutional layer, several progressive cross-scale temporal convolutional layers, and a cross-temporal adaptive fusion module. The spatial graph convolutional layer is used to extract spatial information of the skeleton to obtain spatial features; Several progressive cross-scale temporal convolutional layers are used to perform temporal processing on spatial features using convolutional kernels of different sizes, capture the temporal dependence of long and short-range actions, and obtain the temporal features of short-time branches and long-time branches respectively. The cross-time adaptive fusion module is used to fuse the time-series features of short-time branches and long-time branches to obtain fused time-series features.

5. A lightweight action recognition method based on EfficientGCN improvement according to claim 4, characterized in that, The process of fusing the temporal features of short-term branches and long-term branches to obtain fused temporal features includes the following steps: The temporal features of the short-term branch and the long-term branch are added together to obtain the preliminary fusion features; The features after initial fusion are subjected to global average pooling along the time dimension to obtain temporal-level global information; The temporal-level global information is divided into two independent features along the channel dimension, and the two independent features are concatenated using convolution to obtain the features of dual-branch differential information. The features of the bi-branch differential information are normalized using the Softmax function to obtain the temporal attention weights; The temporal attention weights are weighted and fused with the temporal features of the two heterogeneous temporal convolutional branches to obtain fused temporal features.

6. The lightweight action recognition method based on EfficientGCN improvement according to claim 5, characterized in that, The expression for the fused temporal features is: ； ； ； In the formula, W T1 Indicates the short-term branch weight; W T2 Indicates the weight of long-term branches; A T1 Represents the temporal attention weights of short-term branches; A T2 Represents the temporal attention weights of long-term branches; T S The temporal characteristics of short-time branches; T L Represents the temporal characteristics of long-term branches; W T This indicates the fusion time sequence characteristics.

7. A lightweight action recognition method based on EfficientGCN improvement according to claim 2, characterized in that, The aggregation of spatiotemporal features from different channels, enabling synergistic enhancement of channel responses at key time points and channel features at important critical points, includes: The input features are divided into several branches in parallel to capture the three-dimensional dependencies of channel time, time space and channel space, respectively, resulting in several attention maps; By aggregating several attention maps using attention weights, the channel responses of key time series and the channel features of important key points are synergistically enhanced.

8. A lightweight action recognition method based on EfficientGCN improvement according to claim 7, characterized in that, The step of dividing the input features into several branches in parallel to capture the three-dimensional dependencies of channel time, time space, and channel space, and obtaining several attention maps, includes the following steps: The input features are divided into channel-time branch, time-space branch and channel-space branch in parallel. The channel time branch reshapes the input feature dimension, extracts time features in conjunction with the feature processing layer, processes the time features based on a function, and obtains the channel time attention map. The temporal-spatial branch performs average pooling on the input features in the temporal frame dimension and the joint dimension to obtain temporal features and spatial features respectively. The temporal features and spatial features are merged and combined with the feature processing layer to extract the spatiotemporal features. The spatiotemporal features are processed based on the function to obtain the temporal-spatial attention map. The channel space branch reshapes the input feature dimension, extracts spatial features in conjunction with the feature processing layer, processes the spatial features based on a function, and obtains the channel space attention map.

9. A lightweight action recognition method based on EfficientGCN improvement according to claim 1, characterized in that, The process of acquiring the input video, extracting key human skeletal points from the input video, and preprocessing it to obtain an information stream includes the following steps: S21. Obtain the input video, extract the key skeletal points of the human body from the input video; S22. Store the coordinates of key human skeletal points frame by frame into the input sequence to obtain human skeletal point data; S23. Preprocess the human skeletal point data to obtain several information streams, including joint stream, bone stream and velocity stream.

10. A lightweight action recognition method based on EfficientGCN improvement according to claim 9, characterized in that, The process of extracting, fusing, enhancing, and aggregating information streams using a human motion recognition model, and determining the human motion category based on the probability of the human motion category, includes the following steps: S31. Using the basic blocks in the human motion recognition model, the joint flow, skeletal flow and velocity flow are standardized respectively. S32. Based on the dual-branch parallel temporal graph convolutional architecture in the human motion recognition model, the normalized joint flow, skeletal flow and velocity flow are extracted respectively to obtain several fused temporal features. S33. Using the hybrid multi-dimensional attention module in the human action recognition model, several fused temporal features are aggregated and enhanced to obtain several optimized temporal features. S34. Several optimized temporal features are concatenated along the channel dimension, and the concatenated optimized temporal features are aggregated based on the mainstream branches in the human action recognition model to obtain action features. S35. Based on the feature extraction network and the global average pooling layer, the action features are aggregated in terms of time and space dimensions to obtain channel-level statistical vectors. S36. Using a fully connected layer, a linear transformation is performed on the channel-level statistical vector, and the Softmax function is used to process the linearly transformed channel-level statistical vector to obtain the probability of human action categories. S37. Determine the category of human action based on the probability of human action category.

Images