A human action recognition method based on multi-stream skeleton spatio-temporal feature fusion enhancement

Abstract

The present application belongs to the technical field of deep learning and computer vision, and proposes a human action recognition method based on multi-stream skeleton spatio-temporal feature fusion enhancement, which is used to solve the problems of low recognition accuracy and insufficient model stability of human action recognition in complex dynamic environment. The method extracts skeleton key points from human action video, constructs four parallel data streams of joint stream, joint motion stream, skeleton stream and skeleton motion stream, and establishes a multi-stream spatio-temporal feature modeling network (MSFD-Net) to jointly model the skeleton topology structure and dynamic characteristics in space and time dimensions. A multi-scale context awareness architecture (MSCA) is proposed to capture short-time burst actions and long-time continuous actions, and a spatio-temporal information enhancement module (STIE) is used for adaptive weighting of key joints and key time periods, realizing the cooperative enhancement of global and local spatio-temporal features. The experimental results show that the method has high recognition accuracy and stability, and can be applied to the fields of security monitoring, human-computer interaction and behavior analysis.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, computer vision and deep learning, and in particular to a human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement, which can accurately recognize human actions in complex dynamic scenes and can be used as a perception module for intelligent robots to actively avoid obstacles and ensure safe interaction. Background Technology

[0002] With the development of intelligent monitoring, public safety, and human-computer interaction systems, human motion recognition technology has been widely applied in scenarios such as dangerous behavior detection, security patrol, and robot collaboration. Existing skeleton motion recognition methods typically rely on the position of key human joints and the skeleton topology, using graph convolutional networks and temporal convolutional networks to model spatial structure and temporal dynamic features to achieve motion recognition. However, traditional methods have certain limitations in complex dynamic scenes: spatial modeling based solely on local joint adjacency relationships struggles to capture global skeleton collaboration patterns; and temporal dynamic modeling cannot simultaneously accommodate short-duration bursts of motion and long-duration continuous motion.

[0003] Especially in multi-person interaction, rapid attack, or complex action scenarios, the collaborative relationships between key joints are influenced by factors across time and space, and existing methods struggle to fully characterize these complex spatiotemporal interaction patterns. To improve the discriminative power and robustness of human action recognition, there is an urgent need to propose a multi-stream spatiotemporal feature modeling method that can simultaneously capture local joint motion features and global skeleton collaborative relationships. Adaptive weighting of key joints and key time segments to improve action recognition performance is a key challenge currently facing the technological field. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] Existing human skeleton-based action recognition methods suffer from shortcomings in spatiotemporal feature modeling, struggling to simultaneously characterize the dynamic evolution of both short-duration bursts and long-duration continuous movements. Furthermore, insufficient fusion of joint and skeleton data streams prevents full utilization of local joint motion features and global skeleton collaboration. Simultaneously, inadequate feature enhancement for key joints and critical time segments makes it difficult to highlight core spatiotemporal interaction information in complex action scenarios, thus impacting the accuracy and robustness of dangerous action recognition. This invention effectively addresses these issues through multi-stream skeleton spatiotemporal feature modeling, a multi-scale context awareness module, and a spatiotemporal information enhancement module, significantly improving the performance of human action recognition.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement includes the following steps: To achieve the above technical objectives, the technical solution provided by this invention is as follows:

[0009] S1. The acquired human motion video is preprocessed, and the key point feature data of the human skeleton is extracted by frame-by-frame analysis to construct a skeleton dataset, which is then divided into training set and test set according to a preset ratio.

[0010] S2. Construct a multi-flow skeleton spatiotemporal feature modeling network (MSFD-Net), which includes joint flow branches and skeleton flow branches. It adopts a multi-scale context-aware module (MSCA) and a cross-branch attention mechanism to achieve parallel modeling of local joint motion features and global skeleton cooperation relationships in skeleton sequences.

[0011] S3. A spatiotemporal information enhancement module (STIE) is added to the MSFD-Net network. The module uses residual connections to maintain the integrity of the original information and uses adaptive spatial and temporal feature modulation to weight key joints and key time segments to strengthen the global and local spatiotemporal relationship and improve the action discrimination capability.

[0012] S4. The MSFD-Net network model is iteratively trained using the training set, and a discriminative multi-stream spatiotemporal feature representation is learned through an optimization algorithm to obtain a trained action recognition model.

[0013] S5. Input the human skeleton sequence to be identified into the trained network model, extract the joint flow and skeleton flow features respectively, perform element-level fusion, and perform action classification prediction through fully connected layers and SoftMax, and output the final action category judgment result.

[0014] S6. The recognition performance and generalization ability of the model are comprehensively evaluated using quantitative evaluation indicators such as classification accuracy and loss function convergence.

[0015] S7. Generate a visual image of the human skeleton, showcasing the skeletal structure and key joints, and overlay it onto the original video to intuitively present the feature extraction and recognition results at each stage.

[0016] Furthermore, in step S1, the steps of collecting and preprocessing human motion videos are as follows:

[0017] S1-1. Collect human motion videos and perform preliminary preprocessing, convert the videos into single-frame images, extract skeleton data using OpenPose, and store them by naming and classifying them according to the action category;

[0018] S1-2. The entire dataset is divided into training and test sets using a random partitioning method.

[0019] Furthermore, in step S2, the specific steps for improving the MSFD-Net network structure are as follows:

[0020] S2-1. Divide the network into joint flow branch and skeleton flow branch, and model the local joint motion features and global skeleton cooperation relationship respectively;

[0021] S2-2. Introduce a multi-scale context-aware module (MSCA) in the joint flow branch. Simultaneously model short-term burst actions and long-term continuous actions through temporal convolution with different expansion rates, and achieve adaptive weighting of multi-scale features through learnable cross-scale attention weights.

[0022] S2-3. Perform element-level fusion of joint flow and skeleton flow features to generate a complete multi-flow spatiotemporal feature representation.

[0023] Furthermore, in step S3, the specific steps for designing the STIE module are as follows:

[0024] S3-1. Perform residual connection processing on the fused multi-stream spatiotemporal features to preserve the original feature information;

[0025] S3-2. Perform global pooling and encoding on the input features in the time and space dimensions to generate time-aware features and space-aware features;

[0026] S3-3. The temporal and spatial features are concatenated along the channel dimension, and intermediate features are generated through 1×1 convolution, batch normalization, and nonlinear activation. The original channel dimension is then restored through 1×1 convolution to generate spatial and temporal modulation factors.

[0027] S3-4. Apply spatial modulation factors and temporal modulation factors to the input features respectively to achieve adaptive enhancement of key joints and key time segments, thereby obtaining enhanced features with global spatiotemporal awareness, and then integrate them with the multi-stream spatiotemporal feature modeling network.

[0028] Furthermore, in step S4, the steps for training the human skeleton data training set are as follows:

[0029] S4-1. The training model uses a gradient descent-based optimization algorithm to iteratively train the graph convolutional network to obtain a trained action recognition model. The improved MSFD-Net network structure with configured parameters is then placed in a computer with a pre-configured environment, and the dataset is randomly partitioned for training.

[0030] S4-2. After several training iterations, the loss value basically converges to a stable value and then remains essentially unchanged. As the number of training iterations of the network model continues to increase, the loss value basically converges to a stable value, indicating that the network model has achieved the expected training effect.

[0031] Furthermore, in step S5, the steps for training the human skeleton data training set are as follows:

[0032] S5-1. Input the human skeleton sequence to be identified into the trained multi-flow skeleton spatiotemporal feature modeling network model, and extract basic spatiotemporal features from the joint flow branch and the skeleton flow branch respectively.

[0033] S5-2. The extracted joint flow features and skeleton flow features are fused element-wise with weights. The adaptive combination of the two types of features is achieved through learnable fusion parameters. The fusion parameters are then input into a fully connected layer and the SoftMax function for action classification prediction, and the final action category determination result is output.

[0034] Further, in step S7, the specific steps for generating a visual image of the human skeleton are as follows:

[0035] S7-1. Perform skeleton recognition on the input human video sequence and overlay the recognized skeleton information onto the original video to intuitively present the feature extraction and recognition results at each stage;

[0036] S7-2. Extract joints with high motion saliency and label them as skeletal hotspots. Combine this with dynamic video display to highlight the temporal changes of key joints and their contribution to motion expression.

[0037] (III) Beneficial Effects

[0038] Compared with existing technologies, this invention provides a human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement, which has the following beneficial effects:

[0039] 1. This invention, through multi-stream skeleton spatiotemporal feature modeling and multi-scale context awareness modules, can simultaneously characterize local joint movements and global skeleton collaboration, fully explore key dynamic features in skeleton temporal and spatial structure, thereby accurately capturing multi-scale spatiotemporal features and effectively improving the recognition accuracy of complex movements.

[0040] 2. The introduced spatiotemporal information enhancement module can adaptively highlight key joints and key time segments, enhance the ability to identify key spatiotemporal information, integrate global and local spatiotemporal relationships, and significantly improve the ability to identify dangerous actions and rapid dynamic actions.

[0041] 3. The multi-scale attention fusion mechanism and efficient feature extraction path enable the model to maintain stable recognition performance when facing different action categories, noisy backgrounds or pose changes, demonstrating strong generalization ability.

[0042] 4. Experimental results show that experiments on datasets such as NTU RGB+D and Kinetics-Skeleton verify its superior recognition accuracy and computational efficiency. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall network framework of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention.

[0044] Figure 2 This is an architecture diagram of the multi-stream spatiotemporal feature modeling module of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention;

[0045] Figure 3 This is a structural diagram of the MSCA module of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention;

[0046] Figure 4 This is a diagram of the spatiotemporal feature modulation module architecture of a human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement as described in this invention.

[0047] Figure 5 This is a visualization result of the action skeleton sequence of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention.

[0048] Figure 6 The figure shows the experimental results of the loss rate of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention.

[0049] Figure 7 This is a confusion matrix diagram of the human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention.

[0050] Figure 8 This is an accuracy graph of the human motion recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement described in this invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Example

[0053] This invention uses the publicly available human motion recognition dataset NTU RGB+D 60 and Kinetics - Skeleton as validation platforms. Standard human motion videos are collected, and OpenPose is used to extract skeleton key points, ultimately constructing a skeleton dataset. The network structure and training process are implemented using the Python programming language and the PyTorch framework. The experimental hardware platform is a computer equipped with a 16th Gen Xeon(R) Gold 6430 and an NVIDIA GeForce RTX 4090 (24GB) high-performance graphics card.

[0054] like Figure 2 As shown, this invention provides a human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement, comprising the following steps:

[0055] S1. Collect and preprocess human motion videos, extract single-frame human skeleton feature data, and build a dataset. Specific steps are as follows:

[0056] Video data was collected from human motion video datasets such as NTU RGB+D60 and Kinetics-Skeleton, and processed frame by frame to extract continuous single-frame images. Then, the OpenPose human pose recognition algorithm was used to extract skeleton key points from each frame to obtain a human skeleton coordinate sequence. The obtained skeleton data was divided into training and testing sets in an 8:2 ratio, and normalization was performed to eliminate the impact of scale differences on training results. The skeleton sequences were standardized in length, and outliers were suppressed for noise.

[0057] Further, in step S2, after preprocessing the original human skeleton data, the joint positions and skeleton topological features are extracted, and static and dynamic data streams of joints and skeleton are constructed. The specific steps for time difference processing using a multilayer perceptron are as follows:

[0058] S2-1. Constructing Joint Flow Joint movement flow Skeleton Flow and skeletal motion flow Four data streams;

[0059] S2-2. Apply a multilayer perceptron to each data stream for temporal difference processing to extract static structure and dynamic motion features. To enhance feature representation, a linear mapping needs to be performed between the difference vector and the key point position vector to unify their representation space and improve distinguishability. This represents the input vector sequence. b1 and b2 are the activation functions and the bias values, respectively.

[0060]

[0061]

[0062]

[0063] S2-3. Normalize the features through standardized mapping to obtain the basic skeleton feature representation that can be input into the network. The mapping method can be set as follows:

[0064]

[0065] Furthermore, in step S3, the specific steps for designing parallel modeling of the static and motion data streams of joints and skeleton are as follows:

[0066] S3-1. Input the four data streams into the MSFD-Net network for parallel modeling: In the spatial dimension, a graph convolutional network is used to capture the skeleton topology and inter-joint cooperation relationships; in the temporal dimension, a temporal convolutional network is used to model the dynamic evolution features of the action sequence, i.e.:

[0067]

[0068]

[0069] in, Let p be the p-th sub-adjacency matrix. Here, p represents the number of spatial subsets, and p represents the corresponding learnable weight parameters. This represents the parameters of the temporal convolution kernel.

[0070] S3-2. A multi-scale context-aware module (MSCA) is proposed: it uses temporal convolutions with different dilation rates to simultaneously model short-term burst actions and long-term continuous actions, and achieves adaptive weighting of multi-scale features through learnable cross-scale attention weights.

[0071]

[0072]

[0073] in, This represents the set of timescales corresponding to different expansion rates, with different branches focusing on short-term explosive actions and long-term continuous actions, respectively.

[0074] S3-3. Element-level fusion of joint flow and skeleton flow features to generate a complete multi-flow spatiotemporal feature representation:

[0075]

[0076] Similarly, the skeleton data stream and skeleton motion stream are fused to produce output features:

[0077]

[0078] This fusion method can effectively complement static structural information and dynamic change information while maintaining consistency in feature scale.

[0079] Furthermore, in step S4, the specific steps for proposing the spatiotemporal information enhancement module are as follows:

[0080] S4-1. Multi-stream feature input spatiotemporal information enhancement module after fusion: Maintains the integrity of the original information through residual connections, and performs global pooling on the temporal and spatial dimensions respectively to generate time-aware features and spatial-aware features:

[0081]

[0082]

[0083] To fully explore the long-term spatiotemporal dependencies in human skeleton sequences, the input features are encoded one-dimensionally in both the temporal and spatial dimensions. Pooling kernels of size (1,T) and (V,1) along the temporal and spatial directions are used respectively to compress sequence information and extract global features, with node v... i The output of the Cth channel express.

[0084] S4-2. Spatial and temporal modulation factors are generated through 1×1 convolution, batch normalization, and nonlinear activation functions. These modulation factors are then applied to the input features to achieve adaptive enhancement of key joints and key time segments.

[0085]

[0086]

[0087] in This indicates a channel-level concatenation operation. Here, f is the ReLU activation function, and f is the intermediate feature map with channel dimension . r is a channel compression ratio parameter used to reduce computational complexity and enhance the compactness of feature representation.

[0088] S4-3. The enhanced global spatiotemporal awareness features are obtained and seamlessly integrated with the original multi-stream features.

[0089]

[0090] As a core component of STIE, TSFM can adaptively highlight key joints and key time segments in action sequences, significantly enhancing the ability to identify dangerous attack actions, and seamlessly integrates with multi-stream spatiotemporal feature modeling networks to achieve global spatiotemporal feature enhancement.

[0091] Furthermore, in step S5, the network training steps are as follows:

[0092] S5-1. Before model training begins, set the following hyperparameters to ensure effective network convergence: Initial learning rate is set to 0.001 to control the step size of parameter updates; batch size is set to 32, meaning 32 samples are selected from the training set to form a mini-batch in each iteration; the number of training epochs is set to 100, indicating that the entire training set will be iterated 100 times to fully optimize the model parameters; weight decay is set to 5 × 10⁻⁶. -4 It is used to suppress overfitting and improve the generalization ability of the model;

[0093] S5-2. Deploy the network structure with completed parameter settings to the computing platform with the configured training environment, and train the model using the allocated training set image data. During the training process, the loss function value gradually decreases and tends to stabilize. As the number of training rounds increases, the loss gap of the training set continuously narrows and eventually converges, indicating that the network model has achieved the expected training effect.

[0094] Step S6, Result Testing: After training is complete, the learned optimal model is used to predict the human motion video to be tested, and the results are obtained.

[0095] Step S7: To accurately evaluate the results of the human motion video assessment, this study employed a series of quantitative evaluation indicators, including Top-1 accuracy, Top-5 accuracy, and loss rate, to quantify the results. Experiments validated the effectiveness and accuracy of the established model, and the method was applied in practice with good results. The relevant calculation formulas are as follows:

[0096]

[0097]

[0098] Furthermore, in step 8, the specific steps are as follows:

[0099] S8-1. The OpenPose deep pose estimation algorithm is used to process the input video sequence frame by frame to extract the coordinates of key points of the human body's two-dimensional skeleton. Based on the recognition results, a visual skeleton image is generated, and the skeleton lines and key joint nodes are superimposed on the original video frames to form a skeleton-original image overlay, which intuitively shows the dynamic changes of human posture over time. Among them, key joints with high motion activity are highlighted as skeletal hotspots to reflect the temporal key areas during the execution of the action.

[0100] S8-2. The overlay sequence can be synthesized into a video file, preferably in AVI format, for subsequent analysis, display, and archiving. This visualization result can be used for dynamic monitoring during training, comparative analysis of feature extraction effects at each stage of the model, and interpretable presentation of recognition results, improving the overall system's debugging efficiency and application visualization level.

[0101] Matters not covered in this invention are common knowledge.

[0102] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement, characterized in that, Includes the following steps: S1. Dataset preprocessing: Collect human motion videos, perform standardized preprocessing on the video data, extract key features of human skeleton from the video sequence frame by frame, and divide the processed skeleton dataset into training set and test set according to a preset ratio. S2. Skeleton Feature Separation and Extraction Steps: After preprocessing the original human skeleton data, joint positions and skeleton topological features are extracted. Static and dynamic data streams of joints and skeletons are constructed and processed by temporal difference through a multilayer perceptron. At the same time, the sequence length is unified, noise is suppressed, and features are mapped to obtain the basic skeleton feature representation that can be input into the network. S3. Multi-stream spatiotemporal feature modeling steps: The static and motion data streams of joints and skeleton are modeled in parallel, with a total of four streams. In the spatial dimension, graph convolution is used to capture the topological structure of the skeleton and the joint coordination relationship. In the temporal dimension, temporal convolution is used to characterize the dynamic evolution features of the action sequence. Batch normalization and activation functions are combined to enhance the nonlinear expression. S4. Context-aware step: The proposed multi-scale context-aware module (MSCA) is introduced into the joint flow and skeleton flow respectively to jointly model short-term burst actions and long-term continuous actions. Adaptive weighting is achieved through learnable cross-scale attention weights, and the features of the joint flow and skeleton flow are fused at the element level to obtain a complete spatiotemporal feature representation. S5. Spatiotemporal Information Enhancement Step: The proposed Spatiotemporal Information Enhancement Module (STIE) is processed based on the spatiotemporal feature inputs of the fused joint flow and skeleton flow. The original information is kept intact through residual connections, and key joints and key time segments are weighted by using cross-branch attention mechanism and adaptive temporal and spatial feature modulation to strengthen the global and local spatiotemporal relationship and obtain enhanced features with spatiotemporal awareness, thereby improving the ability to discriminate human movements and seamlessly integrating with the multi-flow spatiotemporal feature modeling network. S6. Classification and prediction steps: The enhanced joint branch and skeleton branch features are input into the fully connected layer and global average pooling for mapping and compression, respectively. SoftMax classification and prediction are performed on the joint flow and skeleton flow, and the two prediction results are fused to obtain the final human action recognition result. S7. Performance Evaluation: The recognition performance and generalization ability of the human skeleton recognition model are comprehensively evaluated using quantitative evaluation indicators such as classification accuracy and loss function convergence. S8. Result Visualization: Generate a visual image of the human skeleton based on the network output results, showing the distribution of hotspots of attention of the human skeleton structure and key joints, so as to achieve intuitive analysis of the recognition process.

2. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S1, the data preprocessing includes cutting the video into single-frame images and extracting human skeleton data through OpenPose, dividing it into training set and test set.

3. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S2, the specific steps for extracting joint positions and skeleton topological features, constructing static and dynamic data streams of joints and skeleton, and performing temporal difference processing using a multilayer perceptron are as follows: S2-1. Preprocess the original human skeleton data to extract joint position features and skeleton topology features; S2-2. Construct joint data stream, joint motion stream, skeleton data stream, and skeleton motion stream respectively, and perform temporal difference processing through multilayer perceptron (MLP) to characterize the feature differences between static structure and dynamic motion in human movement; S2-3. The skeleton sequence is processed to a uniform length, and noise suppression and feature mapping are performed to obtain the basic skeleton feature representation that can be input into the network.

4. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S3, the static and motion data streams of the joints and skeleton are modeled in parallel. The specific steps for enhancing the nonlinear expression by combining batch normalization and activation functions are as follows: S3-1. Parallel modeling of joint data stream, joint motion stream, skeleton data stream, and skeleton motion stream; modeling skeleton topology through graph convolutional networks and temporal convolutional networks to capture inter-joint cooperative relationships; and enhancing nonlinear expression by combining batch normalization and activation functions. S3-2. A multi-scale context-aware module is proposed, which jointly models short-term burst actions and long-term continuous actions through temporal convolution with different dilation rates, and applies learnable cross-scale attention weights to the multi-scale branches to achieve adaptive weighting of multi-scale features. S3-3. Perform element-level fusion of joint flow features and skeleton flow features respectively to obtain spatiotemporal feature representations of joint branches and skeleton branches.

5. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S5, the spatiotemporal features of the fused joints and skeleton are further enhanced with information. The specific steps for integrating the global and local spatiotemporal relationships of multi-stream features through residual connections and cross-branch attention mechanisms are as follows: S5-1. Input multi-stream spatiotemporal features into the spatiotemporal information enhancement architecture, and adaptively weight key joints and key time segments through the spatiotemporal feature modulation module; S5-2. Perform global pooling and encoding on the input features in the time and spatial dimensions respectively to obtain time-aware features and spatial-aware features; S5-3. The time-aware features and spatial-aware features are concatenated along the channel dimension and mapped through 1×1 convolution, batch normalization and non-linear activation function to generate intermediate feature representations; then the intermediate features are decomposed along the time and space dimensions, the original channel dimensions are restored through two 1×1 convolution layers, and spatial modulation factors and time modulation factors are generated by Sigmoid normalization. S5-4. The spatial modulation factor and the temporal modulation factor are applied to the input features respectively to achieve adaptive enhancement of key joints and key time segments, thereby obtaining enhanced global spatiotemporal perception features; The enhanced features can be directly fused with a multi-stream skeleton spatiotemporal feature modeling network to improve the discrimination ability of human action recognition.

6. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S6, the MSFD-Net network, the MSCA module, and the STIE module are deployed to a pre-configured computing platform, and the network model is iteratively trained on the computing platform to optimize the skeleton recognition performance and perform action recognition determination.

7. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: Step S7 employs various quantitative evaluation metrics, including accuracy and loss rate, to conduct a quantitative analysis of the classification results.

8. The human action recognition method based on multi-stream skeleton spatiotemporal feature fusion enhancement according to claim 1, characterized in that: In step S8, a skeleton recognition operation is performed on the input human video sequence, and the recognized skeleton information is superimposed on the original video in a visual manner, so as to intuitively present the feature extraction and recognition results of each stage in the processing flow, and to compare and analyze the expression effect of different stages.

Images