Skeleton action recognition method and system based on dynamic and static separation strategy

Abstract

The invention discloses a graph convolution human skeleton action recognition method and system based on a dynamic and static partition strategy. The method comprises the steps that a human behavior video is acquired and preprocessed; constructing a dynamic and static partition mask strategy; constructing a space-time channel attention mechanism module which comprises a space-time attention sub-module and a channel attention sub-module; integrating a dynamic and static partition mask strategy and a time-space channel attention mechanism module into a CTR-GCN model based on a PYSKL library to obtain a new human skeleton action recognition model; and sending the features processed by the new action recognition model into a classifier to obtain a prediction classification score, thereby obtaining a final recognition result. Compared with the prior art, the method has the advantages that the motion information in the motion sequence is fully paid attention to, the model can accurately capture the crucial motion part, the attention mechanism is combined, the feature expression ability based on the gradient mask is remarkably improved, and the recognition and classification precision of the complex motion is remarkably optimized.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning, and specifically relates to a skeleton action recognition method and system based on a static-dynamic separation strategy. Background Art

[0002] Human motion recognition technology is widely used in many fields such as intelligent monitoring, virtual reality, human-computer interaction, medical health, etc. It has excellent application prospects and is an important and highly concerned research topic in the field of computer vision. Compared with traditional feature extraction methods based on videos or images, recognition methods based on skeleton data have the advantages of strong anti-interference, sensitivity to posture and structure, small data volume and high computational efficiency. Therefore, they have significant advantages in scenarios with high real-time requirements.

[0003] With the development of deep learning technology, deep learning methods have made great progress in the skeleton-based field, and convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been applied to the field of skeleton action recognition. However, skeletons naturally form a graph structure in non-Euclidean space. These methods either ignore the skeleton topology information between joints, or the design process is complicated and cannot utilize the graph structure of skeleton data. It is difficult to generalize to any form of skeleton, making these methods less competitive than graph-based methods in popular benchmarks. In contrast, inspired by the physical topological structure of the human body, graph convolutional networks (GCNs) extend convolution from images to graph structures and have been successfully applied to many scenarios.

[0004] Despite this, existing skeleton action recognition models still face some challenges, especially in recognizing complex actions with obvious changes in details. Action sequences contain both dynamic information with drastic changes (such as rapid movement of limbs) and relatively stable static information (such as the relative immobility of the head or body). Traditional methods often do not fully utilize the different features of these dynamic and static areas, and therefore have limitations in capturing detailed changes and the overall coherence of the action. Summary of the invention

[0005] The present invention is conducted to solve the above-mentioned problems, and its purpose is to propose a skeleton action recognition method and system based on the dynamic and static separation strategy. By combining the partition mask strategy and the spatiotemporal channel attention mechanism, the model is optimized to extract and process the dynamic and static area features, thereby effectively improving the recognition accuracy and robustness of complex actions.

[0006] The technical solution adopted by the present invention is as follows: a skeleton motion recognition method and system based on a dynamic-static separation strategy, comprising a power supply module, a motion acquisition module, a data transmission module, a processor module, an image processing module, a skeleton motion recognition module, and a recognition feedback module, wherein the output end of the power supply module is connected to the input end of the motion acquisition module, the output end of the motion acquisition module is connected to the input end of the data transmission module, the output end of the data transmission module is connected to the input end of the processor module, the output end of the processor module is connected to the input end of the image processing module, the output end of the image processing module is connected to the input end of the skeleton motion recognition module, and the output end of the skeleton motion recognition module is connected to the input end of the recognition feedback module.

[0007] In a preferred embodiment, the motion acquisition module uses Hikvision's EZVIZ network camera as the image acquisition device terminal. The terminal side of the motion acquisition module uses a Python program to complete the acquisition of image sources and requests to the server. First, obtaining the image data of the network camera requires the specific IP address and port number Port, user name user, password and other information of the network camera, and then uses the VideoCapture method in OpenCV to obtain the video stream image of the camera according to the real-time streaming protocol.

[0008] In a preferred embodiment, the skeleton action recognition module includes the following steps: constructing a dynamic and static partitioning mask strategy; performing mask pooling to extract partition features; constructing a spatiotemporal channel attention mechanism module, including two sub-modules: spatiotemporal attention and channel attention; integrating the dynamic and static partitioning mask strategy and the spatiotemporal channel attention mechanism module into the CTR-GCN model based on the PYSKL library to obtain a new human skeleton action recognition model.

[0009] In a preferred embodiment, the data transmission module selects a USB interface as the external communication physical interface of the system. During the communication data transmission process, the data transmission module selects a TPIC8101DWR sensor interface as the USB interface of the encrypted transmission system of the system. The working power supply voltage of this sensor interface is 5V. Applying the HS-825-100 transmission device to the encrypted transmission line of the system and connecting it to the USB interface can realize the compilation and conversion of the bit stream signal, and can also realize the transmission and storage of skeleton motion data.

[0010] In a preferred embodiment, the image processing module performs wavelet transformation processing on the human motion image, and the steps of the image processing module are as follows: the power supply module decomposes the original signal into multiple wavelet basis functions of different scales; the "scale" here refers to the time or space resolution of the wavelet function, which is usually recursively decomposed by binary division; the motion acquisition module transforms the wavelet basis function at each scale to obtain wavelet coefficients; the wavelet coefficients reflect the frequency characteristics and time domain characteristics of the original signal at this scale; the data transmission module selects to retain certain scales and wavelet coefficients, and performs compression and noise reduction and other processing; then the wavelet coefficients are inversely transformed to obtain a reconstructed signal.

[0011] In a preferred embodiment, the image processing module performs sharpening filtering and enhancement processing on the human action image, including the following steps: the power supply module performs blur processing on the original human action image to smooth out the low-frequency components in the human action image, and a commonly used Gaussian blur filter is used; the action acquisition module subtracts the blurred image from the original image to obtain a high-frequency component, that is, edge information, and this step is achieved by a differential operator high-pass filter; the data transmission module performs weighted synthesis on the obtained high-frequency component and the original image to obtain an enhanced image, and the formula used for weighted synthesis is shown in the following formula: Tp=To+k*Th Where Tp represents the original image, Th represents the high-frequency component, and k is an adjustment parameter with a value of 1 to 2.

[0012] In a preferred embodiment, step 1 in the skeleton action recognition module to construct a dynamic and static partition mask strategy includes the following four sub-steps: Step S1-1, the characteristics of each sample are averaged in the time dimension to obtain the estimated value of the static characteristics. The main purpose of this operation is to eliminate the time changes in the action sequence, so as to obtain relatively stable information about the relative position and state of each joint point during the action execution. This information will be used as a static benchmark for further analysis. The average characteristics in the time dimension are defined as: ,in represents the bone feature of the i-th sample at time t, where T is the number of time steps.

[0013] Furthermore, a structure containing two layers of fully connected networks is used to process the initial static features to introduce learnable parameters. The formula in this process is: , in this process, and is the weight matrix of the fully connected layer, and is the bias term.

[0014] Through the back-propagation algorithm, the model can continuously correct these learnable parameters, so that the static features are no longer fixed average values, but learnable representations that can dynamically adapt to changes in data. This flexibility improves the adaptability and accuracy of the model.

[0015] Step S1-2, the acquired skeleton sequence features , using differential processing to generate an initial mask. This step is introduced because previous studies failed to fully consider the information of the time dimension when extracting dynamic area features in action sequences. Therefore, the present invention proposes to generate an initial mask based on differential calculation of the input feature map, the purpose of which is to capture the instantaneous changes in motion and lay the foundation for subsequent mask optimization and dynamic feature extraction. The specific process mathematical expression is as follows: ,in Represents the activation function.

[0016] Step S1-3, using the Global Average Pooling (GAP) technique, the acquired skeleton sequence features and relative static characteristics A calculation is performed to evaluate the cosine similarity between the two to quantify the difference between the feature and the static baseline. The mathematical representation of this process is: ,in, Represents the dot product. Through this process, we can identify the regions that have a significant impact on the final prediction results, and these regions will be focused on in the subsequent steps. The back propagation technology will return the gradient information of the similarity to the skeleton sequence features and pool them in the time and channel dimensions to obtain the importance weights : ,in Represents the activation function. After obtaining the importance weights, the gradient information is further converted into a class activation map (CAM). The class activation map is obtained by averaging the importance weights, which is specifically expressed as: ,in Represents the activation function, and C represents the number of channels. Using the class activation map, the model can not only better understand the relationship between features, but also provide a clear perspective for subsequent decision-making.

[0017] Step S1-4 combines the initial mask with the generated cam to generate the final partition mask. This step effectively integrates dynamic features and static features to ensure that the model can perform effective action recognition in a variety of complex situations. The formula is expressed as: ,in, represents element-wise multiplication, The adjacency matrix of the action sequence used for smoothing can further enhance the spatiotemporal dependency of features. With the help of smoothing, the mask can not only accurately capture the dynamic features, but also retain key information during the evolution of the action, avoiding the degradation of recognition performance caused by noise or irrelevant features.

[0018] In a preferred embodiment, step S2 in the skeleton action recognition module performs mask pooling to extract partition features, which includes the following two sub-steps: Step S2-1, define input feature map and mask: Input feature map , where N is the batch size, C is the number of channels, T is the number of time steps, and V is the number of nodes; mask A binary identifier used to represent a specific area, which comes from the partitioning strategy defined in step S2. The purpose of this mask is to help the model focus on key information during feature extraction by identifying dynamic and static areas.

[0019] Step S2-2, feature extraction is performed through masking to obtain dynamic region features and static region features. The specific calculation method is: Calculation of dynamic area feature p: , calculation of static area feature c: ,in, is a very small value used to prevent division by zero errors.

[0020] In a preferred embodiment, step S3 in the skeleton action recognition module constructs a spatiotemporal channel attention mechanism module, firstly processes the input features through the spatiotemporal attention submodule to generate a spatiotemporal joint weight matrix, and then multiplies the weight matrix with the original input to obtain the updated features. The process formula is expressed as follows: ,in, , , is the corresponding weight, and They are Sigmoid and Hardswish activation functions respectively.

[0021] Furthermore, the obtained features are passed to the channel attention submodule for processing. In the channel module, the input features are first pooled in time and space dimensions, and then calculated to obtain the comprehensive features: , Further global average pooling is performed to generate a global feature representation of each channel in the entire time series and spatial dimension: .

[0022] The weights are calculated through two fully connected layers and combined with a nonlinear activation function to output the final channel attention weights: ,in, and is the corresponding weight, and They are Sigmoid and Hardswish activation functions respectively.

[0023] Finally, the channel attention weight is multiplied by the spatiotemporal attention weight, and the result is added to the input feature Multiply again to form the final output features of the spatiotemporal channel attention module: ,At the same time, residual connections are used to enhance the stability of the module.

[0024] In a preferred embodiment, step S4 in the skeleton action recognition module integrates the dynamic and static partition mask strategy and the spatiotemporal channel attention mechanism module into the CTR-GCN model based on the PYSKL library, thereby constructing a new human skeleton action recognition model. The new model uses the CTR-GCN model as the benchmark model, and the structure includes an architecture consisting of ten layers of feature extraction modules, whose output channels are 64, 64, 64, 64, 128, 128, 128, 256, 256, 256, 10 layers of graph convolutional network (GCN) and temporal convolutional network (TCN) combined units. The GCN module first performs a depth-separable convolution process on the input feature vector, which is divided into three parallel branches. The features in each branch are processed by the CTR-GC basic block, and the processing results are finally merged and passed to the TCN module. In the TCN module, a multi-branch temporal convolutional network is constructed, which includes four different branch structures. Each branch starts with a 1×1 convolution to reduce the channel dimension and enhance computational efficiency. The first three branches are each embodied in the design of two temporal convolutional layers with different expansion rates and one maximum pooling layer. The results obtained by splicing after processing each branch constitute the final output of the network, providing rich feature information for subsequent processing.

[0025] In the ten-layer model, after the second and eighth layers, the output feature data will be processed by the spatiotemporal channel attention module, which aims to capture the detailed features in the skeleton action sequence more finely, and improve the accuracy of action recognition by increasing the model's attention to relevant features. The processed features will then be passed to step S2, where the dynamic and static separation strategy is applied for similarity comparison, and finally a dynamic and static partition mask is generated. This mask plays a key role in the subsequent step S3, which finally extracts meaningful dynamic and static features by weighting the skeleton sequence features by dynamic and static. At the same time, residual connections are added in the second and eighth layers to enhance the stability of the model.

[0026] In a preferred embodiment, the feedback method of the identification feedback module includes, when it is detected that the page is in a triggerable state, notifying the skeleton motion detection system to start the skeleton motion self-loop detection, so that the camera device and / or the touch detection device in the skeleton motion detection system automatically and cyclically detects whether there is skeleton motion information; determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system; triggering the page to perform the response action; wherein, determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system further includes: querying the response action corresponding to the layout state of the page and the skeleton motion information in a pre-set response information library, and determining the queried response action as the response action to be performed by the page; the layout state of the page refers to the distribution of page elements in the page. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is the overall system block diagram of the present invention; Figure 2 This is a system block diagram of the skeleton action recognition module of the present invention; Figure 3 This is a schematic diagram of the structure of the dynamic and static partition mask strategy of the present invention; Figure 4 This is a schematic diagram of the structure of the spatiotemporal channel attention mechanism module of the present invention; Figure 5 This is a schematic diagram of the accuracy of the present invention on the NTU RGB+D 60 dataset.

[0028] Figure 6 It is a schematic diagram of the accuracy of the present invention on the NTU RGB+D 120 dataset.

[0029] Markings in the figure: 1-power supply module, 2-motion acquisition module, 3-data transmission module, 4-processor module, 5-image processing module, 6-skeleton motion recognition module, 7-recognition feedback module. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution, creative features and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Example

[0031] A skeleton motion recognition feature system and a feedback method thereof, comprising a power supply module, a motion acquisition module, a data transmission module, a processor module, an image processing module, a skeleton motion recognition module, and a recognition feedback module, wherein the output end of the power supply module is connected to the input end of the motion acquisition module, the output end of the motion acquisition module is connected to the input end of the data transmission module, the output end of the data transmission module is connected to the input end of the processor module, the output end of the processor module is connected to the input end of the image processing module, the output end of the image processing module is connected to the input end of the skeleton motion recognition module, and the output end of the skeleton motion recognition module is connected to the input end of the recognition feedback module.

[0032] The motion acquisition module uses Hikvision's EZVIZ network camera as the image acquisition device terminal. The terminal side of the motion acquisition module uses a Python program to complete the acquisition of image sources and requests to the server. First, to obtain the image data of the network camera, the specific IP address and port number Port, username user, password and other information of the network camera are required. Then, the VideoCapture method in OpenCV is used to obtain the video stream image of the camera according to the real-time streaming protocol.

[0033] The skeleton action recognition module includes the following steps: constructing a dynamic and static partitioning mask strategy; performing mask pooling to extract partition features; constructing a spatiotemporal channel attention mechanism module, including two sub-modules: spatiotemporal attention and channel attention; integrating the dynamic and static partitioning mask strategy and the spatiotemporal channel attention mechanism module into the CTR-GCN model based on the PYSKL library to obtain a new human skeleton action recognition model.

[0034] The data transmission module selects the USB interface as the external communication physical interface of this system. During the communication data transmission process, the data transmission module selects the TPIC8101DWR sensor interface as the USB interface of the encrypted transmission system of this system. The working power supply voltage of this sensor interface is 5V. Applying the HS-825-100 transmission device to the encrypted transmission line of this system and connecting it with the USB interface can realize the compilation and conversion of the bit stream signal, and can also realize the transmission and storage of skeleton motion data.

[0035] The image processing module performs wavelet transformation processing on the human motion image, and the steps of the image processing module are as follows: the power supply module decomposes the original signal into multiple wavelet basis functions of different scales; the "scale" here refers to the time or space resolution of the wavelet function, which is usually recursively decomposed by binary division; the motion acquisition module transforms the wavelet basis function at each scale to obtain wavelet coefficients; the wavelet coefficients reflect the frequency characteristics and time domain characteristics of the original signal at this scale; the data transmission module selects to retain certain scales and wavelet coefficients, and performs compression and noise reduction and other processing; then the wavelet coefficients are inversely transformed to obtain the reconstructed signal.

[0036] The image processing module performs sharpening filtering and enhancement processing on the human action image, including the following steps: the power supply module performs blur processing on the original human action image to smooth out the low-frequency components in the human action image, and a commonly used Gaussian blur filter is used; the action acquisition module subtracts the blurred image from the original image to obtain the high-frequency component, that is, the edge information, and this step is achieved through a differential operator high-pass filter; the data transmission module performs weighted synthesis on the obtained high-frequency component and the original image to obtain the enhanced image, and the formula used for weighted synthesis is shown in the following formula: Tp=To+k*Th Where Tp represents the original image, Th represents the high-frequency component, and k is an adjustment parameter with a value of 1 to 2.

[0037] Step 1 in the skeleton action recognition module constructs the static and dynamic partition mask strategy, which includes the following four sub-steps: Step S1-1, the characteristics of each sample are averaged in the time dimension to obtain the estimated value of the static characteristics. The main purpose of this operation is to eliminate the time changes in the action sequence, so as to obtain relatively stable information about the relative position and state of each joint point during the action execution. This information will be used as a static benchmark for further analysis. The average characteristics in the time dimension are defined as: ,in represents the bone feature of the i-th sample at time t, where T is the number of time steps.

[0038] Furthermore, a structure containing two layers of fully connected networks is used to process the initial static features to introduce learnable parameters. The formula in this process is: , in this process, and is the weight matrix of the fully connected layer, and is the bias term.

[0039] Through the back-propagation algorithm, the model can continuously correct these learnable parameters, so that the static features are no longer fixed average values, but learnable representations that can dynamically adapt to changes in data. This flexibility improves the adaptability and accuracy of the model.

[0040] Step S1-2, the acquired skeleton sequence features , using differential processing to generate an initial mask. This step is introduced because previous studies failed to fully consider the information of the time dimension when extracting dynamic area features in action sequences. Therefore, the present invention proposes to generate an initial mask based on differential calculation of the input feature map, the purpose of which is to capture the instantaneous changes in motion and lay the foundation for subsequent mask optimization and dynamic feature extraction. The specific process mathematical expression is as follows: ,in Represents the activation function.

[0041] Step S1-3, using the Global Average Pooling (GAP) technique, the acquired skeleton sequence features and relative static characteristics A calculation is performed to evaluate the cosine similarity between the two to quantify the difference between the feature and the static baseline. The mathematical representation of this process is: ,in, Represents the dot product. Through this process, we can identify the regions that have a significant impact on the final prediction results, and these regions will be focused on in the subsequent steps. The back propagation technology will return the gradient information of the similarity to the skeleton sequence features and pool them in the time and channel dimensions to obtain the importance weights : ,in Represents the activation function. After obtaining the importance weights, the gradient information is further converted into a class activation map (CAM). The class activation map is obtained by averaging the importance weights, which is specifically expressed as: ,in Represents the activation function, and C represents the number of channels. Using the class activation map, the model can not only better understand the relationship between features, but also provide a clear perspective for subsequent decision-making.

[0042] Step S1-4 combines the initial mask with the generated cam to generate the final partition mask. This step effectively integrates dynamic features and static features to ensure that the model can perform effective action recognition in a variety of complex situations. The formula is expressed as: ,in, represents element-wise multiplication, The adjacency matrix of the action sequence used for smoothing can further enhance the spatiotemporal dependency of features. With the help of smoothing, the mask can not only accurately capture the dynamic features, but also retain key information during the evolution of the action, avoiding the degradation of recognition performance caused by noise or irrelevant features.

[0043] Step S2 in the skeleton action recognition module performs mask pooling to extract partition features, which includes the following two sub-steps: Step S2-1, define input feature map and mask: Input feature map , where N is the batch size, C is the number of channels, T is the number of time steps, and V is the number of nodes; mask A binary identifier used to represent a specific area, which comes from the partitioning strategy defined in step S2. The purpose of this mask is to help the model focus on key information during feature extraction by identifying dynamic and static areas.

[0044] Step S2-2, feature extraction is performed through masking to obtain dynamic region features and static region features. The specific calculation method is: Calculation of dynamic area feature p: , calculation of static area feature c: ,in, is a very small value used to prevent division by zero errors.

[0045] Step S3 in the skeleton action recognition module builds a spatiotemporal channel attention mechanism module. First, the input features are processed by the spatiotemporal attention submodule to generate a spatiotemporal joint weight matrix. Then, the weight matrix is ​​multiplied by the original input to obtain the updated features. The process formula is expressed as follows: ,in, , , is the corresponding weight, and They are Sigmoid and Hardswish activation functions respectively.

[0046] Furthermore, the obtained features are passed to the channel attention submodule for processing. In the channel module, the input features are first pooled in time and space dimensions, and then calculated to obtain the comprehensive features: , Further global average pooling is performed to generate a global feature representation of each channel in the entire time series and spatial dimension: .

[0047] The weights are calculated through two fully connected layers and combined with a nonlinear activation function to output the final channel attention weights: ,in, and is the corresponding weight, and They are Sigmoid and Hardswish activation functions respectively.

[0048] Finally, the channel attention weight is multiplied by the spatiotemporal attention weight, and the result is added to the input feature Multiply again to form the final output features of the spatiotemporal channel attention module: ,At the same time, residual connections are used to enhance the stability of the module.

[0049] In step S4 of the skeleton action recognition module, the dynamic and static partition mask strategy and the spatiotemporal channel attention mechanism module are integrated into the CTR-GCN model based on the PYSKL library, thereby constructing a new human skeleton action recognition model. The new model uses the CTR-GCN model as the benchmark model, and the structure includes an architecture consisting of ten layers of feature extraction modules, whose output channels are 64, 64, 64, 64, 128, 128, 128, 256, 256, 256, 256, which constitute a ten-layer graph convolutional network (GCN) and temporal convolutional network (TCN) combination unit. The GCN module first performs depth-separable convolution processing on the input feature vector, which is divided into three parallel branches. The features in each branch are processed by the CTR-GC basic block, and the processing results are finally merged and passed to the TCN module. In the TCN module, a multi-branch temporal convolutional network is constructed, which contains four different branch structures. Each branch starts with a 1×1 convolution to reduce the channel dimension and enhance computational efficiency. The first three branches are each embodied in the design of two temporal convolutional layers with different expansion rates and one maximum pooling layer. The results obtained by splicing after processing each branch constitute the final output of the network, providing rich feature information for subsequent processing.

[0050] In the ten-layer model, after the second and eighth layers, the output feature data will be processed by the spatiotemporal channel attention module, which aims to capture the detailed features in the skeleton action sequence more finely, and improve the accuracy of action recognition by increasing the model's attention to relevant features. The processed features will then be passed to step S2, where the dynamic and static separation strategy is applied for similarity comparison, and finally a dynamic and static partition mask is generated. This mask plays a key role in the subsequent step S3, which finally extracts meaningful dynamic and static features by weighting the skeleton sequence features by dynamic and static. At the same time, residual connections are added in the second and eighth layers to enhance the stability of the model.

[0051] The feedback method of the identification feedback module includes, when it is detected that the page is in a triggerable state, notifying the skeleton motion detection system to start the skeleton motion self-loop detection, so that the camera device and / or the touch detection device in the skeleton motion detection system automatically and cyclically detects whether there is skeleton motion information; determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system; triggering the page to perform the response action; wherein, determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system further includes: querying the response action corresponding to the layout state of the page and the skeleton motion information in a pre-set response information library, and determining the queried response action as the response action to be performed by the page; the layout state of the page refers to the distribution of page elements in the page.

Claims

1. A skeleton action recognition method and system based on a dynamic and static separation strategy, comprising a power supply module, a motion acquisition module, a data transmission module, a processor module, an image processing module, a skeleton action recognition module, and a recognition feedback module, characterized in that: The output end of the power supply module is connected to the input end of the motion acquisition module, the output end of the motion acquisition module is connected to the input end of the data transmission module, the output end of the data transmission module is connected to the input end of the processor module, the output end of the processor module is connected to the input end of the image processing module, the output end of the image processing module is connected to the input end of the skeleton motion recognition module, and the output end of the skeleton motion recognition module is connected to the input end of the recognition feedback module.

2. A skeleton action recognition method and system based on a dynamic-static separation strategy as claimed in claim 1, characterized in that: The motion acquisition module uses Hikvision's EZVIZ network camera as the image acquisition device terminal. The terminal side of the motion acquisition module uses a Python program to complete the acquisition of image sources and requests to the server. First, to obtain the image data of the network camera, the specific IP address and port number Port, username user, password and other information of the network camera are required. Then, the VideoCapture method in OpenCV is used to obtain the video stream image of the camera according to the real-time streaming protocol.

3. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: The skeleton action recognition module includes the following steps: constructing a dynamic and static partitioning mask strategy; performing mask pooling to extract partition features; constructing a spatiotemporal channel attention mechanism module, including two sub-modules: spatiotemporal attention and channel attention; integrating the dynamic and static partitioning mask strategy and the spatiotemporal channel attention mechanism module into the CTR-GCN model based on the PYSKL library to obtain a new human skeleton action recognition model.

4. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: The data transmission module selects the USB interface as the external communication physical interface of this system. During the communication data transmission process, the data transmission module selects the TPIC8101DWR sensor interface as the USB interface of the encryption transmission system of this system. The working power supply voltage of this sensor interface is 5V. Applying the HS-825-100 transmission device to the encryption transmission line of this system and connecting it with the USB interface can realize the compilation and conversion of the bit stream signal, and can also realize the transmission and storage of skeleton motion data.

5. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: The image processing module performs wavelet transform processing on the human action image, and the steps of the image processing module are as follows: the power supply module decomposes the original signal into a plurality of wavelet basis functions of different scales; the "scale" here refers to the time or space resolution of the wavelet function, and the dichotomy method is usually used for recursive decomposition; The motion acquisition module transforms the wavelet basis function at each scale to obtain the wavelet coefficients; The wavelet coefficients reflect the frequency characteristics and time domain characteristics of the original signal at this scale; The data transmission module selects to retain certain scales and wavelet coefficients, performs compression and noise reduction, and then performs inverse transformation on the wavelet coefficients to obtain the reconstructed signal.

6. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: The image processing module performs sharpening filtering and enhancement processing on the human action image, including the following steps: the power supply module performs blur processing on the original human action image to smooth out the low-frequency components in the human action image, and a commonly used Gaussian blur filter is used; the action acquisition module subtracts the blurred image from the original image to obtain a high-frequency component, that is, edge information, and this step is achieved by a differential operator high-pass filter; the data transmission module performs weighted synthesis on the obtained high-frequency component and the original image to obtain an enhanced image, and the formula used for weighted synthesis is shown in the following formula: Tp=To+k*Th Where Tp represents the original image, Th represents the high-frequency component, and k is an adjustment parameter with a value of 1 to 2.

7. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: Step 1 in the skeleton action recognition module constructs a dynamic and static partition mask strategy, which includes the following four sub-steps: Step S1-1, the characteristics of each sample are averaged in the time dimension to obtain the estimated value of the static characteristics. The main purpose of this operation is to eliminate the time changes in the action sequence, so as to obtain relatively stable information about the relative position and state of each joint point during the action execution. This information will be used as a static benchmark for further analysis. The average characteristics in the time dimension are defined as: ,in represents the skeleton feature of the i-th sample at time t, where T is the number of time steps. Furthermore, a structure containing two layers of fully connected networks is used to process the initial static features to introduce learnable parameters. The formula in this process is: , in this process, and is the weight matrix of the fully connected layer, and is the bias term. Through the back propagation algorithm, the model can continuously correct these learnable parameters, so that the static features are no longer fixed average values, but learnable representations that can dynamically adapt to changes in data. This flexibility improves the adaptability and accuracy of the model. Step S1-2, the acquired skeleton sequence features , using differential processing to generate an initial mask. This step is introduced because previous studies failed to fully consider the information of the time dimension when extracting dynamic area features in action sequences. Therefore, the present invention proposes to generate an initial mask based on differential calculation of the input feature map, the purpose of which is to capture the instantaneous changes in motion and lay the foundation for subsequent mask optimization and dynamic feature extraction. The specific process mathematical expression is as follows: ,in In step S1-3, the Global Average Pooling (GAP) technique is used to obtain the skeleton sequence features. and relative static characteristics A calculation is performed to evaluate the cosine similarity between the two to quantify the difference between the feature and the static baseline. The mathematical representation of this process is: ,in, Represents the dot product. Through this process, we can identify the regions that have a significant impact on the final prediction results, and these regions will be focused on in the subsequent steps. The back propagation technology will return the gradient information of the similarity to the skeleton sequence features and pool them in the time and channel dimensions to obtain the importance weights : ,in Represents the activation function. After obtaining the importance weights, the gradient information is further converted into a class activation map (CAM). The class activation map is obtained by averaging the importance weights, which is specifically expressed as: ,in Represents the activation function, and C represents the number of channels. Using the class activation map, the model can not only better understand the relationship between features, but also provide a clear perspective for subsequent decision-making. Step S1-4 combines the initial mask with the generated cam to generate the final partition mask. This step effectively integrates dynamic features and static features to ensure that the model can perform effective action recognition in a variety of complex situations. The formula is expressed as: ,in, represents element-wise multiplication, The adjacency matrix of the action sequence used for smoothing can further enhance the spatiotemporal dependency of features. With the help of smoothing, the mask can not only accurately capture the dynamic features, but also retain key information during the evolution of the action, avoiding the degradation of recognition performance caused by noise or irrelevant features.

8. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: Step S2 in the skeleton action recognition module performs mask pooling to extract partition features, which includes the following two sub-steps: Step S2-1, define input feature map and mask: Input feature map , where N is the batch size, C is the number of channels, T is the number of time steps, and V is the number of nodes; mask A binary identifier used to represent a specific area. The identifier comes from the partitioning strategy defined in step S2. The role of this mask is to help the model focus on key information during feature extraction by identifying dynamic and static areas. Step S2-2, extract features through the mask to obtain dynamic area features and static area features. The specific calculation method is: Calculation of dynamic area feature p: , calculation of static area feature c: ,in, is a very small value used to prevent division by zero errors.

9. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: Step S3 in the skeleton action recognition module constructs a spatiotemporal channel attention mechanism module. First, the input features are processed by the spatiotemporal attention submodule to generate a spatiotemporal joint weight matrix. Then, the weight matrix is ​​multiplied by the original input to obtain the updated features. The process formula is expressed as follows: ,in, , , is the corresponding weight, and They are Sigmoid and Hardswish activation functions respectively. Further, the obtained features are passed to the channel attention submodule for processing. In the channel module, the input features are firstly pooled in time and space dimensions respectively, and then the comprehensive features are obtained by calculation: , further subjected to global average pooling, generates a global feature representation of each channel in the entire time series and spatial dimension: The weights are calculated through two fully connected layers and combined with a nonlinear activation function to output the final channel attention weights: ,in, and is the corresponding weight, and are Sigmoid and Hardswish activation functions respectively. Finally, the channel attention weight is multiplied by the spatiotemporal attention weight, and the result is added to the input feature Multiply again to form the final output features of the spatiotemporal channel attention module: ,At the same time, residual connections are used to enhance the stability of the module.

10. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: Step S4 in the skeleton action recognition module integrates the static and dynamic partitioning mask strategy and the spatiotemporal channel attention mechanism module into the CTR-GCN model based on the PYSKL library, thereby constructing a new human skeleton action recognition model. The new model uses the CTR-GCN model as the benchmark model, and the structure includes an architecture consisting of ten layers of feature extraction modules, whose output channels are 64, 64, 64, 64, 128, 128, 128, 256, 256, 256, 256, forming a ten-layer graph convolutional network (GCN) and temporal convolutional network (TCN) combination unit. The GCN module first performs depth-separable convolution processing on the input feature vector, which is divided into three parallel branches. The features in each branch are processed by the CTR-GC basic block, and the processing results are finally merged and passed to the TCN module. In the TCN module, a multi-branch temporal convolutional network is constructed, which contains four different branch structures. Each branch starts with a 1×1 convolution to reduce the channel dimension and enhance computational efficiency. The first three branches are each embodied in the design of two temporal convolutional layers with different expansion rates and a maximum pooling layer. The results obtained by splicing after processing each branch constitute the final output of the network, providing rich feature information for subsequent processing. In the ten-layer model, after the second and eighth layers, the output feature data will be processed by the spatiotemporal channel attention module, which aims to capture the detailed features in the skeleton action sequence more finely, and improve the accuracy of action recognition by increasing the model's attention to relevant features. The processed features will then be passed to step S2, and the dynamic and static separation strategy will be applied for similarity comparison, and finally a dynamic and static partition mask will be generated. This mask plays a key role in the subsequent step S3. By performing dynamic and static weighted processing on the skeleton sequence features, meaningful dynamic and static features are finally extracted. At the same time, residual connections are added in the second and eighth layers to enhance the stability of the model.

11. The skeleton action recognition method and system based on the static-dynamic separation strategy as claimed in claim 1, characterized in that: The feedback method of the identification feedback module includes, when it is detected that the page is in a triggerable state, notifying the skeleton motion detection system to start the skeleton motion self-loop detection, so that the camera device and / or the touch detection device in the skeleton motion detection system automatically and cyclically detects whether there is skeleton motion information; determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system; triggering the page to perform the response action; wherein, determining the response action to be performed by the page according to the layout state of the page and the skeleton motion information detected by the skeleton motion detection system further includes: querying the response action corresponding to the layout state of the page and the skeleton motion information in a pre-set response information library, and determining the queried response action as the response action to be performed by the page; the layout state of the page refers to the distribution of page elements in the page.

Images