Dynamic gesture recognition method based on wearable multi-sensor

Abstract

This invention proposes a dynamic gesture recognition method based on wearable multi-sensor technology, belonging to the field of behavioral feature recognition technology. The method includes the following steps: modeling the wearable multi-sensor system as a graph structure, using sensors as graph nodes, constructing a prior adjacency matrix based on physical proximity and functional coupling relationships, and then normalizing it; performing graph convolution on the preprocessed multi-channel time-series signal based on the normalized adjacency matrix, aggregating the collaborative features between each graph node and its neighboring nodes, and extracting spatial structural features; further aggregating along the graph node dimension and the time dimension to obtain a fixed-length gesture representation vector; finally, inputting the vector into a classification layer to output scores for each gesture category, with the highest score being the recognition result. This invention overcomes the problem of insufficient modeling of non-Euclidean topological information by explicitly modeling the spatial collaborative relationships between sensor nodes through graph convolution, significantly improving the accuracy and stability of gesture recognition.

Description

Technical Field

[0001] This invention relates to the field of behavioral feature recognition technology, and in particular to a dynamic gesture recognition method based on wearable multi-sensor technology, which can be applied to scenarios such as smart home control, virtual and augmented reality (VR / AR) interaction, rehabilitation training assessment, and collaborative robot control. Background Technology

[0002] Gesture recognition solutions are mainly divided into vision-based non-contact solutions and wearable sensor-based contact solutions. Vision-based solutions exhibit significant performance degradation under varying lighting conditions, occlusion, and complex backgrounds, and also present privacy and power consumption issues. Contact-based wearable solutions, on the other hand, collect kinematic signals using surface sensors such as flexible bending sensors, inertial measurement units (IMUs), and surface electromyography (sEMG). These solutions are insensitive to ambient light, have low latency, and offer good privacy, making them an increasingly important research direction.

[0003] Driven by deep learning, models such as Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), and CNN-LSTM can automatically learn temporal features from multi-channel sensor signals, achieving high recognition accuracy. However, these models essentially process multi-channel signals in an Euclidean manner, treating each sensor channel as an independent or tiled input dimension, lacking explicit modeling mechanisms for the topological relationships and cooperative motion patterns between sensor nodes. In fact, there is strong spatial cooperation between contact sensor nodes. For example, adjacent fingers move in tandem due to interphalangeal tendon connections and shared flexor and extensor muscle groups, and the overall palm posture globally modulates the bending of each finger. These non-Euclidean structural information contain key discriminative clues, but are difficult to fully capture by the local convolutional kernels of CNNs or the channel-by-channel temporal modeling of LSTMs.

[0004] Furthermore, due to differences in wearer's hand size, force application habits, and sensor placement, there are significant individual differences among users. The features learned by traditional time series models are highly dependent on the signal distribution of the training users. Under cross-user conditions (where training and test users do not overlap), covariate shifts occur, leading to a sharp decrease in the accuracy of identifying unknown users. Summary of the Invention

[0005] This invention proposes a dynamic gesture recognition method based on wearable multi-sensor technology, which solves the problem that existing machine learning models lack explicit modeling mechanisms for the topological relationships and cooperative motion patterns between sensor nodes, making it difficult to fully capture the key discrimination clues contained in non-Euclidean structure information, resulting in low recognition accuracy.

[0006] The technical solution of this invention is implemented as follows: The first aspect of this invention provides a dynamic gesture recognition method based on wearable multi-sensor devices, comprising the following steps: Step S1: Acquire multi-channel time-series signals synchronously collected by the wearable multi-sensor system and preprocess them to obtain a spatiotemporal graph tensor of uniform specifications; Step S2: Treat each sensor as a graph node, model the sensor system as a graph structure, construct a prior adjacency matrix based on the physical proximity and functional coupling relationships between sensors, and normalize the prior adjacency matrix to obtain a normalized adjacency matrix. Step S3: Using the normalized adjacency matrix as the topological basis for graph convolution, perform graph convolution operation on the spatiotemporal graph tensor, aggregate the collaborative features between each graph node and its adjacent graph nodes, and extract spatial structure features. Step S4: Aggregate the spatial structure features along the graph node dimension and time dimension to obtain a fixed-length gesture representation vector; Step S5: Input the gesture representation vector into the classification layer, output the score of each gesture category, and take the category with the highest score as the gesture recognition result.

[0007] Specifically, in step S1, the preprocessing includes: Frame number unification: Each sample is resampled along the time axis to a uniform length T using linear interpolation to obtain an aligned time reference; Normalization: Apply Z-score standardization to each channel to eliminate dimensional differences; Node feature alignment: When the feature dimensions of different graph nodes are inconsistent, all graph nodes are unified to the C dimension by copying and filling or linear mapping; After preprocessing, each sample is organized into an N×C×T spatiotemporal graph tensor, where N is the total number of graph nodes, C is the node feature dimension, and T is the unified time step.

[0008] Specifically, in step S2, the graph structure G is represented as: G=(V, E, A); Where V is the set of sensor nodes, with each sensor corresponding to one sensor node; E is the set of edges, with each edge corresponding to the connection between two sensor nodes; and A is the adjacency matrix, used to represent the connection relationship between each sensor node. The elements of the prior adjacency matrix are defined as follows: ; in, Indicates the position of the first element in the prior adjacency matrix. line, number Column elements; The principles for establishing edges include: establishing local coupling connections between physically adjacent or functionally coupled graph nodes; establishing global reference connections between reference nodes reflecting the overall attitude and each local node; and establishing self-connections for each graph node. The formula for normalizing the prior adjacency matrix is: ; in, This is the normalized adjacency matrix; For degree matrix, The prior adjacency matrix before normalization. It is an identity matrix.

[0009] Preferably, in step S2, a learnable adaptive perturbation matrix is ​​superimposed on the prior adjacency matrix: ; in, This is the adaptive adjacency matrix after superimposing the adaptive perturbation matrix; For the adaptive perturbation matrix, Shared globally among all users and samples, initialized as an all-zero matrix, with a parameter count of [number missing]. , This represents the total number of nodes in the graph. To constrain the perturbation amplitude, a bounded activation function is used. Compress and then stack: ; in, It is the hyperbolic tangent function; right Substitution after normalization Used for subsequent graph convolution operations.

[0010] Specifically, in step S3, performing graph convolution operations on the spatiotemporal graph tensor includes: At each time step, the graph node feature matrix is ​​multiplied by the normalized adjacency matrix to update the features of each graph node to a weighted sum of its own features and the features of its neighboring graph nodes, thereby realizing the propagation of collaborative information between sensing nodes. The aggregated features are then linearly transformed using a learnable weight matrix to change the feature channel dimension, and non-linear activation is applied to obtain the output of the graph convolution of this layer. No. The operation of layer graph convolution is represented as: ; in, For the first The node features of the layer input, For the normalized adjacency matrix, For the first The learnable weight matrix of the layer, It is a non-linear activation function; For the first The node features output by the first layer are used as the first layer's node features. The node features of the layer input.

[0011] Specifically, in step S4, the spatial structure features are aggregated along the graph node dimension and the time dimension using global average pooling: ; in, For gesture representation vectors, The total number of nodes in the graph. For time steps, Spatial features output by graph convolution at nodes Time step eigenvectors.

[0012] Optionally, on the spatial features of the graph convolution output, a one-dimensional convolution is first applied along the time axis to perform temporal modeling. The one-dimensional convolution only slides in the time dimension and does not change the graph node dimension. The one-dimensional convolution operation and the graph convolution operation are stacked alternately and combined with residual connections to form a spatiotemporal block. Then, the output of the spatiotemporal block is subjected to global average pooling along the graph node dimension and the time dimension to obtain a gesture representation vector of fixed length.

[0013] Specifically, in step S5, the gesture representation vector is... Input a fully connected classification layer and output scores for each gesture category: ; in, This represents the original score vectors for each category output by the classification layer; This is the weight matrix of the classification layer. For the bias term of the classification layer; The cross-entropy loss function is used throughout the network training phase, and the average loss over all samples is taken as the total loss. ; in, The total number of samples in the batch. For the first One-hot encoding of each sample category label, For the model to the first The predicted value for the nth sample is... The probability of a class This represents the total loss value.

[0014] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the gesture recognition method.

[0015] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the gesture recognition method.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention models the wearable multi-sensor system as a graph structure, with sensors as graph nodes and physical proximity and functional coupling relationships defining graph edges. It also uses graph convolution operations to aggregate the collaborative features of adjacent nodes along the graph topology, thereby realizing explicit modeling of the non-Euclidean space collaborative relationship between sensor nodes. This overcomes the shortcomings of traditional Euclidean models such as CNN and LSTM, which isolate each sensor channel and fail to capture topological discrimination information, and significantly improves the accuracy and stability of gesture recognition. (2) This invention superimposes a globally shared, zero-initialized, learnable adaptive perturbation matrix with only N×N parameters onto the prior adjacency matrix. The zero initialization provides a physical prior starting point with good generalization. The extremely small number of parameters fundamentally limits the capacity of the model to remember user-specific noise. The globally shared gradient cancellation mechanism forces the perturbation to retain only the corrections that are beneficial to all users. Under the premise of almost no increase in model complexity and inference overhead, it significantly improves the generalization ability in cross-user scenarios and the recognition accuracy in general scenarios, effectively solving the problem of the sharp decline in recognition accuracy of traditional methods under cross-user conditions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the dynamic gesture recognition method based on wearable multi-sensor according to the present invention.

[0019] Figure 2 This is an overall architecture diagram of the dynamic gesture recognition method based on wearable multi-sensor of the present invention.

[0020] Figure 3 This is a comparison diagram of the initial adjacency matrix and the adaptive adjacency matrix in an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] The core method of this invention lies in: introducing graph neural networks into gesture recognition for contactless wearable sensors, abstracting each sensor as a graph node, and abstracting the collaborative relationships between sensor nodes as edges of the graph, explicitly aggregating the collaborative information between nodes using graph convolution to improve user recognition accuracy and stability; and superimposing a learnable adaptive adjacency matrix, globally shared among all users and samples, initialized to all zeros, with only N×N parameters, onto a static prior graph constructed based on the sensor topology. End-to-end refinement of the graph topology improves cross-user recognition accuracy.

[0023] Reference Figures 1 to 3 The first aspect of this invention provides a dynamic gesture recognition method based on wearable multi-sensor devices, comprising the following steps: Step S1: Acquire multi-channel time-series signals synchronously collected by a wearable multi-sensor system (in this embodiment, the sensors can be one or more of flexible bending sensors, IMUs, sEMGs, etc.) and preprocess them to obtain a spatiotemporal graph tensor of uniform specifications; the preprocessing includes: Frame number unification: Each sample is resampled along the time axis to a uniform length T using linear interpolation to obtain an aligned time reference; Normalization: Apply Z-score standardization to each channel to eliminate dimensional differences; Node feature alignment: When the feature dimensions of different graph nodes are inconsistent, all graph nodes are unified to the C dimension by copying and filling or linear mapping; After preprocessing, each sample is organized into an N×C×T spatiotemporal graph tensor, where N is the total number of graph nodes, C is the node feature dimension, and T is the unified time step.

[0024] This step transforms the original heterogeneous signals from multiple sensors into a unified spatiotemporal graph tensor by standardizing the number of frames, Z-score normalization, and node feature alignment. This provides standardized data input for subsequent graph convolution operations, ensuring that signals from different sensor types and with different dimensions can be processed jointly within the same mathematical framework.

[0025] Step S2: Treat each sensor as a graph node, model the sensor system as a graph structure, construct a prior adjacency matrix based on the physical proximity and functional coupling relationships between sensors, and normalize the prior adjacency matrix to obtain a normalized adjacency matrix.

[0026] The graph structure G is represented as follows: G=(V, E, A); Where V is the set of sensor nodes, with each sensor corresponding to one sensor node; E is the set of edges, with each edge corresponding to the connection between two sensor nodes; A is the adjacency matrix, used to represent the connection relationships between each sensor node; the principles for establishing edges include: Local coupling connection: establish edges between physically adjacent sensing nodes or between nodes with functional / physiological coupling (e.g., adjacent fingers move together due to interphalangeal tendon synapses and shared flexor and extensor muscle groups). Global reference connection: Connects the reference node reflecting the overall attitude (such as the palm-back IMU) to each local node, encoding the coupling from global to local; Self-connection: Add a self-loop to each node to preserve its own response in graph aggregation.

[0027] The elements of the prior adjacency matrix are defined as follows: ; in, Indicates the position of the first element in the prior adjacency matrix. line, number Column elements; The formula for normalizing the prior adjacency matrix is: ; in, This is the normalized adjacency matrix; It is a degree matrix (the diagonal elements are the degrees of each node). The prior adjacency matrix before normalization. This is the identity matrix (introducing a self-loop). The resulting... Injecting inductive biases in a sparse and interpretable manner provides a low-error topological starting point. It is this approach of "modeling contact sensor systems as graphs and aggregating node collaborations through graph convolution" that enables this invention to explicitly characterize non-Euclidean topological collaboration information that CNNs / LSTMs cannot capture.

[0028] because A fixed topology, determined solely by the sensing topology and remaining constant across different users, introduces structural mismatch errors as the actual node collaboration relationships drift with users. Therefore, such as... Figure 2 As shown, this embodiment superimposes a learnable adaptive perturbation matrix on top of the prior adjacency matrix. : ; in, This is the adaptive adjacency matrix after superimposing the adaptive perturbation matrix; For the adaptive perturbation matrix, Shared globally among all users and samples, initialized as an all-zero matrix, with a parameter count of [number missing]. , This represents the total number of nodes in the graph. These are the model parameters, optimized end-to-end via backpropagation.

[0029] To constrain the perturbation amplitude and maintain training stability, a bounded activation function is used. Compress and then stack: ; in, It is the hyperbolic tangent function; right Substitution after normalization Used for subsequent graph convolution operations.

[0030] The ability to simultaneously improve cross-user generalization and enhance overall accuracy stems from a triple-layered set of constraints: (1) The parameter size is extremely small (only N×N parameters, far fewer than the backbone network), which inherently limits its capacity to memorize user-specific noise; (2) Zero initialization makes the optimization starting point a static prior with good generalization, and the training only makes small corrections in its neighborhood (the N×N parameters are updated in small steps along the gradient direction during training); it is equivalent to adding a small additive correction to each edge weight of the prior graph, rather than regenerating the entire graph. (3) Gradient cancellation brought about by global sharing, unique Since the gradient is shared among all users and samples, the gradient used for each update is the average of the gradients of all samples; connections that benefit only a few users while harming others will cancel each other out in the gradient direction, making stable accumulation impossible. Therefore... Convergence is achieved through stable, user-invariant cooperative relationships, rather than user-specific noise.

[0031] thus, This approach improves recognition accuracy across user scenarios and enhances overall recognition performance in general scenarios with minimal increase in model complexity. In this embodiment, the initial adjacency matrix and the adaptive adjacency matrix are as follows: Figure 3 As shown, Figure 3 In the diagram, (a) represents the initial adjacency matrix. (b) is the adaptive adjacency matrix; Figure 3 The annotations in both Chinese and English are as follows: Thumb: thumb; Index: index finger; Middle: middle finger; Ring: the ring finger; Little: the little finger; Palm IMU: An inertial measurement unit placed on the palm or back of the hand to collect overall hand motion attitude information (three-axis acceleration + three-axis angular velocity), which serves as a global reference node and establishes connections with each finger node.

[0032] Edge weight: The weight of an edge, representing the strength of the connection between two sensor nodes in a graph neural network. Figure 3 In the heatmap, the value of the edge weight is represented by the color intensity and numerical label. The higher the weight, the stronger the collaborative relationship between the two sensors corresponding to that edge.

[0033] These six nodes (five fingers + IMU on the back of the hand) together constitute the six graph nodes in the graph structure, and the edge weight corresponds to the connection weight value between nodes in the adaptive adjacency matrix.

[0034] Step S3: Using the normalized adjacency matrix as the topological basis for graph convolution, perform graph convolution operation on the spatiotemporal graph tensor, aggregate the collaborative features between each graph node and its adjacent graph nodes, and extract spatial structure features.

[0035] Graph convolution operations on spatiotemporal graph tensors include: Graph aggregation: At each time step, the graph node feature matrix is ​​multiplied by the normalized adjacency matrix, and the features of each graph node are updated to be the weighted sum of its own features and the features of its neighboring graph nodes, so as to realize the propagation of collaborative information between sensing nodes. Channel transformation: The aggregated features are then linearly transformed using a learnable weight matrix to change the feature channel dimension; Apply a nonlinear activation to obtain the output of the graph convolution of this layer; No. The operation of layer graph convolution is represented as: ; in, For the first The node features of the layer input, For the normalized adjacency matrix, For the first The learnable weight matrix of the layer, It is a non-linear activation function; For the first The node features output by the first layer are used as the first layer's node features. The method uses two stacked graph convolutional layers, with the number of node feature channels increasing from C to 16 to 32, extracting spatial collaborative features from low to high levels layer by layer. Graph aggregation is performed independently at each time step, thus preserving the complete temporal dimension.

[0036] This step utilizes graph convolution to aggregate collaborative information of neighboring nodes along the graph topology at each time step, enabling each node's feature update to incorporate its own joint response with its neighbors. This explicitly captures non-Euclidean discriminative cues that are difficult to extract using CNN local convolution kernels or LSTM channel-by-channel modeling, resulting in a more discriminative and stable spatial feature representation.

[0037] Step S4: Aggregate the spatial structure features along the graph node dimension and the time dimension to obtain a fixed-length gesture representation vector; specifically, global average pooling is used to aggregate the spatial structure features along the graph node dimension and the time dimension. ; in, For gesture representation vectors, The total number of nodes in the graph. For time steps, Spatial features output by graph convolution at nodes Time step eigenvectors.

[0038] As an alternative, temporal attention pooling (which learns weights for each time step and then sums them up) can be used instead of global average pooling to enhance the feature contribution of key time steps.

[0039] This step compresses the graph node dimension and time dimension into a fixed-length gesture representation vector through global average pooling (or temporal attention pooling), eliminating the dependence on the length of the input time series and the number of nodes. This allows the subsequent classification layer to make judgments based on global spatiotemporal statistical features, improving the robustness and generalization ability of the model.

[0040] As an optional implementation, on the spatial features of the graph convolution output, a one-dimensional convolution is first applied along the time axis to perform temporal modeling. The one-dimensional convolution only slides in the time dimension and does not change the graph node dimension. The one-dimensional convolution operation and the graph convolution operation are stacked alternately and combined with residual connections to form a spatiotemporal block. Then, the output of the spatiotemporal block is subjected to global average pooling along the graph node dimension and the time dimension to obtain a gesture representation vector of fixed length.

[0041] This optional approach uses alternating stacking of one-dimensional convolution and graph convolution, along with residual connections, to further capture the evolution of gestures over time on the basis of spatial collaborative modeling, thereby improving the model's ability to distinguish dynamic temporal changes. Meanwhile, the temporal modeling module is optional and can be flexibly enabled or omitted based on computing resources during actual deployment, balancing performance and efficiency.

[0042] Step S5: Input the gesture representation vector into the classification layer, output the score for each gesture category, and take the category with the highest score as the gesture recognition result; the specific method is as follows: Gesture representation vector Input a fully connected classification layer and output scores for each gesture category: ; in, This represents the original score vectors for each category output by the classification layer; This is the weight matrix of the classification layer. For the bias term of the classification layer; The cross-entropy loss function is used throughout the network training phase, and the average loss over all samples is taken as the total loss. ; in, The total number of samples in the batch. For the first One-hot encoding of each sample category label, For the model to the first The predicted value for the nth sample is... The probability of a class This represents the total loss value.

[0043] During the training phase, data augmentation strategies may also be applied, including one or more of time scaling, time pruning, Gaussian noise addition, and amplitude scaling. These data augmentation strategies are applied only during the training phase to further improve the model's generalization ability.

[0044] This step will The weights of the graph convolutional layer, the weights of the classification layer, and the bias are all incorporated into the end-to-end joint optimization, which enables topology adaptation and feature extraction to evolve synergistically under the same loss function, avoiding suboptimal solutions caused by staged training. Finally, the recognition result is output with the highest-scoring category, realizing a complete recognition process from the original sensor signal to the gesture category.

[0045] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the gesture recognition method.

[0046] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the gesture recognition method.

[0047] To verify the effectiveness of the above method on contact sensors, this invention also provides a hardware verification platform based on a data glove: integrating 5 flexible bending sensors (attached to the five fingers) and 1 six-axis IMU (placed on the back of the hand), which are synchronously collected by an ESP32 microcontroller at 50Hz and uploaded to a host computer via a wireless module. At this time, N=6 (nodes 0 to 4 are the five fingers, and node 5 is the palm IMU).

[0048] The platform collected eight common interactive gestures from multiple subjects (natural relaxation, waving left and right, double-tapping with index finger, swiping with palm, rotating with fist, OK gesture, pushing forward with palm, and pressing down with thumb) and verified them under the Leave One User (LOSO) protocol.

[0049] By introducing graph neural networks into contact sensors and explicitly aggregating the collaborative relationships between sensing nodes using graph convolution, the problem of insufficient modeling of non-Euclidean topological information by Euclidean models such as CNN / LSTM is overcome, resulting in more discriminative and stable feature representations; adaptive adjacency matrix While improving cross-user performance degradation (significantly improving accuracy for the worst-case scenario and reducing inter-user fluctuations), it also improves recognition accuracy in general cases (including the same user and the overall system); as shown in Table 1 below: Table 1 Performance comparison of different models and the present invention

[0050] In Table 1, Accuracy represents the comparison of recognition accuracy of each model under random sample partitioning, Macro-F1 is the arithmetic mean of F1 scores for all gesture categories, CAmin is the lowest recognition accuracy among testers under Leave-One-Stop (LOSO), CA is the average recognition accuracy of testers under LOSO, and CAstd is the standard deviation of recognition accuracy under LOSO. Leave-One-Stop (LOSO) involves removing all samples from a user in the dataset each time, using the remaining samples to train the model, and then using the removed user samples to test the model's generalization performance, ensuring that the samples tested are from unseen users.

[0051] Experimental results show that, compared with HMM, 1D-CNN, LSTM, and CNN-LSTM, this method (AST-GCN) significantly improves the accuracy of identifying unknown users and reduces inter-user fluctuations. The hardware platform and verification results described are only used to demonstrate the effectiveness of the method. The technical solution of this invention does not depend on the specific hardware mentioned above and can be applied to any contact-based wearable multi-sensor system.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic gesture recognition method based on wearable multi-sensor, characterized in that, Includes the following steps: Step S1: Acquire multi-channel time-series signals synchronously collected by the wearable multi-sensor system and preprocess them to obtain a spatiotemporal graph tensor of uniform specifications; Step S2: Treat each sensor as a graph node, model the sensor system as a graph structure, construct a prior adjacency matrix based on the physical proximity and functional coupling relationships between sensors, and normalize the prior adjacency matrix to obtain a normalized adjacency matrix. Step S3: Using the normalized adjacency matrix as the topological basis for graph convolution, perform graph convolution operation on the spatiotemporal graph tensor, aggregate the collaborative features between each graph node and its adjacent graph nodes, and extract spatial structure features. Step S4: Aggregate the spatial structure features along the graph node dimension and time dimension to obtain a fixed-length gesture representation vector; Step S5: Input the gesture representation vector into the classification layer, output the score of each gesture category, and take the category with the highest score as the gesture recognition result.

2. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 1, characterized in that, In step S1, the preprocessing includes: Frame number unification: Each sample is resampled along the time axis to a uniform length T using linear interpolation to obtain an aligned time reference; Normalization: Apply Z-score standardization to each channel to eliminate dimensional differences; Node feature alignment: When the feature dimensions of different graph nodes are inconsistent, all graph nodes are unified to the C dimension by copying and filling or linear mapping; After preprocessing, each sample is organized into an N×C×T spatiotemporal graph tensor, where N is the total number of graph nodes, C is the node feature dimension, and T is the unified time step.

3. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 1, characterized in that, In step S2, the graph structure G is represented as: G=(V, E, A); Where V is the set of sensor nodes, with each sensor corresponding to one sensor node; E is the set of edges, with each edge corresponding to the connection between two sensor nodes; and A is the adjacency matrix, used to represent the connection relationship between each sensor node. The elements of the prior adjacency matrix are defined as follows: ； in, Indicates the position of the first element in the prior adjacency matrix. line, number Column elements; The principles for establishing edges include: establishing local coupling connections between physically adjacent or functionally coupled graph nodes; establishing global reference connections between reference nodes reflecting the overall attitude and each local node; and establishing self-connections for each graph node. The formula for normalizing the prior adjacency matrix is: ； in, This is the normalized adjacency matrix; For degree matrix, The prior adjacency matrix before normalization. It is an identity matrix.

4. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 3, characterized in that, In step S2, a learnable adaptive perturbation matrix is ​​superimposed on the prior adjacency matrix: ； in, This is the adaptive adjacency matrix after superimposing the adaptive perturbation matrix; For the adaptive perturbation matrix, Shared globally among all users and samples, initialized as an all-zero matrix, with a parameter count of [number missing]. , This represents the total number of nodes in the graph. To constrain the perturbation amplitude, a bounded activation function is used. Compress and then stack: ； in, It is the hyperbolic tangent function; right Substitution after normalization Used for subsequent graph convolution operations.

5. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 1, characterized in that, Step S3, performing graph convolution on the spatiotemporal graph tensor, includes: At each time step, the graph node feature matrix is ​​multiplied by the normalized adjacency matrix to update the features of each graph node to a weighted sum of its own features and the features of its neighboring graph nodes, thereby realizing the propagation of collaborative information between sensing nodes. The aggregated features are then linearly transformed using a learnable weight matrix to change the feature channel dimension, and non-linear activation is applied to obtain the output of the graph convolution of this layer. No. The operation of layer graph convolution is represented as: ； in, For the first The node features of the layer input, For the normalized adjacency matrix, For the first The learnable weight matrix of the layer, It is a non-linear activation function; For the first The node features output by the first layer are used as the first layer's node features. The node features of the layer input.

6. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 1, characterized in that, In step S4, the spatial structure features are aggregated along the graph node dimension and the time dimension using global average pooling: ； in, For gesture representation vectors, The total number of nodes in the graph. For time steps, Spatial features output by graph convolution at nodes Time step eigenvectors.

7. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 6, characterized in that, On the spatial features of the graph convolution output, a one-dimensional convolution is first applied along the time axis to perform temporal modeling. The one-dimensional convolution only slides in the time dimension and does not change the graph node dimension. The one-dimensional convolution operation and the graph convolution operation are stacked alternately and combined with residual connections to form a spatiotemporal block. Then, the output of the spatiotemporal block is subjected to global average pooling along the graph node dimension and the time dimension to obtain a gesture representation vector of fixed length.

8. The dynamic gesture recognition method based on wearable multi-sensor as described in claim 1, characterized in that, In step S5, the gesture representation vector is... Input a fully connected classification layer and output scores for each gesture category: ； in, This is the original score vector for each category output by the classification layer; This is the weight matrix of the classification layer. For the bias term of the classification layer; The cross-entropy loss function is used throughout the network training phase, and the average loss over all samples is taken as the total loss. ； in, The total number of samples in the batch. For the first One-hot encoding of each sample category label, For the model to the first The predicted value for the nth sample is... The probability of a class This represents the total loss value.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the gesture recognition method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the gesture recognition method as described in any one of claims 1 to 8.

Images