A lightweight wireless gesture recognition method based on attention enhancement

Abstract

This paper discloses a lightweight wireless gesture recognition method based on attention enhancement. By designing a lightweight network model based on attention enhancement (AELite), combined with a bidirectional gated recurrent unit (BiGRU) network, a residual convolutional block (Res-Conv1D), and a channel attention module (CAM), it fully extracts gesture features. This method effectively recognizes gestures while keeping model parameters and size under control. While improving system performance, this method emphasizes lightweight model design to meet the needs of in-vehicle scenarios.

Description

Technical Field

[0001] The present invention relates to the field of gesture recognition, and in particular to a lightweight wireless gesture recognition method based on attention enhancement. Background Art

[0002] Gesture recognition technology enables control devices to recognize and respond to gestures as input commands, providing a natural, user-friendly way for human-computer interaction without the need for physical contact or traditional input devices such as keyboards or touchscreens. The intuitive interaction between the vehicle-mounted system and the driver enabled by gesture recognition can significantly reduce driver distraction and improve driving safety. Current research on in-vehicle gesture recognition systems typically utilizes sensors, cameras, radar, and their integrated applications. These technologies rely on additional dedicated hardware and also need to consider changes in in-vehicle lighting and privacy issues or are limited by fixed detection ranges and angles. However, wireless gesture recognition systems based on WiFi technology show significant application potential due to their high-precision perception, low-cost deployment, and ease of integration and scalability. They effectively extract and recognize gesture features by collecting and analyzing channel state information (CSI). CSI describes the channel characteristics between the transmitter and receiver in a WiFi network, including variations in time delay, amplitude attenuation, and phase offset.

[0003] In intelligent connected cars, drivers can connect directly to the WiFi hotspot in the in-vehicle infotainment system (IVIS) via their smartphones to achieve wireless gesture recognition. Given the mobility and limited space of the vehicle, radio signals are more prone to fluctuations and are more sensitive to changes in gestures, thus placing higher demands on the accuracy of CSI recognition. On the other hand, most in-vehicle WiFi devices are fixed in the IVIS's center console, but the driver's smartphone can be placed flexibly, which will lead to different variations in the CSI characteristics and bring additional robustness challenges to in-vehicle gesture recognition. In addition, due to the limited computing resources of smartphones, the complexity of the gesture recognition system will directly affect the device's operating time and consume the vehicle's power during the interaction process. Therefore, considering lightweight model design while improving system performance is crucial for in-vehicle scenarios. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a lightweight wireless gesture recognition method based on attention enhancement.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A lightweight wireless gesture recognition method based on attention enhancement includes the following steps:

[0007] S1. After selecting, grouping, and processing the collected CSI data of time and subcarriers containing different gesture information from the subcarrier dimension, the CSI data is reduced from [M, T] to [G, T]; where M and G are subcarrier dimensions and T is the time dimension;

[0008] S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is [1, T, G], ensuring the effective representation of gesture features;

[0009] S3. The bidirectional gated recurrent unit network extracts features from the input CSI data to capture the time series information of the gesture data. The number of hidden units in the bidirectional gated recurrent unit network is set to Y, the number of layers is set to L, and the dimension of the output feature Y becomes [1, T, D], where D = U × L.

[0010] S4. Transpose the gesture feature dimension of the output feature Y to obtain the gesture feature Y T , gesture feature Y T is [1, D, T], so that it meets the input requirements of the convolutional layer;

[0011] S5, 3 connected residual convolution blocks sequentially perform hand gesture feature Y T Perform local extraction and enhance gesture feature Y T spatial representation of;

[0012] S6. Use the output feature R of the last residual convolution block as the input of the channel attention module;

[0013] Key gesture features A output by the channel attention module c Expressed as:

[0014] A c =σ(MLP(AvgPool(R))+MLP(MaxPool(R)));

[0015] Among them, AvgPool represents the average pooling operation, MaxPool represents the maximum pooling operation, MLP is the multi-layer perceptron, and σ is the logistic activation function;

[0016] S7. Key gesture feature A c Perform classification to obtain the final classification of gesture actions.

[0017] The step S1 is specifically as follows:

[0018] S11. The dimension of the CSI data containing different gesture information is [M, T], where M is the subcarrier dimension and T is the time dimension. According to the CSI amplitudes corresponding to different subcarriers, subcarriers with CSI amplitudes greater than a first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set The number of subcarriers included is N, and the subcarrier feature dimension of CSI data is reduced from M to N;

[0019] S12, Collection The set of subcarriers whose CSI amplitude changes within T constitutes the set The remaining subcarrier sets constitute the set gather in, And i∈{1,…,N}, N=N1+N2, N1 is the set The number of subcarriers included, N2 is the set The number of subcarriers included;

[0020] S13, according to the subcarrier grouping interval g1, g2 respectively gather Further grouping, the number of groups generated is in Indicates rounding up; in Indicates rounding down and merging the last N2-g2G2 subcarriers into the previous group; G=G1+G2;

[0021] S14. Process the data of group G to achieve dimensionality reduction, and define v1(k)=1+(k-1)g1, v2(k)=1+(k-1)g2, where k represents the number of The group index in ;

[0022] Indicates the average value of CSI in the kth subcarrier group in the T dimension:

[0023]

[0024] The obtained average CSI set is: So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is [G, T].

[0025] In step S3, at time t, the complete hidden state y output by the bidirectional gated recurrent unit network is t Expressed as:

[0026]

[0027] in, represent the forward and backward output hidden states respectively, Represents a vector concatenation operation.

[0028] The step S5 is specifically as follows:

[0029] S51, first gesture feature Y T Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1;

[0030] S52, batch normalization layer normalizes the convolutional features;

[0031] S53, applying the ReLU activation function to the standardized output features;

[0032] S54. Finally, the output features of the ReLU function are added to the input features of the residual block to obtain the output of the residual connection.

[0033] The step S7 is specifically as follows:

[0034] S71. Key gesture feature A c Flattened by the fully connected layer into the output vector Z;

[0035] S72. The softmax function converts the output vector Z into a true classification probability distribution, which is specifically expressed as:

[0036]

[0037] Among them, P represents a specific category index, p represents the traversal category index, and C is the total number of classification categories; each softmax output value S P represents the predicted probability of each gesture category.

[0038] At the same time, the present invention provides:

[0039] A server includes a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0040] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] This method uses a BiGRU network to extract features from CSI data, capturing the forward and backward temporal dependencies of gestures. It also uses residual convolution blocks for local feature extraction, deepening gesture features from a spatial dimension and accelerating training to mitigate the gradient explosion problem. The method incorporates convolution operations, batch normalization, ReLU activation functions, and residual connections. It also incorporates a channel-wise attention mechanism to adaptively enhance gesture features to improve model robustness, achieved through max pooling, average pooling, a multi-layer perceptron, and a logistic activation function. Finally, it uses a softmax function to classify gestures, providing an accurate classification probability distribution.

[0043] While improving system performance, the present invention focuses on the design of lightweight models to meet the needs of vehicle-mounted scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the architecture of the in-vehicle wireless gesture recognition system.

[0045] Figure 2 Schematic diagram of the structure of the lightweight network model based on attention enhancement.

[0046] Figure 3 Schematic diagram of large-scale action.

[0047] Figure 4 This is a diagram of a small-scale action.

[0048] Figure 5 This is the gesture recognition result diagram. DETAILED DESCRIPTION

[0049] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0050] A lightweight wireless gesture recognition method based on attention enhancement includes the following steps:

[0051] S1. After selecting, grouping, and processing the collected CSI data of time and subcarriers containing different gesture information from the subcarrier dimension, the CSI data is reduced from [M, T] to [G, T]; where M and G are subcarrier dimensions and T is the time dimension;

[0052] For example, the CSI data dimension is reduced from [256, 50] to [13, 50];

[0053] S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is [1, T, G], ensuring the effective representation of gesture features;

[0054] That is, the output data dimension is [1,50,13];

[0055] S3. The Bidirectional Gated Recurrent Unit (BiGRU) network extracts features from the input CSI data to capture the time series information of the gesture data. The number of hidden units and the number of layers of the Bidirectional Gated Recurrent Unit network are set to U, and the dimension of the output feature Y becomes [1, T, D], where D = U × L.

[0056] That is, the number of hidden units is set to 32, the number of layers is 2, and the output feature Y dimension becomes [1, 50, 64];

[0057] S4. Transpose the gesture feature dimension of the output feature Y to obtain the gesture feature Y T , gesture feature Y T is [1, D, T], so that it meets the input requirements of the convolutional layer;

[0058] That is, gesture feature Y T is [1,64,50];

[0059] S5, 3 connected residual convolution blocks sequentially perform hand gesture feature Y T Perform local extraction and enhance gesture feature Y T spatial representation of;

[0060] The residual convolution block (Res-Conv1D) consists of a convolution operation module, a batch normalization layer, a ReLU activation function and a residual module connected in sequence;

[0061] S6. Use the output feature R of the last residual convolution block as the input of the channel attention module, and set the channel compression ratio to 16;

[0062] Key gesture features A output by the channel attention module c (enhanced by the attention mechanism) is expressed as:

[0063] A c =σ(MLP(AvgPool(R))+MLP(MaxPool(R)));

[0064] Among them, AvgPool represents the average pooling operation, MaxPool represents the maximum pooling operation, MLP is the multi-layer perceptron, and σ is the logistic activation function;

[0065] S7. Key gesture feature A c Perform classification to obtain the final classification of gesture actions.

[0066] The classification module processes feature A c To complete the final classification of gesture actions.

[0067] The step S1 is specifically as follows:

[0068] S11. The dimension of the CSI data containing different gesture information is [M, T], where M is the subcarrier dimension and T is the time dimension. According to the CSI amplitudes corresponding to different subcarriers, subcarriers with CSI amplitudes greater than a first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set The number of subcarriers included is N, and the subcarrier feature dimension of CSI data is reduced from M to N;

[0069] S12, Collection The set of subcarriers whose CSI amplitude changes within T constitutes the set The remaining subcarrier sets constitute the set gather in, And i∈{1,…,N}, N=N1+N2, N1 is the set The number of subcarriers included, N2 is the set The number of subcarriers included;

[0070] S13, according to the subcarrier grouping interval g1, g2 respectively gather Further grouping, the number of groups generated is in Indicates rounding up; in Indicates rounding down and merging the last N2-g2G2 subcarriers into the previous group; G=G1+G2;

[0071] S14. Process the data of group G to achieve dimensionality reduction, and define v1(k)=1+(k-1)g1, v2(k)=1+(k-1)g2, where k represents the number of The group index in ;

[0072] Indicates the average value of CSI in the kth subcarrier group in the T dimension:

[0073]

[0074] The obtained average CSI set is: So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is [G, T].

[0075] The CSI data is reduced from [M, T] to [G, T], where M and G are both expressed as specific numerical values. The "dimensionality reduction" here means that, assuming that we have G groups of data after step S13, each group contains g specific numerical values, and the actual data volume is G×g; and in the post-grouping processing process, that is, as shown in step S14, we average the data in each group, then the initial g numerical values (for example: 1, 2, 3, 4, 5) are only expressed as 1 average value after averaging (for example: (1+2+…+5) / 5=3), and the data volume is also reduced from G×g to G×1, that is, G, that is, each group of data is finally converted into its corresponding single average value.

[0076] In step S3, at time t, the complete hidden state y output by the bidirectional gated recurrent unit network is t Expressed as:

[0077]

[0078] in, represent the forward and backward output hidden states respectively, Represents a vector concatenation operation.

[0079] At each time step t, the hidden state y output by BiGRU is t The dimension of is U, which corresponds to the number of hidden units of the bidirectional gated recurrent unit network in step S3. The output feature Y dimension is [1, T, D], where D = U × L, indicating that y t The dimension U is the basis for the final feature dimension D, which is obtained by concatenating the hidden states of L layers in series.

[0080] The step S5 is specifically as follows:

[0081] S51, first gesture feature Y T Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1;

[0082] S52, batch normalization layer normalizes the convolutional features;

[0083] S53, applying the ReLU activation function to the standardized output features;

[0084] S54. Finally, the output features of the ReLU function are added to the input features of the residual block to obtain the output of the residual connection.

[0085] The step S7 is specifically as follows:

[0086] S71. Key gesture feature A c Flattened by the fully connected layer into the output vector Z;

[0087] S72. The softmax function converts the output vector Z into a true classification probability distribution, which is specifically expressed as:

[0088]

[0089] Among them, P represents a specific category index, p represents the traversal category index, and C is the total number of classification categories; each softmax output value S P represents the predicted probability of each gesture category.

[0090] At the same time, the present invention provides:

[0091] A server includes a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0092] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0093] This paper proposes a lightweight wireless gesture recognition method based on attention enhancement, which is applied to vehicle-mounted wireless gesture recognition systems (such as Figure 1 ), by designing a lightweight network model based on attention enhancement (AELite), combining the bidirectional gated recurrent unit network (BiGRU), residual convolution block (Res-Conv1D), and channel attention module (CAM) to fully extract gesture features, it can effectively recognize gesture actions under the conditions that model parameters and model size are controlled.

[0094] The structure of the attention-enhanced lightweight network model (AELite) is as follows Figure 2 As shown in Figure 1, the network consists of a bidirectional gated recurrent unit network (corresponding to step S3), three residual convolutional blocks (corresponding to steps S4 and S5), a channel attention module (corresponding to step S6), a fully connected layer (corresponding to step S7), and a softmax function classification (corresponding to step S7). The remaining steps S1 and S2 can be understood as preprocessing steps before data input, which is input into the AELie model from steps S3 to S7.

[0095] The gesture recognition method proposed in the present invention is suitable for different mobile phone placement postures and different action sizes (such as Figure 3 、 4 ), can effectively identify 6 types of gestures (slide up and down, slide left and right, draw Z-shaped, draw a circle, push forward, wave), and the recognition accuracy of each type of action is above 92% (such as Figure 5 ).

[0096] The BiGRU layer of the present invention can be replaced by other network structures, such as using a bidirectional long short-term memory network (BiLSTM) to capture bidirectional time dependencies and using a Transformer network to capture global time dependencies.

[0097] The one-dimensional convolution operation in the residual convolution module can be replaced by two-dimensional convolution, depth-separable convolution, grouped convolution, etc. to achieve similar local spatial feature extraction effects.

[0098] The batch normalization method in the residual convolution module can also use layer normalization and instance normalization to standardize the data.

[0099] In addition to the ReLU activation function in the residual convolution module, Leaky ReLU, ELU, or Swish activation functions can be used to improve the ability of nonlinear feature extraction.

[0100] The channel attention enhancement module can be replaced by other attention mechanisms, such as combining or directly replacing the spatial attention mechanism, multi-head attention mechanism, etc.

[0101] The output dimension of the fully connected layer of the classification module depends on the number of gesture types and can be modified according to the specific number of gesture types.

[0102] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A lightweight wireless gesture recognition method based on attention enhancement, characterized in that: The following steps are involved: S1. After selecting, grouping, and processing the collected CSI data of time and subcarriers containing different gesture information from the subcarrier dimension, the CSI data is reduced from [M, T] to [G, T]; where M and G are subcarrier dimensions and T is the time dimension; The S1 is specifically as follows: S11. The dimension of the CSI data containing different gesture information is [M, T], where M is the subcarrier dimension and T is the time dimension. According to the CSI amplitudes corresponding to different subcarriers, subcarriers with CSI amplitudes greater than a first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set The number of subcarriers included is N, and the subcarrier feature dimension of CSI data is reduced from M to N; S12, Collection The set of subcarriers whose CSI amplitude changes within T constitutes the set The remaining subcarrier sets constitute the set gather in, And i∈{1,…,N}, N=N1+N2, N1 is the set The number of subcarriers included, N2 is the set The number of subcarriers included; S13, according to the subcarrier grouping interval g1, g2 respectively gather Further grouping, the number of groups generated is in Indicates rounding up; in Indicates rounding down and merging the last N2-g2G2 subcarriers into the previous group; G=G1+G2; S14. Process the G group of data to achieve dimensionality reduction, and define v1(k)=1+(k-1)g1, v2(k)=1+(k-1)g2, where k represents the group index in the set H; Indicates the average value of CSI in the kth subcarrier group in the T dimension: The obtained average CSI set is: So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is [G, T]; S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is [1, T, G], ensuring the effective representation of gesture features; S3. The bidirectional gated recurrent unit network extracts features from the input CSI data to capture the time series information of the gesture data. The number of hidden units and the number of layers of the bidirectional gated recurrent unit network are set to U, and the dimension of the output feature Y becomes [1, T, D], where D = U × L. At time t, the complete hidden state y output by the bidirectional gated recurrent unit network is t Expressed as: in, represent the forward and backward output hidden states respectively, Represents vector concatenation operation; The number of hidden units is set to 32 and the number of layers is 2; S4. Transpose the gesture feature dimension of the output feature Y to obtain the gesture feature Y T , gesture feature Y T is [1, D, T], so that it meets the input requirements of the convolutional layer; S5, 3 connected residual convolution blocks sequentially perform hand gesture feature Y T Perform local extraction and enhance gesture feature Y T The spatial representation is as follows: S51, first gesture feature Y T Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1; S52, batch normalization layer normalizes the convolutional features; S53, applying the ReLU activation function to the standardized output features; S54. Finally, the output feature of the ReLU function is added to the input feature of the residual block to obtain the output of the residual connection; S6. Use the output feature R of the last residual convolution block as the input of the channel attention module; Key gesture features A output by the channel attention module c Expressed as: A c =σ(MLP(AvgPool(R))+MLP(MaxPool(R))); Among them, AvgPool represents the average pooling operation, MaxPool represents the maximum pooling operation, MLP is the multi-layer perceptron, and σ is the logistic activation function; S7. Key gesture feature A c Perform classification to obtain the final classification of gesture actions.

2. The lightweight wireless gesture recognition method based on attention enhancement according to claim 1 is characterized in that: The S7 is specifically as follows: S71. Key gesture feature A c Flattened by the fully connected layer into the output vector Z; S72. The softmax function converts the output vector Z into a true classification probability distribution, which is specifically expressed as: Among them, P represents a specific category index, p represents the traversal category index, and C is the total number of classification categories; each softmax output value S P represents the predicted probability of each gesture category.

3. A server comprising a processor and a memory, characterized in that: At least one program is stored in the memory, and the program is loaded and executed by the processor to implement the lightweight wireless gesture recognition method based on attention enhancement as described in any one of claims 1 or 2.

4. A computer-readable storage medium storing at least one program, characterized in that: The program is loaded and executed by a processor to implement the lightweight wireless gesture recognition method based on attention enhancement as described in any one of claims 1 or 2.

Images