Lightweight wireless gesture recognition method based on attention enhancement

Abstract

The invention discloses a lightweight wireless gesture recognition method based on attention enhancement, and the method comprises the steps: designing a lightweight network model (AELite) based on attention enhancement, and combining a bidirectional gated loop unit network (BiGRU), a residual convolution block (Res-Conv1D) and a channel attention module (CAM) to fully extract gesture features, the gesture action can be effectively recognized under the condition that the model parameters and the model size are controlled. According to the wireless gesture recognition method disclosed by the invention, the design of a lightweight model is emphasized while the system performance is improved, so that the requirements of a vehicle-mounted scene are met.

Description

Technical Field

[0001] The present invention relates to the field of gesture recognition, and in particular to a lightweight wireless gesture recognition method based on attention enhancement. Background Art

[0002] Gesture recognition technology enables control devices to recognize and respond to gestures as input commands, providing a natural and user-friendly way of human-computer interaction without physical contact or traditional input devices such as keyboards or touch screens. The intuitive interaction between the vehicle system and the driver based on gesture recognition can significantly reduce the driver's distraction and improve driving safety. At present, the research on vehicle gesture recognition systems usually uses sensors, cameras, radars and their integrated applications. These technologies rely on additional dedicated hardware devices and also need to consider changes in light in the car and privacy issues or are limited by fixed detection ranges and angles. The wireless gesture recognition system based on WiFi technology shows significant application potential due to its high-precision perception, low-cost deployment, and easy integration and expansion. It collects and analyzes channel state information (CSI) to achieve effective extraction and recognition of gesture features. The CSI information describes the channel characteristics between the transmitter and the receiver in the WiFi network, including changes in time delay, amplitude attenuation, and phase offset.

[0003] In intelligent connected cars, drivers can directly connect to the WiFi hotspot in the in-vehicle infotainment system (IVIS) through their smartphones to achieve wireless gesture recognition. Given the mobility and limited space of the vehicle, radio signals are more prone to fluctuations and are more sensitive to gesture changes, which requires higher accuracy in CSI recognition. On the other hand, most of the WiFi devices in the car are fixed in the center console of the IVIS, but the driver's smartphone is placed flexibly, which will lead to different variation characteristics of the CSI and bring additional robustness challenges to in-vehicle gesture recognition. In addition, due to the limited computing resources of smartphones, the complexity of the gesture recognition system will directly affect the running time of the device and consume the car's power during the interaction process. Therefore, it is crucial to consider lightweight model design for in-vehicle scenarios while improving system performance. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a lightweight wireless gesture recognition method based on attention enhancement.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A lightweight wireless gesture recognition method based on attention enhancement includes the following steps:

[0007] S1. After the collected CSI data of time and subcarrier containing different gesture information is selected, grouped and processed from the subcarrier dimension, the CSI data is Reduced to ;in, , is the subcarrier dimension, is the time dimension;

[0008] S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is ,Ensure the effective representation of gesture features;

[0009] S3, the bidirectional gated recurrent unit network extracts features from the input CSI data and captures the time series information of the gesture data; the number of hidden units of the bidirectional gated recurrent unit network is set to , the number of layers is , the output features The dimension becomes ,in ;

[0010] S4. Output features The gesture feature dimension is transposed to obtain the gesture feature , gesture features for , so that it meets the input requirements of the convolutional layer;

[0011] S5, three connected residual convolution blocks sequentially perform hand gesture features Perform local extraction and enhance gesture features spatial representation of;

[0012] S6, the output features of the last residual convolution block As the input of the channel attention module;

[0013] Key gesture features output by the channel attention module It is expressed as:

[0014] ;

[0015] in, represents the average pooling operation, represents the maximum pooling operation, is a multi-layer perceptron, is the logistic activation function;

[0016] S7. Key gesture features Classify and obtain the final classification of gesture actions.

[0017] The step S1 is specifically as follows:

[0018] S11. The dimension of CSI data containing different gesture information is , is the subcarrier dimension, is the time dimension; according to the corresponding CSI amplitudes on different subcarriers, subcarriers with CSI amplitudes greater than the first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set , the number of subcarriers included is , the subcarrier feature dimension of CSI data is given by Reduced to ;

[0019] S12, Collection The CSI amplitude is The subcarrier set that changes within constitutes a set , the remaining subcarrier sets constitute the set ;gather ;in, ,and , , For collection The number of subcarriers included, For collection The number of subcarriers included;

[0020] S13, according to the subcarrier grouping interval , Separately for the set ,gather Further grouping, the number of groups generated is ,in Indicates rounding up; ,in Indicates rounding down and the final subcarriers are merged into the previous group; ;

[0021] S14, yes Perform data processing on the group data to achieve dimensionality reduction and define , ,in Indicates in the collection The group index in ;

[0022] express In terms of dimension, The average value of CSI in the subcarrier groups:

[0023]

[0024] The obtained average CSI set is: ,So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is .

[0025] In step S3, at time The complete hidden state of the bidirectional gated recurrent unit network output is Expressed as:

[0026] ;

[0027] in, , represent the forward and backward output hidden states respectively, Represents a vector concatenation operation.

[0028] The step S5 is specifically as follows:

[0029] S51. First, the gesture features Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1;

[0030] S52, batch normalization layer normalizes the convolutional features;

[0031] S53, applying a ReLU activation function to the standardized output features;

[0032] S54. Finally, the output feature of the ReLU function is added to the input feature of the residual block to obtain the output of the residual connection.

[0033] The step S7 is specifically as follows:

[0034] S71. Key gesture features Flattened by the fully connected layer into an output vector ;

[0035] S72, The function will output the vector Transformed into the real classification probability distribution, specifically expressed as:

[0036] ;

[0037] in, Represents a specific category index, Indicates traversal of category indexes. is the total number of categories; each Output value represents the predicted probability of each gesture category.

[0038] At the same time, the present invention provides:

[0039] A server includes a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0040] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] The present invention uses BiGRU network to extract features from CSI data and capture the forward and backward temporal dependencies of gestures. It uses residual convolution blocks to extract local features, deepen gesture features from the spatial dimension, and accelerate the training process to alleviate the gradient explosion problem. It includes convolution operations, batch normalization, ReLU activation function and residual connection. It combines the channel attention mechanism to adaptively enhance gesture features to improve the robustness of the model, which is achieved through maximum pooling, average pooling, multi-layer perceptron and logical activation function. Finally, the softmax function is used to classify gestures and provide accurate classification probability distribution.

[0043] While improving system performance, the present invention focuses on the design of lightweight models to meet the needs of vehicle-mounted scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the architecture of the in-vehicle wireless gesture recognition system.

[0045] Figure 2 Schematic diagram of the structure of a lightweight network model based on attention enhancement.

[0046] Figure 3 A schematic diagram of a large-scale action.

[0047] Figure 4 This is a schematic diagram of a small-scale action.

[0048] Figure 5 This is the gesture recognition result diagram. DETAILED DESCRIPTION

[0049] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0050] A lightweight wireless gesture recognition method based on attention enhancement includes the following steps:

[0051] S1. After the collected CSI data of time and subcarrier containing different gesture information is selected, grouped and processed from the subcarrier dimension, the CSI data is Reduced to ;in, , is the subcarrier dimension, is the time dimension;

[0052] For example, the CSI data dimension is reduced from [256,50] to [13,50];

[0053] S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is ,Ensure the effective representation of gesture features;

[0054] That is, the output data dimension is [1,50,13];

[0055] S3, Bidirectional Gated Recurrent Unit Network (BiGRU) extracts features from the input CSI data to capture the time series information of gesture data; the number of hidden units of the Bidirectional Gated Recurrent Unit Network is set to , the number of layers is , the output features The dimension becomes ,in ;

[0056] That is, the number of hidden units is set to 32, the number of layers is 2, and the output features are The dimension becomes [1,50,64];

[0057] S4. Output features The gesture feature dimension is transposed to obtain the gesture feature , gesture features for , so that it meets the input requirements of the convolutional layer;

[0058] Gesture Features is [1,64,50];

[0059] S5, three connected residual convolution blocks sequentially perform hand gesture features Perform local extraction and enhance gesture features spatial representation of;

[0060] The residual convolution block (Res-Conv1D) consists of a convolution operation module, a batch normalization layer, a ReLU activation function and a residual module connected in sequence;

[0061] S6, the output features of the last residual convolution block As the input of the channel attention module, the channel compression ratio is set to 16;

[0062] Key gesture features output by the channel attention module (enhanced by the attention mechanism) is expressed as:

[0063] ;

[0064] in, represents the average pooling operation, represents the maximum pooling operation, is a multi-layer perceptron, is the logistic activation function;

[0065] S7. Key gesture features Classify and obtain the final classification of gesture actions.

[0066] The classification module processes features To complete the final classification of gesture actions.

[0067] The step S1 is specifically as follows:

[0068] S11. The dimension of CSI data containing different gesture information is , is the subcarrier dimension, is the time dimension; according to the corresponding CSI amplitudes on different subcarriers, subcarriers with CSI amplitudes greater than the first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set , the number of subcarriers included is , the subcarrier feature dimension of CSI data is given by Reduced to ;

[0069] S12, Collection The CSI amplitude is The subcarrier set that changes within constitutes a set , the remaining subcarrier sets constitute the set ;gather ;in, ,and , , For collection The number of subcarriers included, For collection The number of subcarriers included;

[0070] S13, according to the subcarrier grouping interval , Separately for the set ,gather Further grouping, the number of groups generated is ,in Indicates rounding up; ,in Indicates rounding down and the final subcarriers are merged into the previous group; ;

[0071] S14, yes Perform data processing on the group data to achieve dimensionality reduction and define , ,in Indicates in the collection The group index in ;

[0072] express In terms of dimension, The average value of CSI in the subcarrier groups:

[0073]

[0074] The obtained average CSI set is: ,So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is .

[0075] CSI data is provided by Reduced to , and are all expressed as specific values. The “dimensionality reduction” here means that, assuming that we have Group data, each group contains The actual amount of data is ; In the post-grouping processing, as shown in step S14, we average the data in each group, so the initial values ​​(for example: 1, 2, 3, 4, 5), after taking the average, it is only represented as 1 average value (for example: (1+2+…+5) / 5=3), and the amount of data is also reduced Descended to , that is, , that is, each group of data is finally transformed into its corresponding single average value.

[0076] In step S3, at time The complete hidden state of the bidirectional gated recurrent unit network output is Expressed as:

[0077] ;

[0078] in, , represent the forward and backward output hidden states respectively, Represents a vector concatenation operation.

[0079] At each time step , the hidden state of the BiGRU output The dimension is , which corresponds to the number of hidden units of the bidirectional gated recurrent unit network in step S3. The output feature The dimension is ,in ,illustrate Dimensions Is the final feature dimension The basis of the final feature dimension By connecting The hidden state of the layer is obtained .

[0080] The step S5 is specifically as follows:

[0081] S51. First, the gesture features Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1;

[0082] S52, batch normalization layer normalizes the convolutional features;

[0083] S53, applying a ReLU activation function to the standardized output features;

[0084] S54. Finally, the output feature of the ReLU function is added to the input feature of the residual block to obtain the output of the residual connection.

[0085] The step S7 is specifically as follows:

[0086] S71. Key gesture features Flattened by the fully connected layer into an output vector ;

[0087] S72, The function will output the vector Transformed into the real classification probability distribution, specifically expressed as:

[0088] ;

[0089] in, Represents a specific category index, Indicates traversal of category indexes. is the total number of categories; each Output value represents the predicted probability of each gesture category.

[0090] At the same time, the present invention provides:

[0091] A server includes a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0092] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned lightweight wireless gesture recognition method based on attention enhancement.

[0093] The present invention proposes a lightweight wireless gesture recognition method based on attention enhancement, which is applied to a vehicle-mounted wireless gesture recognition system (such as Figure 1 ), by designing a lightweight network model based on attention enhancement (AELite), combining the bidirectional gated recurrent unit network (BiGRU), residual convolution block (Res-Conv1D), and channel attention module (CAM) to fully extract gesture features, and effectively recognize gesture actions under the conditions that model parameters and model size are controlled.

[0094] The structure of the attention-enhanced lightweight network model (AELite) is as follows Figure 2 As shown in the figure, it is composed of a bidirectional gated recurrent unit network (corresponding to step S3), three residual convolution blocks (corresponding to steps S4 and S5), a channel attention module (corresponding to step S6), a fully connected layer (corresponding to step S7), and a Softmax function classification (corresponding to step S7). The remaining steps S1 and S2 can be understood as preprocessing steps before data input, which are input into the AELie model from step S3 to step S7.

[0095] The gesture recognition method proposed in the present invention can recognize different mobile phone placement postures and different action sizes (such as Figure 3 , 4 ), can effectively recognize 6 types of gestures (slide up and down, slide left and right, draw a Z shape, draw a circle, push forward, wave), and the recognition accuracy of each type of action is above 92% (such as Figure 5 ).

[0096] The BiGRU layer of the present invention can be replaced by other network structures, such as using a bidirectional long short-term memory network (BiLSTM) to capture bidirectional time dependencies, and using a Transformer network to capture global time dependencies.

[0097] The one-dimensional convolution operation in the residual convolution module can be performed by two-dimensional convolution, depth-wise separable convolution, grouped convolution, etc. to achieve similar local spatial feature extraction effects.

[0098] The batch normalization method in the residual convolution module can also use layer normalization and instance normalization to standardize the data.

[0099] In addition to the ReLU activation function in the residual convolution module, Leaky ReLU, ELU or Swish activation functions can be used to improve the ability of nonlinear feature extraction.

[0100] The channel attention enhancement module can be replaced by other attention mechanisms, such as combining or directly replacing the spatial attention mechanism, the multi-head attention mechanism, etc.

[0101] The output dimension of the fully connected layer of the classification module depends on the number of gesture types and can be modified according to the specific number of gesture types.

[0102] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A lightweight wireless gesture recognition method based on attention enhancement, characterized in that: The following steps are involved: S1. After the collected CSI data of time and subcarrier containing different gesture information is selected, grouped and processed from the subcarrier dimension, the CSI data is Reduced to ;in, , is the subcarrier dimension, is the time dimension; S2, the input layer processes the CSI data and flips the time dimension and subcarrier dimension of the gesture information so that the output data dimension is ,Ensure the effective representation of gesture features; S3, the bidirectional gated recurrent unit network extracts features from the input CSI data and captures the time series information of the gesture data; the number of hidden units of the bidirectional gated recurrent unit network is set to , the number of layers is , the output features The dimension becomes ,in ; S4. Output features The gesture feature dimension is transposed to obtain the gesture feature , gesture features for , so that it meets the input requirements of the convolutional layer; S5, three connected residual convolution blocks sequentially perform hand gesture features Perform local extraction and enhance gesture features spatial representation of; S6, the output features of the last residual convolution block As the input of the channel attention module; Key gesture features output by the channel attention module It is expressed as: ； in, represents the average pooling operation, represents the maximum pooling operation, is a multi-layer perceptron, is the logistic activation function; S7. Key gesture features Classify and obtain the final classification of gesture actions.

2. The lightweight wireless gesture recognition method based on attention enhancement according to claim 1 is characterized in that: The step S1 is specifically as follows: S11. The dimension of CSI data containing different gesture information is , is the subcarrier dimension, is the time dimension; according to the corresponding CSI amplitudes on different subcarriers, subcarriers with CSI amplitudes greater than the first preset value are selected from all subcarriers, and their corresponding CSI amplitudes constitute a set , the number of subcarriers included is , the subcarrier feature dimension of CSI data is given by Reduced to ; S12, Collection The CSI amplitude is The subcarrier set that changes within constitutes a set , the remaining subcarrier sets constitute the set ;gather ;in, ,and , , For collection The number of subcarriers included, For collection The number of subcarriers included; S13, according to the subcarrier grouping interval , Separately for the set ,gather Further grouping, the number of groups generated is ,in Indicates rounding up; ,in Indicates rounding down and the final subcarriers are merged into the previous group; ; S14, yes Perform data processing on the group data to achieve dimensionality reduction and define , ,in Indicates in the collection The group index in ; express In terms of dimension, The average value of CSI in the subcarrier groups: The obtained average CSI set is: ,So After being processed by the input layer of step S2, it is used as the CSI data input of the bidirectional gated recurrent unit network, and its data dimension is .

3. The lightweight wireless gesture recognition method based on attention enhancement according to claim 1 is characterized in that: In step S3, at time The complete hidden state of the bidirectional gated recurrent unit network output is Expressed as: ； in, , represent the forward and backward output hidden states respectively, Represents a vector concatenation operation.

4. The lightweight wireless gesture recognition method based on attention enhancement according to claim 1 is characterized in that: The step S5 is specifically as follows: S51. First, the gesture features Perform a one-dimensional convolution operation with a kernel size of 3, a stride of 1, and a padding of 1; S52, batch normalization layer normalizes the convolutional features; S53, applying a ReLU activation function to the standardized output features; S54. Finally, the output feature of the ReLU function is added to the input feature of the residual block to obtain the output of the residual connection.

5. The lightweight wireless gesture recognition method based on attention enhancement according to claim 1 is characterized in that: The step S7 is specifically as follows: S71. Key gesture features Flattened by the fully connected layer into an output vector ; S72, The function will output a vector Transformed into the real classification probability distribution, specifically expressed as: ； in, Represents a specific category index, Indicates traversal of category indexes. is the total number of categories; each Output value represents the predicted probability of each gesture category.

6. A server, comprising a processor and a memory, characterized in that: At least one program is stored in the memory, and the program is loaded and executed by the processor to implement the lightweight wireless gesture recognition method based on attention enhancement as described in any one of claims 1 to 5.

7. A computer-readable storage medium, wherein at least one program is stored in the storage medium, characterized in that: The program is loaded and executed by a processor to implement the lightweight wireless gesture recognition method based on attention enhancement as described in any one of claims 1 to 5.

Images