Radar gesture recognition method based on convolutional neural network

Abstract

This invention relates to a radar gesture recognition method based on a convolutional neural network (CNN). The method includes the following steps: acquiring radar gesture signals and preprocessing them to obtain a radar gesture dataset composed of images that can be input into the CNN; constructing a CNN, which includes an adaptive feature fusion convolutional module (AFFM), multiple adaptive mean pooling layers, multiple 3×3 convolutional layers, and an improved CBAM module; inputting the radar gesture feature map into the AFFM, then processing it through 3×3 convolutional layers and adaptive mean pooling before inputting it into the improved CBAM module. The input to the improved CBAM module is the output of the previous adaptive mean pooling layer. The output of the improved CBAM module is multiplied element-wise with the output of the previous adaptive mean pooling layer, and then processed through another adaptive mean pooling layer and a linear layer to obtain the radar gesture recognition result. This invention effectively increases recognition accuracy without increasing the number of parameters or runtime.

Description

Technical Field

[0001] This invention relates to the field of radar gesture recognition technology, and more specifically to a radar gesture recognition method based on a convolutional neural network. Background Technology

[0002] With the rapid development of smart wearable and mobile devices, Human-Computer Interaction (HCI) has become a hot topic over the past decade. While traditional contact-based physical devices offer high measurement accuracy, they suffer from poor convenience. Non-contact methods, on the other hand, provide users with greater freedom and convenience. Hand Gesture Recognition (HGR) stands out as one of the most intuitive and effective methods in non-contact HCI.

[0003] Currently, mainstream gesture recognition (HGR) methods can be divided into two types: optical HGR and radar-based HGR. Optical HGR methods include those using cameras and infrared sensors. The measurement accuracy of these methods is easily affected by external environmental factors such as lighting and temperature, leading to reduced accuracy. Furthermore, camera-based methods require capturing user images before recognizing gestures, which may result in user privacy breaches. Radar-based HGR methods, on the other hand, effectively protect user privacy and are unaffected by environmental conditions. They are also low-cost, low-power, and fast, making them suitable for gesture recognition in various environments.

[0004] Existing radar-based HGR methods also have drawbacks. Some researchers use 3D convolutional neural networks for gesture recognition, but this results in excessive parameters and long runtime. Others use LSTM or Transformer architectures, but these require multi-layer architectures to improve accuracy, leading to long runtimes. Hybrid models complicate the model, increasing runtime and the number of parameters. In short, current methods, if aiming for high recognition accuracy, suffer from excessive runtime and numerous model parameters, hindering practical applications of gesture recognition. Therefore, there is an urgent need for a radar gesture recognition method that can achieve high-precision recognition while maintaining lower runtime and fewer model parameters. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide a radar gesture recognition method based on convolutional neural networks. This recognition method incorporates an adaptive feature fusion convolutional module and an improved CBAM module, achieving a recognition accuracy of over 99%, while requiring less running time and fewer parameters.

[0006] The technical solution adopted by the present invention to solve the aforementioned technical problem is as follows:

[0007] A radar gesture recognition method based on a convolutional neural network, the method comprising the following:

[0008] The radar gesture signals are acquired and preprocessed to obtain a radar gesture dataset composed of images that can be input into a convolutional neural network.

[0009] Convolutional neural networks are constructed, including an adaptive feature fusion convolutional module (AFFM), multiple adaptive mean pooling layers and multiple 3×3 convolutional layers, and an improved CBAM module.

[0010] The radar gesture feature map is input into the adaptive feature fusion convolutional module AFFM, and then processed by a 3×3 convolutional layer and adaptive mean pooling before being input into the improved CBAM module. The input of the improved CBAM module is the output of the upper-level adaptive mean pooling. The output of the improved CBAM module is multiplied element-wise with the output of the upper-level adaptive mean pooling, and then processed by an adaptive mean pooling and linear layer to obtain the radar gesture recognition result.

[0011] The adaptive feature fusion convolutional module AFFM includes an instance normalization layer. After the input radar gesture feature map is processed by the instance normalization layer, it is divided into five branches for further processing. The first branch performs a 1×1 convolution operation and a ReLU function, the second branch performs a 3×3 convolution operation and a ReLU function, the third branch performs a 5×5 convolution operation and a ReLU function, the fourth branch performs a 1×1 convolution operation, a ReLU function, and max pooling, and the fifth branch performs a 1×1 convolution operation and a Sigmoid function. The output features of the first four branches are concatenated along the channel dimension and then multiplied element-wise with the output of the fifth branch. After processing by a batch normalization layer and an adaptive mean pooling layer, the final fused feature is output.

[0012] The improved CBAM module includes a spatial attention mechanism and an improved channel attention mechanism. The improved channel attention mechanism introduces a learnable parameter α, multiplies the output features of max pooling and average pooling in the channel attention mechanism by 1-α and α respectively, and then sums them. The sum is then integrated using a 1×1 grouped convolution with the number of groups equal to the number of input channels. Finally, the channel attention C is obtained by processing with the Sigmoid function.

[0013] The convolutional neural network is trained using a radar gesture dataset, and the trained convolutional neural network is used for gesture recognition.

[0014] Furthermore, the radar gesture dataset is obtained by image transformation of the publicly available gesture dataset in the IEEE data port. The publicly available gesture dataset in the IEEE data port contains 12 gestures consisting of a total of 4609 gesture records. These gestures range in length from 5 to 81 frames, and each frame contains information on up to 80 detection points. Each detection point contains the corresponding distance, velocity, x-coordinate, y-coordinate, and signal amplitude. The image transformation converts the data into a format of frame number * detection points = 80 * 80. For frames and detection points with fewer than 80, zeros are used to pad them.

[0015] Furthermore, during training, random initialization is used for parameter initialization, and the loss function is the cross-entropy loss function. In network training, mini-batch gradient descent and Adam optimization algorithms are used to adjust network parameters, with a batch size of 8.

[0016] Furthermore, the image that can be input into the convolutional neural network is high-dimensional channel image data with more than 3 channels.

[0017] Furthermore, the recognition method has a recognition accuracy of no less than 99.0%, and while ensuring high recognition accuracy, the time complexity is within 2.2s, and the number of model parameters (Params) is within 4.0M.

[0018] The present invention also protects a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the identification method.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] The convolutional neural network constructed by the method of this invention uses an Adaptive Feature Fusion Module (AFFM) in the input layer to extract and fuse features from different receptive fields, capture and extract common features of the same gestures at different speeds and positions, reduce the influence of background interference, and eliminate the influence caused by gesture differences. An improved CBAM module is used before the output layer to improve recognition accuracy while reducing the number of model parameters, which can efficiently and accurately recognize radar gesture signals.

[0021] The model in this invention is significantly superior to existing methods such as LSTM, Transformer, and hybrid models in terms of runtime. In terms of the number of parameters, it is significantly less than that of three-dimensional convolutional neural networks and hybrid models. This invention can effectively increase the recognition accuracy without increasing the number of parameters or the runtime.

[0022] The most prominent and essential feature of this invention is:

[0023] a) An innovative adaptive feature fusion convolution module AFFM is proposed, which comprehensively utilizes multi-scale fusion strategies and residual weights. By extracting and fusing features from different receptive fields, it captures and extracts common features of the same gestures at different speeds and positions, thereby improving the model's recognition accuracy.

[0024] (b) The network of this invention significantly improves the accuracy of gesture recognition, achieving over 99% accuracy across 12 gestures, and exhibits lower time and space complexity compared to most existing methods. A deep learning-based radar gesture recognition model is used to classify and recognize image-processed radar gesture data. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of a convolutional neural network according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the adaptive feature fusion convolutional module AFFM.

[0027] Figure 3 This is a schematic diagram of the structure of a traditional CBAM module.

[0028] Figure 4 This is a schematic diagram of the improved channel attention mechanism in this invention.

[0029] Figure 5 This is a diagram of 12 gestures in a publicly available dataset. Detailed Implementation

[0030] The present invention will be further explained below with reference to the embodiments and accompanying drawings, but this is not intended to limit the scope of protection of this application.

[0031] The task definition for radar gesture recognition in this invention is:

[0032] Radar gesture recognition is essentially a classification problem. After processing the acquired radar gesture signals using signal processing methods, the input X of the network is obtained. In this invention, the convolutional neural network can essentially be viewed as a function F(.).

[0033] Y = F(X) (1)

[0034] The input X can be classified by F to obtain the category Y of the gesture, thereby recognizing the gesture.

[0035] The overall structure of the convolutional neural network in this invention is as follows: Figure 1 As shown, the convolutional neural network includes an adaptive feature fusion convolutional module (AFFM), multiple adaptive mean pooling layers and multiple 3×3 convolutional layers, as well as an improved CBAM module.

[0036] The radar gesture feature map is input into the adaptive feature fusion convolutional module AFFM, and then processed by a 3×3 convolutional layer and adaptive mean pooling before being input into the improved CBAM module. The input of the improved CBAM module is the output of the upper-level adaptive mean pooling. The output of the improved CBAM module is multiplied element-wise with the output of the upper-level adaptive mean pooling, and then processed by an adaptive mean pooling and linear layer to obtain the radar gesture recognition result.

[0037] AFFM preprocesses the input feature map, extracting gesture information and reducing the feature map size while overcoming gesture differences and background interference. It utilizes an improved CBAM module to select the feature map before the fully connected layer, highlighting important information that is beneficial to classification while suppressing unimportant information, thus improving the accuracy of gesture recognition. It also has fewer parameters and runs faster.

[0038] Specifically, the structure of AFFM is as follows: Figure 2 As shown, the algorithm includes an instance normalization layer. After the input radar gesture feature map is processed by the instance normalization layer, it is divided into five branches for further processing. The first branch performs a 1×1 convolution operation and a ReLU function, the second branch performs a 3×3 convolution operation and a ReLU function, the third branch performs a 5×5 convolution operation and a ReLU function, the fourth branch performs a 1×1 convolution operation, a ReLU function, and max pooling, and the fifth branch performs a 1×1 convolution operation and a Sigmoid function. The output features of the first four branches are concatenated along the channel dimension and then multiplied element-wise with the output of the fifth branch. After processing by a batch normalization layer and an adaptive mean pooling layer, the final fused feature is output.

[0039] Because gesture performers have different habits, the location and speed of gestures vary. Even the same person making the same gesture in different situations may show significant differences in speed, location, etc. This difference manifests as data duplication, offset, or missing data in the input image X. The adaptive feature fusion convolution module AFFM can effectively reduce interference caused by gesture differences and external interference information.

[0040] Assuming the processed radar signal is X, an instance normalization layer is first used to transform X, pulling the distributed data back to the standard distribution and eliminating differences introduced by different gesture locations. Subsequently, multiple branches are used to extract features, each with different convolutional kernels (including 1×1, 3×3, and 5×5). These multiple branches are used to capture feature representations of the input data under different receptive fields. Figure 2 After feature extraction, the four branches on the left are concatenated along the channel dimension to obtain the concatenated feature X. f X fBy integrating complementary information extracted from different branches, the accuracy of HGR is further improved, and this multi-branch structure reduces the number of model parameters. Figure 2 The rightmost 1×1 convolution is used to transform the input data and is processed using the Sigmoid activation function to generate residual weights X. w Later with X f Perform element-wise multiplication to obtain the weighted fusion feature X. z The data is then transformed using a batch normalization layer to reduce the model's time complexity, and adaptive mean pooling is used to reduce the input size to half of its original size. This reduces the model's time complexity while retaining sufficient feature information, and outputs the final fused features.

[0041] The spatial channel attention mechanism CBAM includes channel attention mechanism and spatial attention mechanism, and its block diagram is as follows: Figure 3 As shown, the spatial channel attention mechanism consists of two stages: The input feature format is N*H*W, where N is the number of channels, and H and W are the height and width of the matrix, respectively. The channel attention mechanism compresses the input in the spatial dimension (compressing H*W to 1), that is, the input features are processed by parallel max pooling and average pooling to obtain two types of features, which are then fed into a shared fully connected layer to integrate the features and sum them. After that, the sigmoid activation function is used to process them to generate channel attention C. C is multiplied element-wise with the input and used as the input Z1 of the spatial attention mechanism. The spatial attention mechanism compresses the input Z1 in the channel dimension (compressing N to 1), processes the features into two one-channel feature maps by mean pooling and max pooling, concatenates them, uses a convolutional layer to extract features, and then uses the sigmoid activation function to process them to obtain spatial attention S. S is multiplied element-wise with Z1 to obtain the final output feature map Z.

[0042] This embodiment improves the channel attention mechanism by multiplying the output features of max pooling and average pooling by 1-α and α, respectively, and then summing them. The sum is then combined using a 1×1 grouped convolution with the number of groups equal to the number of input channels. Finally, the channel attention C is obtained by processing with the sigmoid function. For the specific structure of the improved channel attention mechanism, see [link to implementation details]. Figure 4 The learnable parameter α is automatically determined during training.

[0043] The improved channel attention mechanism calculation process is as follows: Max pooling and average pooling operations are performed on the input features to generate two feature maps Z. max With Z avg The two feature maps are weighted and summed using a learnable parameter α, then integrated using a 1x1 grouped convolution, and finally the channel attention C is generated by applying the Sigmoid activation function. The calculation process can be expressed as follows:

[0044] C = σ{conv1[α.Z] avg +(1-α).Z max ]} (2)

[0045] Where σ represents the Sigmoid operation and conv1 represents 1*1 grouped convolution.

[0046] Spatial attention mechanism operation such as Figure 3 As shown in (b), this is the existing structure. The specific processing procedure is as follows: mean pooling and max pooling are applied to the channel dimension to obtain two one-channel feature maps Z. a With Z m The two feature maps are concatenated along the channel dimension to obtain a two-channel feature map Z2. A two-dimensional convolution operation is then performed on Z2, followed by a Sigmoid activation function to obtain the final spatial attention S. The calculation formula is as follows:

[0047] SS=σ{conv2[Z avg Z max ]} (3)

[0048] Where conv2 represents a two-dimensional convolution operation, [] represents concatenating two feature maps along the channel dimension, and σ represents the Sigmoid operation.

[0049] By using an improved CBAM module to assign weights to feature maps, important information in the feature maps is highlighted and unimportant information is suppressed, thereby effectively improving the recognition accuracy of the model.

[0050] In this invention, radar gesture signals are preprocessed to obtain a radar gesture dataset composed of images that can be input into a convolutional neural network. The input feature X is a radar gesture feature map, which can be image data of any channel or high-dimensional channel image data (more than 3 channels). The input format is batch * number of channels * height * width.

[0051] Example 1

[0052] 1) Obtain multiple gesture records from the radar gesture dataset, divide all gesture records into training set and test set, and obtain multi-channel images after preprocessing each gesture record.

[0053] 2) Construct a convolutional neural network for gesture recognition. The network includes an adaptive feature fusion convolutional module, an improved CBAM module and three convolutional layers, three ReLU activation functions, three batch normalization layers, three adaptive mean pooling layers and linear layers.

[0054] 3) Input the training set into the neural network model constructed in step 2) for training.

[0055] 4) After preprocessing the radar gesture records to be identified, the trained neural network model is used to perform gesture recognition and obtain gesture classification results.

[0056] The preprocessing procedures in 1) and 4) are specifically as follows:

[0057] The radar's two-dimensional point cloud data is read into a high-dimensional matrix form. The data is then divided into subsets based on information such as range, velocity, x-coordinate, y-coordinate, and signal strength. Each subset is processed into a matrix format representing the number of frames multiplied by the number of sampling points. The matrices containing information from each subset are then concatenated to obtain multi-channel image data.

[0058] The specific model structure of the convolutional neural network in step 2) is as follows:

[0059] The adaptive feature fusion convolutional module is the first structure of the network, followed by the following layers in sequence: first convolutional layer, first adaptive average pooling layer, second convolutional layer, third convolutional layer, second adaptive average pooling layer, improved CBAM module, third adaptive average pooling layer, and linear layer. The output of the adaptive feature fusion convolutional module is connected to the input of the first convolutional layer. The output of the first convolutional module, after passing through a ReLU activation function and a batch normalization layer, is connected to the input of the first adaptive average pooling layer. The output of the first adaptive average pooling layer is connected to the input of the second convolutional layer. The output of the first convolutional layer is connected to the input of the third convolutional layer after passing through the ReLU activation function and batch normalization layer. The output of the third convolutional layer is then connected to the input of the second adaptive average pooling layer after passing through the ReLU activation function and batch normalization layer. The output of the second average pooling layer is connected to the input of the improved CBAM module. The output of the improved CBAM module and the output of the second average pooling layer are weighted and then connected to the input of the third adaptive average pooling layer. The output of the third average pooling layer is connected to the input of the linear layer. The output of the linear layer is processed by Softmax to obtain the probability distribution, thus obtaining the classification result.

[0060] The adaptive feature fusion convolutional module includes an instance normalization layer, three 1×1 convolutions, one 3×3 convolution, one 5×5 convolution, a max pooling layer, a batch normalization layer, and an adaptive mean pooling layer. The output of the instance normalization layer is connected to the input of all the aforementioned convolutions. The first 1×1 convolution is followed by a ReLU activation function and used as part of the concatenated features; the 3×3 convolution is followed by a ReLU activation function and used as part of the concatenated features; the 5×5 convolution is followed by a ReLU activation function and used as part of the concatenated features; the second 1×1 convolution is followed by a ReLU activation function, and its output is used as the input of the max pooling layer, whose output is also used as part of the concatenated features. The third 1×1 convolution is used as residual weights. A Sigmoid activation function is applied after the convolutional layers to assign weights to the concatenated features, resulting in a weighted fusion feature. The weighted fusion features are used as input to the batch normalization layer, which is then connected to the adaptive mean pooling layer. The result of the adaptive mean pooling layer is the result of the adaptive feature fusion convolution module.

[0061] In a convolutional neural network, the first convolutional layer consists of a 3×3 two-dimensional convolution, followed by a ReLU activation function and a batch normalization layer. The padding of the convolution is (1,1), the stride is (1,1), and the input and output channels are 128 and 256, respectively.

[0062] The second convolutional layer consists of a 3×3 two-dimensional convolution, followed by a ReLU activation function and a batch normalization layer. The convolution padding is (1,1), the stride is (1,1), and the input and output channels are 256 and 512, respectively.

[0063] The third convolutional layer consists of a 3×3 two-dimensional convolution, followed by a ReLU activation function and a batch normalization layer. The convolution is padded with (0,0) and has a stride of (1,1). The input and output channels are 512 and 512, respectively.

[0064] The first and second adaptive mean pooling layers both reduce the feature map size to half of its original size, while the third mean pooling layer reduces the feature map size to 1×1.

[0065] The improved CBAM module comprises one spatial max pooling layer, one channel-level max pooling layer, one spatial mean pooling layer, one channel-level mean pooling layer, one 1×1 convolutional layer, and one 1×1 grouped convolutional layer. The input feature map is processed by the spatial max pooling and spatial mean pooling layers to obtain two feature maps. These two feature maps are weighted and summed using a trainable parameter α to obtain a new feature map. This new feature map is used as the input to the 1×1 grouped convolutional layer. The output of the 1×1 grouped convolutional layer is transformed by a sigmoid function to obtain the channel attention C. The channel attention mechanism result is processed by the channel-level mean pooling and channel-level max pooling layers. The two resulting feature maps are concatenated to obtain a two-channel input feature map. This feature map is used as the input to the 1×1 convolutional layer, and the output is transformed by a sigmoid activation function to obtain the final spatial attention S.

[0066] Example 2

[0067] This embodiment uses a radar gesture recognition method based on convolutional neural networks to recognize and classify radar gestures processed into five-channel image data. The specific steps are as follows:

[0068] The dataset used is the publicly available gesture dataset from the IEEE data port. This dataset contains 4609 gesture records across 12 different gestures, such as... Figure 5 As shown, these gestures represent arm movement to the left, arm movement to the right, hand moving away, hand moving closer, arm raised, arm lowered, palm up, palm down, hand to the left, hand to the right, horizontal fist, and vertical fist. Each gesture corresponds to several data files stored in CSV format. These gestures range in length from 5 to 81 frames, with each frame containing point cloud data information for up to 80 detection points. Each detection point includes corresponding distance, velocity, x-coordinate, y-coordinate, and signal amplitude. The number of channels is 5, resulting in a size of 80*5*80. Image transformation is performed on the point cloud image data of the dataset. According to different information types, the data is organized into a format of frame number * detection points (80*80). Frames and detection points with fewer than 80 points are padded with zeros to obtain the radar gesture dataset.

[0069] The aforementioned convolutional neural network was trained using a radar gesture dataset, with random initialization used for parameter initialization during training. The cross-entropy loss function was used, and mini-batch gradient descent and Adam optimization algorithms were employed to adjust the network parameters, with a batch size of 8. The radar gesture dataset was randomly divided into training, validation, and test sets in a 6:2:2 ratio.

[0070] The training process was conducted entirely on a personal computer, with the specific configuration shown in Table 1.

[0071] Table 1 Experimental Environment

[0072] Parameter name Parameter value operating system Windows 10 GPU graphics card NVIDIA RTX 1050 Ti (2GB) CUDA version 11.4 PyTorch version 1.12 Programming environment Python 3.8

[0073] During training, three metrics were set: recognition accuracy (Acc), time complexity (Time), and model parameter count (Params). Acc was calculated as follows:

[0074]

[0075] N T N represents the number of correctly identified numbers, and N is the total number of numbers in the test set. The time calculation rule is to start reading the file and finish loading the model to process the file and obtain the recognition results. To avoid randomness, each model is run 1000 times and the average value is used as the time complexity when calculating the model's time complexity, in seconds (s). Params is the number of floating-point numbers required for storage, expressed in millions (M).

[0076] The internal processing flow of a convolutional neural network is as follows: Input X is fed into the constructed convolutional neural network. First, AFFM is used to perform weighted feature fusion on the input and reduce the size of the input features to obtain feature Y. A convolution operation with a kernel size of 3*3 is performed on Y, and adaptive mean pooling is used to obtain the intermediate feature map Y1.

[0077] Two convolution operations with a kernel size of 3*3 are performed on Y1, and adaptive mean pooling is used to obtain the intermediate feature map Y2. An improved CBAM module is applied to feature map Y2 and then multiplied with Y2 to obtain Y3, which highlights useful information in the feature map and suppresses useless information and background noise.

[0078] Adaptive mean pooling is used to downsample Y3, reducing the feature map size without altering the feature information within it. The downsampled Y3 is then flattened to obtain Y4, transforming the original batch * number of channels * height * width format into batch * (number of channels * height * width) format.

[0079] By using a linear layer, the weights and biases between different features of Y4 are calculated. The linear layer can output the probability of each category, thus obtaining the classification result Z.

[0080] To verify the effectiveness of the model proposed in this invention, the following comparative models are used in this embodiment:

[0081] a) DCS-CTN: Uses 3DCNN and Transformer to classify radar gestures.

[0082] b) DDFDRN: Utilizes dual-stream fusion of 2DCNN and 3DCNN to recognize radar gestures.

[0083] c) S3D: A model proposed for video recognition, applicable to frame data, and improved by incorporating attention blocks.

[0084] d) 3DCNN+LSTM: After initial feature extraction using 3DCNN, gesture recognition is performed using LSTM.

[0085] e)CMFF-HGR: Gesture recognition is achieved using a residual neural network that integrates multiple streams.

[0086] The experimental environment for training the comparison model is the same as that in Table 1. The results of the comparison experiment are shown in Table 2, which records the specific information of the three indicators set in the experiment. Recognition accuracy is expressed as a percentage, time complexity is in seconds, and the number of model parameters is in the millions. The comparison shows that, compared with other models, the model proposed in this invention has higher recognition accuracy, lower time complexity than most models, and fewer network parameters than most models. This model has high recognition accuracy and low time and space complexity, making it more suitable for practical applications in real-time radar gesture recognition systems.

[0087] Table 2. Experimental Results of Model Comparison

[0088] Network Model Acc(%) Time(s) Params(M) DCS-CTN 97.8 2.30 6.23 DFDRN 96.7 2.71 25.64 S3D 95.6 2.38 20.44 3DCNN+LSTM 95.5 2.49 5.63 CMFF-HGR 94.0 0.92 58.64 This invention 99.3 2.16 3.87

[0089] To verify the effectiveness of the proposed module, ablation experiments were conducted on the aforementioned radar gesture dataset, and the results are shown in Table 3. In the ablation experiments, to verify the effectiveness of the AFFM and improved CBAM modules, ablation experiments were performed on both modules separately. The experiments included four groups: with AFFM and improved CBAM module; with AFFM and non-improved CBAM module (i.e., conventional CBAM module); without AFFM and improved CBAM module; and without AFFM and without improved CBAM module. In the absence of AFFM, convolutional layers were used instead; in the absence of improved CBAM module, a spatial channel attention mechanism (CBAM module) was used. The experimental results show that both the proposed AFFM and improved CBAM modules effectively improved the recognition accuracy, and the improved CBAM module, compared to the original, reduced the time and number of parameters, making it more conducive to gesture recognition.

[0090] Table 3 Ablation Experiment Results

[0091]

[0092] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A radar gesture recognition method based on convolutional neural networks, characterized in that, The identification method includes the following: The radar gesture signals are acquired and preprocessed to obtain a radar gesture dataset composed of images that can be input into a convolutional neural network. Convolutional neural networks are constructed, including an adaptive feature fusion convolutional module (AFFM), multiple adaptive mean pooling layers and multiple 3×3 convolutional layers, and an improved CBAM module. The radar gesture feature map is input into the adaptive feature fusion convolutional module AFFM, and then processed by a 3×3 convolutional layer and adaptive mean pooling before being input into the improved CBAM module. The input of the improved CBAM module is the output of the upper-level adaptive mean pooling. The output of the improved CBAM module is multiplied element-wise with the output of the upper-level adaptive mean pooling, and then processed by an adaptive mean pooling and linear layer to obtain the radar gesture recognition result. The adaptive feature fusion convolutional module AFFM includes an instance normalization layer. After the input radar gesture feature map is processed by the instance normalization layer, it is divided into five branches for further processing. The first branch performs a 1×1 convolution operation and a ReLU function, the second branch performs a 3×3 convolution operation and a ReLU function, the third branch performs a 5×5 convolution operation and a ReLU function, the fourth branch performs a 1×1 convolution operation, a ReLU function, and max pooling, and the fifth branch performs a 1×1 convolution operation and a Sigmoid function. The output features of the first four branches are concatenated along the channel dimension and then multiplied element-wise with the output of the fifth branch. After processing by a batch normalization layer and an adaptive mean pooling layer, the final fused feature is output. The improved CBAM module includes a spatial attention mechanism and an improved channel attention mechanism. The improved channel attention mechanism introduces a learnable parameter α, multiplies the output features of max pooling and average pooling in the channel attention mechanism by 1-α and α respectively, and then sums them. The sum is then integrated using a 1×1 grouped convolution with the number of groups equal to the number of input channels. Finally, the channel attention C is obtained by processing with the Sigmoid function. The convolutional neural network is trained using a radar gesture dataset, and the trained convolutional neural network is used for gesture recognition.

2. The identification method according to claim 1, characterized in that, The radar gesture dataset is obtained by image transformation of the publicly available gesture dataset in the IEEE data port. The publicly available gesture dataset in the IEEE data port contains 12 gestures consisting of 4609 gesture records. These gestures range in length from 5 to 81 frames. Each frame contains information on up to 80 detection points. Each detection point contains the corresponding distance, velocity, x-coordinate, y-coordinate, and signal amplitude. The image transformation is to organize the data into the format of frame number * detection points = 80 * 80. For frames and detection points with fewer than 80, zeros are used to pad them.

3. The identification method according to claim 1, characterized in that, During training, random initialization is used for parameter initialization, and the loss function is the cross-entropy loss function. In network training, mini-batch gradient descent and Adam optimization algorithms are used to adjust network parameters, with a batch size of 8.

4. The identification method according to claim 1, characterized in that, The images that can be input into the convolutional neural network are high-dimensional channel image data with more than 3 channels.

5. The identification method according to claim 1, characterized in that, The recognition method has an accuracy of no less than 99.0%, and while ensuring high recognition accuracy, the time complexity is within 2.2s, and the number of floating-point numbers required for storing model parameters (Params) is within 4.0M.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can implement the steps of the identification method described in claims 1-5.

Images