Millimeter wave radar gesture recognition method and related equipment

Abstract

The invention discloses a millimeter wave radar gesture recognition method and related equipment. Preprocessed millimeter wave radar gesture signals are converted into a distance-time characteristic pattern, a speed-time characteristic pattern and an angle-time characteristic pattern; and performing feature enhancement on each feature map based on logarithm compression and a Softmax normalization method, inputting each feature map after feature enhancement into a pre-trained millimeter wave radar gesture recognition model, and outputting a gesture recognition result. The millimeter-wave radar gesture recognition model comprises three feature processing modules which are arranged in parallel and are used for processing feature maps after feature enhancement respectively, and each feature processing module comprises a feature extraction module, an Inception module and a multi-head attention mechanism module which are cascaded. The objective of the invention is to reduce the calculation complexity of the model while maintaining the gesture recognition precision of the millimeter wave radar, enhance the extraction and fusion capability of multi-modal features, and improve the adaptability and generalization capability in different data environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction gesture recognition, and particularly relates to a millimeter-wave radar gesture recognition method and related devices. Background Art

[0002] In the field of human-computer interaction technology, millimeter-wave radar gesture recognition technology has been applied in many fields due to its non-contact, high-precision, and strong environmental adaptability characteristics, such as smart home control, virtual reality interaction, automotive driving assistance systems, etc. Although the millimeter-wave radar gesture recognition technology based on deep learning has made progress in recent years, it still faces some problems in practical applications, which restricts its further development and popularization.

[0003] Currently, the mainstream millimeter-wave radar gesture recognition technologies mainly rely on deep learning algorithms, including convolutional neural networks (CNNs), recurrent neural networks (RNNs) and their variants (such as long short-term memory networks LSTMs, gated recurrent units GRUs), and Transformer-based models. These algorithms have achieved good results in extracting gesture features and achieving high-precision recognition, but they are also accompanied by high computational complexity. For example, the CNN-based method can effectively extract spatial features, but its computational complexity is high when dealing with large-scale data; although RNNs and their variants can capture temporal dynamic features, they also face high computational complexity when dealing with large-scale data. In addition, most existing millimeter-wave radar gesture recognition methods focus on the extraction of range-Doppler two-dimensional features, while ignoring the rich multi-modal information contained in millimeter-wave radar signals, such as range, velocity, angle, etc. This single feature extraction method limits the expressive power of the model, making it difficult to further improve the recognition performance in complex gesture or dynamically changing scenarios. At the same time, the complex model structure and high computational overhead also hinder its deployment in environments with limited computing resources, and the lack of time series modeling ability leads to poor recognition robustness across different scenarios and datasets. Summary of the Invention

[0004] Aiming at the problems existing in the prior art, the present invention provides a millimeter-wave radar gesture recognition method and related devices, aiming to reduce the computational complexity of the model, enhance the extraction and fusion ability of multi-modal features, and improve the adaptability and generalization ability in different data environments while maintaining the accuracy of millimeter-wave radar gesture recognition.

[0005] To solve the above technical problems, the present invention is realized through the following technical solutions: According to a first aspect of the present invention, there is provided a millimeter-wave radar gesture recognition method, including: Obtaining a preprocessed millimeter-wave radar gesture signal; Convert the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a velocity-time feature map, and an angle-time feature map; Based on logarithmic compression and Softmax normalization, perform feature enhancement on the distance-time feature map, the velocity-time feature map, and the angle-time feature map respectively to obtain the distance-time feature map, the velocity-time feature map, and the angle-time feature map after feature enhancement; Input the distance-time feature map, the velocity-time feature map, and the angle-time feature map after feature enhancement into a pre-trained millimeter-wave radar gesture recognition model, and output the gesture recognition result; wherein, the millimeter-wave radar gesture recognition model includes three feature processing modules arranged in parallel for processing the distance-time feature map, the velocity-time feature map, and the angle-time feature map after feature enhancement respectively. Each feature processing module includes a cascaded feature extraction module, an Inception module, and a multi-head attention mechanism module. After the three feature processing modules, there is a first feature splicing module and a spliced feature decoding output module cascaded together.

[0006] In a possible implementation manner of the first aspect, the obtaining of the preprocessed millimeter-wave radar gesture signal includes: Obtain the millimeter-wave radar gesture signal; Perform preprocessing on the millimeter-wave radar gesture signal to obtain the preprocessed millimeter-wave radar gesture signal.

[0007] In a possible implementation manner of the first aspect, the preprocessing of the millimeter-wave radar gesture signal includes: Perform static clutter suppression on the millimeter-wave radar gesture signal to obtain the millimeter-wave radar gesture signal after clutter suppression.

[0008] In a possible implementation manner of the first aspect, the conversion of the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a velocity-time feature map, and an angle-time feature map is specifically as follows: Perform distance Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the distance-time feature map; Perform Doppler Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the velocity-time feature map; Perform angle Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the angle-time feature map.

[0009] In a possible implementation manner of the first aspect, the feature enhancement of the distance-time feature map, the velocity-time feature map, and the angle-time feature map based on logarithmic compression and Softmax normalization is specifically as follows:

[0010] In the formula, represents the distance-time feature map, speed-time feature map or angle-time feature map after feature enhancement; represents the distance-time feature map, speed-time feature map or angle-time feature map; represents the feature index number in the distance dimension, speed dimension or angle dimension; represents the feature index number in the time dimension; represents the total number of features in the distance dimension, speed dimension or angle dimension; represents the total number of features in the time dimension.

[0011] In a possible implementation manner of the first aspect, the feature extraction module includes a cascaded first linear layer, first residual block, second residual block and third residual block, and the first residual block, second residual block and third residual block respectively introduce random inactivation layers with a set inactivation rate.

[0012] According to the second aspect of the present invention, a millimeter-wave radar gesture recognition device is provided, including: An acquisition module for acquiring the preprocessed millimeter-wave radar gesture signal; A conversion module for converting the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, speed-time feature map and angle-time feature map; An enhancement module for respectively performing feature enhancement on the distance-time feature map, speed-time feature map and angle-time feature map based on logarithmic compression and Softmax normalization method to obtain the distance-time feature map, speed-time feature map and angle-time feature map after feature enhancement; A recognition module for inputting the distance-time feature map, speed-time feature map and angle-time feature map after feature enhancement into a pre-trained millimeter-wave radar gesture recognition model and outputting a gesture recognition result; wherein, the millimeter-wave radar gesture recognition model includes three parallelly arranged feature processing modules respectively for processing the distance-time feature map, speed-time feature map and angle-time feature map after feature enhancement, and each feature processing module includes a cascaded feature extraction module, Inception module and multi-head attention mechanism module, and a first feature splicing module and a spliced feature decoding output module are jointly cascaded after the three feature processing modules.

[0013] According to the third aspect of the present invention, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the millimeter-wave radar gesture recognition method described above is implemented.

[0014] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the millimeter-wave radar gesture recognition method described above.

[0015] According to a fifth aspect of the present invention, there is provided a computer program product, which when executed by a processor implements the millimeter-wave radar gesture recognition method described above.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: The millimeter-wave radar gesture recognition method provided by the present invention converts millimeter-wave radar data into distance-time, velocity-time, and angle-time feature maps, and adopts a feature enhancement algorithm based on logarithmic compression and Softmax normalization, effectively improving the feature expression ability of radar signals. By enhancing multi-modal features of distance, velocity, and angle, and combining a three-branch structure (processing distance, velocity, and angle features respectively), comprehensive extraction and deep fusion of multi-modal data are achieved, enabling the millimeter-wave radar gesture recognition model to more comprehensively capture the spatio-temporal features of gestures, enhancing the recognition ability for different gesture patterns and effectively improving the recognition accuracy, and being able to maintain a high recognition accuracy in complex gesture or dynamic change scenarios. By introducing a lightweight Inception module, the number of parameters and computational complexity of the model are reduced. Compared with traditional deep learning models, the millimeter-wave radar gesture recognition model has improved computational efficiency while maintaining a high recognition accuracy, and is particularly suitable for environments with limited computing resources. By adopting a multi-head attention mechanism, the millimeter-wave radar gesture recognition model can effectively capture time dependencies and long-term dependencies, improving the recognition robustness in different datasets and environments, making the millimeter-wave radar gesture recognition method of the present invention perform better in cross-scene and cross-dataset applications, and enhancing the generalization ability of the model. The millimeter-wave radar gesture recognition method of the present invention combines a feature enhancement algorithm and a multi-branch structure, and can better capture the spatio-temporal features of gestures.

[0017] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the following will briefly introduce the drawings required for the description of the specific embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1It is a flowchart of a millimeter-wave radar gesture recognition method of the present invention; Figure 2 It is a structural diagram of FIMA-Net in a millimeter-wave radar gesture recognition method of the present invention; Figure 3 It is a structural diagram of a feature extraction module in a millimeter-wave radar gesture recognition method of the present invention; Figure 4 It is a structural diagram of an Inception module in a millimeter-wave radar gesture recognition method of the present invention; Figure 5 It is a structural diagram of an SE attention mechanism module in a millimeter-wave radar gesture recognition method of the present invention; Figure 6 It is a structural diagram of a decoder module in a millimeter-wave radar gesture recognition method of the present invention; Figure 7a It is the gesture classification performance of a millimeter-wave radar gesture recognition method of the present invention on a self-built dataset; Figure 7b It is the gesture classification performance of a millimeter-wave radar gesture recognition method of the present invention on an IEEE dataport peer dataset; Figure 8 It is the comparison before and after feature enhancement of the feature map in a millimeter-wave radar gesture recognition method of the present invention. Detailed implementation manner

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] The millimeter-wave radar gesture recognition method described in the present invention is applicable to various computing environments, including but not limited to the server side, embedded devices, mobile terminals, etc. At the hardware level, it is required to be equipped with at least a processor with a certain computing ability (such as the Intel Core series, ARM Cortex-A series, or higher-level processors), and a hardware interface capable of supporting millimeter-wave radar signal acquisition.

[0022] As Figure 1 shown, the embodiments of the present invention provide a millimeter-wave radar gesture recognition method, mainly to achieve efficient processing and recognition of millimeter-wave radar gesture data, reduce computational complexity, and improve the ability to extract and fuse multi-modal features. The specific steps are as follows: S1. Obtain the preprocessed millimeter-wave radar gesture signal.

[0023] It should be understood that to obtain the preprocessed millimeter-wave radar gesture signal, it is necessary to go through the steps of millimeter-wave radar gesture signal acquisition and millimeter-wave radar gesture signal preprocessing.

[0024] In one implementable manner, regarding obtaining the preprocessed millimeter-wave radar gesture signal, it is as follows: First, obtain the millimeter-wave radar gesture signal.

[0025] That is to say, the millimeter-wave radar acquisition device obtains the original radar data of the target area, which usually contains information in multiple dimensions, such as distance, speed, angle, etc.

[0026] Exemplarily, a millimeter-wave radar sensor supporting FMCW (Frequency Modulated Continuous Wave) is used for gesture signal acquisition.

[0027] For example, the FMCW millimeter-wave radar sensor is based on Texas Instruments' IWR1443 radar module, adopts a frequency-modulated continuous wave waveform, is configured in a multiple-input multiple-output (MIMO) mode, has two transmitters and four receivers, and the operating frequency band is 76 - 81 GHz. The FMCW radar signal sent by the millimeter-wave radar sensor can be expressed as:

[0028] Among them, represents the amplitude, is the carrier center frequency, is the bandwidth of the sampled FMCW radar signal, is the pulse width of a chirp signal (linear frequency modulation signal), is the fast time within the chirp signal, , is the initial phase.

[0029] At the receiving end, after the de-chirp signal and sampling operations, the received signal obtained by fast time sampling can be expressed as:

[0030] Among them, is the complex amplitude of the received signal, is the index number of the chirp signal, is the sample index in a single chirp signal, is the total number of samples in a chirp signal, is the target speed, is the radial distance from the target to the first receiving antenna, is the speed of light, is the distance between receiving channels, is the signal wavelength, is the angle between the radar line of sight and the normal direction of the target and the radar array, is the receiving channel index, is the number of transmitting channels, the number of receiving channels, is the sampling frequency in the fast time domain.

[0031] Next, preprocess the millimeter-wave radar gesture signal to obtain the preprocessed millimeter-wave radar gesture signal.

[0032] Preferably, static clutter suppression is performed on the millimeter-wave radar gesture signal to obtain the millimeter-wave radar gesture signal after clutter suppression. For M the frame sequence, the millimeter-wave radar gesture signal after clutter suppression is obtained by the following formula:

[0033] where represents the millimeter-wave radar gesture signal, is the millimeter-wave radar gesture signal after clutter suppression.

[0034] S2. Convert the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a speed-time feature map, and an angle-time feature map.

[0035] Specifically, the preprocessed millimeter-wave radar gesture signal is converted into the following three feature maps of different dimensions: Distance-time feature map: Combine the distance information collected by the millimeter-wave radar with the time series to generate a two-dimensional distance-time feature map, which represents the distance change of the target object at different time points. The distance-time feature map can reflect the change of the relative distance between the object and the radar over time.

[0036] Speed-time feature map: Combine the object speed measured by the radar with the time to generate a speed-time feature map. The speed-time feature map can reflect the motion speed of the target object and the dynamic process of speed change.

[0037] Angle-time feature map: Utilize the angle measurement data of the millimeter-wave radar and combine with the time information to generate an angle-time feature map. The angle-time feature map can reflect the angle change of the target object relative to the radar, especially suitable for capturing the dynamic angle change information when performing gesture recognition in different orientations.

[0038] In one implementable manner, the preprocessed millimeter-wave radar gesture signal is converted into a distance-time feature map, specifically: performing a distance Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the distance-time feature map.

[0039] In one implementable manner, the preprocessed millimeter-wave radar gesture signal is converted into a velocity-time feature map, specifically: performing a Doppler Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the velocity-time feature map.

[0040] In one implementable manner, the preprocessed millimeter-wave radar gesture signal is converted into an angle-time feature map, specifically: performing an angle Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the angle-time feature map.

[0041] S3. Feature enhancement is respectively performed on the distance-time feature map, the velocity-time feature map, and the angle-time feature map based on logarithmic compression and the Softmax normalization method to obtain the distance-time feature map, the velocity-time feature map, and the angle-time feature map after feature enhancement.

[0042] Specifically, the logarithmic compression and the Softmax normalization method are used to perform feature enhancement on the three feature maps. The distance-time feature map, the velocity-time feature map, and the angle-time feature map after feature enhancement can effectively highlight key information, while suppressing noise interference, improving the expression ability of the feature map, and being able to effectively characterize the spatio-temporal information of the gesture, providing higher-quality data input for feature processing and recognition.

[0043] In one implementable manner, regarding the feature enhancement of the distance-time feature map, the velocity-time feature map, and the angle-time feature map respectively based on logarithmic compression and the Softmax normalization method, specifically:

[0044] In the formula, represents the distance-time feature map, the velocity-time feature map, or the angle-time feature map after feature enhancement; represents the distance-time feature map, the velocity-time feature map, or the angle-time feature map; represents the feature index number in the distance dimension, the velocity dimension, or the angle dimension; represents the feature index number in the time dimension; represents the total number of features in the distance dimension, the velocity dimension, or the angle dimension; represents the total number of features in the time dimension.

[0045] Exemplarily, combined with Figure 8As shown, in the comparative experiment before and after feature enhancement, the enhanced features show more obvious patterns in the distance-time, speed-time, and angle-time dimensions, thus improving the recognition accuracy of the model.

[0046] S4. Input the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement into a pre-trained millimeter-wave radar gesture recognition model to output the gesture recognition result. Among them, the millimeter-wave radar gesture recognition model is obtained by training and iterating FIMA-Net using a sample set. It should be understood that in order to improve the robustness of the millimeter-wave radar gesture recognition model, multiple training data sets can be combined for joint training during the training stage, so that the millimeter-wave radar gesture recognition model can show stronger generalization ability in different scenarios and data environments.

[0047] Combine Figure 2 As shown, the millimeter-wave radar gesture recognition model includes three parallel feature processing modules respectively used to process the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement. Each feature processing module includes a cascaded feature extraction module, an Inception module, and a multi-head attention mechanism module. After the three feature processing modules, there is a first feature splicing module and a spliced feature decoding output module cascaded together.

[0048] That is to say, the millimeter-wave radar gesture recognition model includes three parallel feature processing modules. Each feature processing module is respectively used to process the distance-time feature map, speed-time feature map, and angle-time feature map. The internal structures of each feature processing module are the same and mainly include the following sub-modules: Feature extraction module: Responsible for extracting high-order features from the input original feature maps.

[0049] Inception module: Through multi-scale convolution operations, the Inception module can capture local features of different sizes and fuse them together. In this way, the millimeter-wave radar gesture recognition model can extract richer feature information at different scales, thereby improving the recognition ability in complex environments.

[0050] Multi-head attention mechanism module: By introducing the multi-head attention mechanism, the millimeter-wave radar gesture recognition model can pay more attention to key information and ignore irrelevant parts during the feature processing process. The multi-head attention mechanism can learn different attention weights in parallel in multiple dimensions, thereby enhancing the response ability to key regions or dynamic changes.

[0051] After each feature processing module finishes processing, the output features will be concatenated through a first feature concatenation module to fuse the information in the three feature maps. The concatenated features are decoded through the concatenated feature decoding output module, and finally the result of gesture recognition is obtained.

[0052] In one implementable manner, as Figure 3 shown, the feature extraction module includes a cascaded first linear layer, first residual block, second residual block, and third residual block. The first residual block, second residual block, and third residual block respectively introduce dropout layers with a set dropout rate.

[0053] Specifically, the main function of the feature extraction module is to extract useful feature information from each feature map. Figure 3 In it, B represents the size of the input feature map. The first linear layer is responsible for performing a preliminary linear transformation on each input feature map to reduce the scale variation between different modalities. Residual feature extraction gradually extracts features through three cascaded residual blocks. Each residual block contains two convolutional operations and a residual connection. The first residual block expands the channel dimension from 30 to 128, the second residual block remains unchanged, and the third residual block reduces the channel dimension from 128 to 64. Regularization is applied through dropout after each residual block. Exemplarily, the dropout rate is 0.2. The output feature organization adjusts the dimension of the final output feature map to [B, 64, 30], effectively capturing key features and ensuring the stable training of the deep network.

[0054] In one implementable manner, as Figure 4 shown, the Inception module includes a first multi-scale feature processing module, a second multi-scale feature processing module, a third multi-scale feature processing module, and a fourth multi-scale feature processing module arranged in parallel. After the first multi-scale feature processing module, the second multi-scale feature processing module, the third multi-scale feature processing module, and the fourth multi-scale feature processing module, there are jointly cascaded a second feature concatenation module and an SE attention mechanism module.

[0055] Specifically, the Inception module processes feature information of multiple different scales through parallel multi-scale feature processing modules. The Inception module includes four parallel multi-scale feature processing modules. Each multi-scale feature processing module processes receptive fields of different sizes, then merges information of different scales through concatenation, and further optimizes the weight allocation of features through the SE attention mechanism module. The first multi-scale feature processing module includes a cascaded first one-dimensional convolutional layer, a first batch normalization layer, and a first rectified linear unit layer. The low-dimensional features of the input data are processed through the first one-dimensional convolution operation, and the features are activated through the first batch normalization layer and the first rectified linear unit layer. The second multi-scale feature processing module includes a cascaded second one-dimensional convolutional layer, a third one-dimensional convolutional layer, a second batch normalization layer, and a second rectified linear unit layer. The second multi-scale feature processing module processes information of different scales through multiple convolutions to enhance the model's ability to recognize complex features. The third multi-scale feature processing module includes a cascaded fourth one-dimensional convolutional layer, a fifth one-dimensional convolutional layer, a third batch normalization layer, and a third rectified linear unit layer, further extracting features of different dimensions. The fourth multi-scale feature processing module includes a cascaded max pooling layer (Max Pooling), a sixth one-dimensional convolutional layer, a fourth batch normalization layer, and a fourth rectified linear unit layer. Key features are extracted through the max pooling operation, reducing the computational complexity while improving the robustness of the features. The features processed by the multi-scale feature processing modules are concatenated through the second feature concatenation module to merge the feature information of all scales and provide input for the SE attention mechanism module. Figure 4 In it, batch_size represents the batch sample size, and kernel represents the number of kernels.

[0056] That is to say, the Inception module adopts a parallel multi-branch structure. Each branch processes features at different scales and captures multi-scale temporal features through different convolutional kernel sizes. The Inception module can process feature information of multiple scales and improve the recognition ability of complex gestures. The features of all branches are concatenated along the channel dimension and then recalibrated through the SE attention mechanism to enhance the feature expression ability. Exemplarily, the Inception module uses 1×1 convolution to reduce the computational complexity.

[0057] In one implementable manner, as Figure 5 shown, the SE attention mechanism module includes a cascaded pooling module, a third feature concatenation module, a second linear layer, a fifth rectified linear unit layer, a third linear layer, a Sigmoid activation function layer, and an element-wise matrix multiplication layer; wherein, the pooling module includes a globally average pooling layer and a globally max pooling layer arranged in parallel, and the output of the second feature concatenation module serves as the input of the pooling module and the element-wise matrix multiplication layer.

[0058] Specifically, the purpose of the SE attention mechanism module is to adaptively adjust the weights of different features. Through global average pooling and global max pooling operations, the spatial information of the input feature map is compressed into a channel descriptor, thereby capturing global feature information. After the channel descriptors are concatenated, feature modeling is performed through two linear layers and a rectified linear unit to generate channel weights. Then, the Sigmoid activation function normalizes the weights between 0 and 1 to ensure the effectiveness of the weights. Finally, through element-wise multiplication operations, the generated weights are applied to the original input feature map to achieve feature recalibration, enhance the expression ability of key features, and suppress unimportant features. The multi-head attention mechanism captures temporal dependencies and long-term dependencies, calculates the correlations between features through self-attention, and improves the recognition accuracy of the millimeter-wave radar gesture recognition model. Figure 5 In Figure 5 , batch_sie represents the sample size of the input feature map, channels represents the number of channels, and B represents the size of the output map.

[0059] In one implementable manner, as Figure 2 shown, the concatenated feature decoding output module includes a cascaded flattening layer, a decoder module, and a fourth linear layer. Specifically, the function of the concatenated feature decoding output module is to decode and finally output the feature information obtained through multi-layer feature processing. Among them, the flattening layer flattens the multi-dimensional feature data into one dimension. As Figure 6 shown, the decoder module includes a cascaded fifth linear layer, a fifth batch normalization layer, a sixth rectified linear unit layer, and a sixth linear layer. The sixth rectified linear unit layer introduces a random inactivation layer with a set inactivation rate.

[0060] That is to say, after flattening, feature transformation and dimensionality reduction are performed through the decoder module. First, the concatenated features from the first feature concatenation module are input, with a dimension of 384×64 (that is, the feature dimensions output by each of the three branches are 128×64, and after concatenation, it is 384×64), and the concatenated features from the first feature concatenation module are processed. The concatenated feature vector contains rich information from different modalities. Before entering the decoder module, the concatenated features are first flattened to convert the multi-dimensional features into a one-dimensional vector suitable for linear transformation. In this way, the features are reorganized to better capture the complex patterns and relationships in the data. When processing the concatenated features from the first feature concatenation module, the decoder module first maps the high-dimensional features to a lower-dimensional space through linear transformation. Next, the features after linear transformation are normalized through batch normalization to accelerate the convergence of the model and improve stability. Subsequently, the rectified linear unit is used as the activation function to introduce non-linear expression ability for the features, so as to better capture the complex relationships in the data. To prevent overfitting, a dropout layer is introduced in the decoder module, randomly suppressing some features (the dropout rate is set to 0.2). Finally, the features are mapped to a 4-dimensional output space through another linear layer, corresponding to the prediction probabilities of four gesture categories. This architecture effectively reduces the number of parameters of the millimeter-wave radar gesture recognition model through batch normalization and regularization, and enhances the generalization ability, thus forming an efficient mapping between the high-dimensional feature space and the low-dimensional classification space. Figure 6 In it, B represents the size of the input feature map.

[0061] The millimeter-wave radar gesture recognition method proposed by the present invention effectively improves the recognition accuracy and reduces the computational complexity by extracting and fusing multi-modal features and combining feature enhancement techniques. Through the designed FIMA-Net, using the multi-head attention mechanism and the Inception module, rich spatio-temporal features can be extracted from complex radar data, and efficient and accurate gesture recognition can be achieved.

[0062] Combined with Figure 7a and Figure 7b As shown in, in one embodiment, through experimental verification, the millimeter-wave radar gesture recognition method of the present invention performs excellently in the gesture recognition task, specifically as follows: The millimeter-wave radar gesture recognition method of the present invention achieves a classification accuracy of approximately 99.8% on the self-built dataset and approximately 94.8% on the IEEE dataport public dataset. These results indicate that the millimeter-wave radar gesture recognition method of the present invention has good cross-dataset robustness while maintaining high recognition accuracy. As Figure 7a , on the self-built dataset, the confusion matrix of the model shows that most gesture categories can be accurately recognized, and the misclassification rate is extremely low. As Figure 7b, on the IEEE dataport dataset, although the dataset is sparse point cloud and the number of samples is small, the millimeter-wave radar gesture recognition method of the present invention still shows a high recognition accuracy. This indicates that the millimeter-wave radar gesture recognition method of the present invention not only performs well on the self-built dataset, but also can maintain stable performance on different data sources, and has broad application prospects.

[0063] The millimeter-wave radar gesture recognition method of the present invention can be widely applied to multiple technical and product fields. In the field of smart home, the millimeter-wave radar gesture recognition method of the present invention can achieve non-contact gesture control of devices through millimeter-wave radar. Users can easily adjust the light brightness, switch TV channels or adjust the air conditioner temperature through gestures, greatly improving the intuitiveness and convenience of human-computer interaction. In the field of medical rehabilitation, the high precision and robustness of the millimeter-wave radar gesture recognition method of the present invention enable it to assist medical devices in performing fine operations and rehabilitation training monitoring. For example, gesture control of the robotic arm for minimally invasive surgery or guiding patients to perform rehabilitation movements improves the precision and safety of medical operations. In the field of automotive safety, the millimeter-wave radar gesture recognition method of the present invention can be integrated into the intelligent driving system. The driver can switch the navigation mode, adjust the volume or control the in-vehicle entertainment system through simple gestures, thereby reducing the dependence on physical buttons and improving driving safety. In the field of consumer electronics, such as smartphones and tablets, the millimeter-wave radar gesture recognition method of the present invention can achieve more natural user interaction. For example, taking pictures in the air, scrolling pages or switching applications through gestures brings a new operation experience to users. In the field of industrial automation, the millimeter-wave radar gesture recognition method of the present invention can achieve non-contact control of industrial devices through gesture recognition technology, improving operation efficiency while reducing safety risks in the industrial environment. The lightweight design and high-efficiency recognition ability of the millimeter-wave radar gesture recognition method of the present invention ensure that it can achieve high-precision gesture recognition with low computational complexity on various resource-constrained devices, providing reliable technical support for the field of gesture recognition applications.

[0064] In one embodiment of the present invention, a millimeter-wave radar gesture recognition device is provided, which specifically includes an acquisition module, a conversion module, an enhancement module and a recognition module. Each module is configured as follows: The acquisition module is used to acquire the preprocessed millimeter-wave radar gesture signal.

[0065] The conversion module is used to convert the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a speed-time feature map and an angle-time feature map.

[0066] An enhancement module is used to perform feature enhancement on the distance-time feature map, speed-time feature map, and angle-time feature map respectively based on logarithmic compression and Softmax normalization method, so as to obtain the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement.

[0067] Specifically, performing feature enhancement on the distance-time feature map, speed-time feature map, and angle-time feature map respectively based on logarithmic compression and Softmax normalization method is specifically as follows:

[0068] wherein, represents the distance-time feature map, speed-time feature map, or angle-time feature map after feature enhancement; represents the distance-time feature map, speed-time feature map, or angle-time feature map; represents the feature index number of the distance dimension, speed dimension, or angle dimension; represents the feature index number of the time dimension; represents the total number of features of the distance dimension, speed dimension, or angle dimension; represents the total number of features of the time dimension.

[0069] A recognition module is used to input the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement into a pre-trained millimeter-wave radar gesture recognition model, and output a gesture recognition result; wherein, the millimeter-wave radar gesture recognition model includes three feature processing modules arranged in parallel for processing the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement respectively. Each feature processing module includes a cascaded feature extraction module, an Inception module, and a multi-head attention mechanism module. After the three feature processing modules, there are cascaded a first feature splicing module and a spliced feature decoding output module. Among them, the feature extraction module includes a cascaded first linear layer, a first residual block, a second residual block, and a third residual block, and the first residual block, the second residual block, and the third residual block respectively introduce a random inactivation layer with a set inactivation rate.

[0070] All relevant content of each step involved in the embodiments of the foregoing millimeter-wave radar gesture recognition method can be cited in the function descriptions of the corresponding functional modules of a millimeter-wave radar gesture recognition device in the embodiments of the present invention, and will not be repeated here. The division of modules in the embodiments of the present invention is illustrative, and is only a logical function division. In actual implementation, there may be other division methods. In addition, in various embodiments of the present invention, each functional module may be integrated in a processor, may also exist physically alone, or two or more modules may be integrated in one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0071] In one embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program. The computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function. The processor described in the embodiments of the present invention can be used for the operation of a millimeter-wave radar gesture recognition method.

[0072] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and this storage space stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for gesture recognition by a millimeter-wave radar in the above embodiments.

[0073] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0074] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0075] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the specified functions in Figure 1 one flow or multiple flows and / or blocksFigure 1 The functions specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 One process or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0077] The present invention also provides a computer program product, since the computer program product is used to execute any one of the above-mentioned millimeter-wave radar gesture recognition methods. Since the computer program product provided by the present invention and the above-mentioned millimeter-wave radar gesture recognition method belong to the same inventive concept, the computer program product provided by the present invention has all the advantages of the above-mentioned millimeter-wave radar gesture recognition method. Therefore, the beneficial effects of the computer program product provided by the present invention will not be described in detail herein.

[0078] In the present invention, the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0079] Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A millimeter-wave radar gesture recognition method, characterized in that, Including: Obtain the preprocessed millimeter-wave radar gesture signal; Convert the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a velocity-time feature map, and an angle-time feature map; Based on logarithmic compression and Softmax normalization, perform feature enhancement on the distance-time feature map, the velocity-time feature map, and the angle-time feature map respectively, to obtain the feature-enhanced distance-time feature map, velocity-time feature map, and angle-time feature map; Input the feature-enhanced distance-time feature map, velocity-time feature map, and angle-time feature map into a pre-trained millimeter-wave radar gesture recognition model, and output the gesture recognition result; wherein, the millimeter-wave radar gesture recognition model includes three parallel feature processing modules respectively used to process the feature-enhanced distance-time feature map, velocity-time feature map, and angle-time feature map, and each feature processing module includes a cascaded feature extraction module, an Inception module, and a multi-head attention mechanism module, and after the three feature processing modules, there is a first feature splicing module and a spliced feature decoding output module cascaded together.

2. The millimeter-wave radar gesture recognition method according to claim 1, characterized in that, The obtaining of the preprocessed millimeter-wave radar gesture signal includes: Obtain the millimeter-wave radar gesture signal; Perform preprocessing on the millimeter-wave radar gesture signal to obtain the preprocessed millimeter-wave radar gesture signal.

3. The millimeter-wave radar gesture recognition method according to claim 2, wherein The preprocessing of the millimeter-wave radar gesture signal includes: Perform static clutter suppression on the millimeter-wave radar gesture signal to obtain the millimeter-wave radar gesture signal after clutter suppression.

4. A millimeter-wave radar gesture recognition method according to claim 1, characterized in that The conversion of the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a velocity-time feature map, and an angle-time feature map is specifically: Perform distance Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the distance-time feature map; Perform Doppler Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the velocity-time feature map; Perform angle Fourier transform on the preprocessed millimeter-wave radar gesture signal to obtain the angle-time feature map.

5. The millimeter-wave radar gesture recognition method according to claim 1, characterized in that The feature enhancement of the distance-time feature map, the velocity-time feature map, and the angle-time feature map based on logarithmic compression and Softmax normalization is specifically: In the formula, represents the distance-time feature map, speed-time feature map, or angle-time feature map after feature enhancement; represents the distance-time feature map, speed-time feature map, or angle-time feature map; represents the feature index number in the distance dimension, speed dimension, or angle dimension; represents the feature index number in the time dimension; represents the total number of features in the distance dimension, speed dimension, or angle dimension; represents the total number of features in the time dimension.

6. A millimeter-wave radar gesture recognition method according to claim 1, characterized in that The feature extraction module includes a cascaded first linear layer, a first residual block, a second residual block, and a third residual block, and the first residual block, the second residual block, and the third residual block respectively introduce random inactivation layers with a set inactivation rate.

7. A millimeter-wave radar gesture recognition device, characterized in that, Including: An obtaining module, used to obtain the preprocessed millimeter-wave radar gesture signal; A conversion module, used to convert the preprocessed millimeter-wave radar gesture signal into a distance-time feature map, a velocity-time feature map, and an angle-time feature map; An enhancement module, used to perform feature enhancement on the distance-time feature map, the velocity-time feature map, and the angle-time feature map respectively based on logarithmic compression and Softmax normalization, to obtain the feature-enhanced distance-time feature map, velocity-time feature map, and angle-time feature map; An identification module, configured to input the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement into a pre-trained millimeter-wave radar gesture recognition model, and output a gesture recognition result; wherein, the millimeter-wave radar gesture recognition model includes three feature processing modules arranged in parallel for processing the distance-time feature map, speed-time feature map, and angle-time feature map after feature enhancement respectively. Each feature processing module includes a cascaded feature extraction module, an Inception module, and a multi-head attention mechanism module. After the three feature processing modules, a first feature splicing module and a spliced feature decoding output module are cascaded together.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a millimeter-wave radar gesture recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements a millimeter-wave radar gesture recognition method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, When the computer program product is executed by a processor, it implements a millimeter-wave radar gesture recognition method according to any one of claims 1 to 6.

Images