Gesture recognition method and device based on millimeter wave radar
Through the gesture recognition method based on millimeter wave radar, the distance-Doppler processing and image encoding technology are used, combined with convolutional neural network, the problems of light limitation and high computational complexity in the existing technology are solved, and high accuracy is achieved and all-weather gesture recognition with low resource consumption.
Patent Information
- Application Number
- CN202410964593.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-29
AI Technical Summary
Existing gesture recognition technology has problems such as limited lighting conditions, privacy leakage and high computing complexity, which is difficult to meet the needs of all-weather working and high accuracy without wearing devices, especially on embedded devices.
Using a gesture recognition method based on millimeter wave radar, the distance-Doppler processing is obtained by obtaining gesture action data, the gesture features are extracted and image encoding is performed, and the gesture features are converted by weighted averaging method and image encoding. In image form, it is adapted to convolutional neural network for recognition.
It realizes high-accuracy gesture recognition without retaining a large amount of data, reduces the consumption of storage and computing resources, and meets the needs of all-weather work and privacy protection.
Smart Images

Figure CN119045654A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to radar and gesture recognition, and in particular to a gesture recognition method and device based on millimeter wave radar. Background Art
[0002] Gesture recognition technology continues to develop and has become an important part of human-computer interaction, playing an important role in games, medical care, smart homes, and smart driving.
[0003] Early gesture recognition technology relied on wearable devices, such as sensors such as accelerometers and gyroscopes. However, this type of technology cannot meet people's daily needs for human-computer interaction without wearing additional devices, so it has not been widely used. With the development of computer technology, gesture recognition technology based on computer vision has developed. Users do not need to wear additional devices and can interact with the computer with their bare hands. However, visual devices are greatly limited by lighting conditions and have privacy leakage problems. Visual gesture recognition technology is based on deep learning. Although it has a high accuracy rate, it also has the problem of high computational complexity and is difficult to apply to embedded devices. Although infrared gesture devices solve the problem of privacy leakage, they are greatly affected by the environment and have high requirements for the working environment.
[0004] Millimeter-wave radar is not affected by light or weather conditions, and has the advantages of being able to work around the clock and protecting privacy. Compared with cameras or infrared, it has stronger penetration and can be installed inside the device. Millimeter-wave radar obtains multi-dimensional information such as the distance, speed, angle, and micro-Doppler of the target, and has an unparalleled advantage in recognizing gestures, especially dynamic gestures. Summary of the invention
[0005] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a gesture recognition method and device based on millimeter wave radar.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] Specifically, a gesture recognition method based on millimeter wave radar is proposed, including the following:
[0008] Acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, the size of which is the number of virtual channels*the number of chirps per frame*the number of sampling points per chirp;
[0009] For each frame of radar data, first perform range dimension FFT processing and then perform Doppler dimension FFT processing to obtain multi-channel range-Doppler data;
[0010] Preprocessing the multi-channel range-Doppler data to obtain processed data;
[0011] The processed data are modulo-added according to the channel dimension to obtain a distance-Doppler map, and the distance feature and the Doppler feature of the gesture feature data are obtained based on the distance-Doppler map;
[0012] Estimating the azimuth and pitch angles of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtaining azimuth features and pitch angle features of the gesture feature data based on the azimuth data, the pitch angle data and the range-Doppler map;
[0013] Accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data;
[0014] The time series of each feature in the gesture feature time series is converted into an image in a time series image coding manner to obtain a two-dimensional image of the distance, Doppler, azimuth and pitch angle features;
[0015] The two-dimensional images of distance, Doppler, azimuth and pitch angle features are combined into a 4-channel image, and the 4-channel image is used as the input of the trained convolutional neural network. The output of the trained convolutional neural network is the gesture recognition result.
[0016] Further, specifically, preprocessing the multi-channel range-Doppler data to obtain processed data includes:
[0017] According to the preset distance range of the user's hand, the multi-channel range-Doppler data is intercepted in the distance dimension to obtain processed data.
[0018] Further, specifically, the distance feature and Doppler feature of the gesture feature data are obtained based on the distance-Doppler map, including:
[0019] The range-Doppler map is denoted as X = [x ij ] Nrng×Ndop , when X = [x ij ] Nrng×Ndop In the example, the amplitude value is used as the weight to calculate the weighted average of distance and Doppler. The obtained data is the distance feature RngAvg and Doppler feature DopAvg of the gesture feature data. The calculation formula is as follows:
[0020]
[0021] Further, specifically, the azimuth angle feature and the pitch angle feature of the gesture feature data are obtained based on the azimuth angle data, the pitch angle data and the range-Doppler map, including:
[0022] The azimuth data is recorded as A = [a ij ] Nrng×Ndop , the pitch angle data is recorded as E = [e ij ] Nrng×Ndop , using the corresponding data in the range-Doppler map as the weight, calculate the weighted average of the azimuth and elevation angles, and use them as the azimuth feature AziAvg and the elevation feature EleAvg of the gesture feature data. The calculation formula is as follows:
[0023]
[0024] Further, based on the Gram's angle field, the time series of each feature in the gesture feature time series is converted into an image in the manner of time series image coding. Specifically,
[0025] Normalize the sequence to the maximum value, use the timestamp as the radius, and use the arc cosine of the normalized scaled value as the angle to generate polar coordinates;
[0026] The Gram matrix is calculated as follows:
[0027]
[0028] Among them, φ1~φ n is the angle of the sequence in polar coordinates, then the matrix G corresponds to a two-dimensional image.
[0029] Furthermore, specifically, the trained convolutional neural network performs three consecutive convolution operations on the 4-channel image, using a 3×3 convolution kernel and moving with a step size of 1, wherein a pooling MaxPool is added after the first convolution operation; subsequently, GlobalAvgPool is used to reduce the data to one dimension, and through two fully connected layer operations and SoftMax, the probabilities of 5 action categories are output.
[0030] The present invention also proposes a gesture recognition device based on millimeter wave radar, comprising the following:
[0031] A data acquisition module is used to acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, and the size is the number of virtual channels * the number of chirps per frame * the number of sampling points per chirp;
[0032] The data processing module is used to perform range-dimensional FFT processing and then Doppler-dimensional FFT processing on each frame of radar data to obtain multi-channel range-Doppler data;
[0033] A preprocessing module, used for preprocessing the multi-channel range-Doppler data to obtain processed data;
[0034] A first feature extraction module, configured to take the modulus of the processed data and add them according to the channel dimension to obtain a distance-Doppler map, and obtain a distance feature and a Doppler feature of the gesture feature data based on the distance-Doppler map;
[0035] A second feature extraction module is used to estimate the azimuth and pitch angle of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtain azimuth features and pitch angle features of the gesture feature data based on the azimuth data, pitch angle data and range-Doppler map;
[0036] A gesture feature time series calculation module is used to accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data;
[0037] An image conversion module is used to convert the time sequence of each feature in the gesture feature time sequence into an image in a time sequence image coding manner to obtain a two-dimensional image of distance, Doppler, azimuth and pitch angle features;
[0038] The gesture recognition module is used to combine the two-dimensional images of distance, Doppler, azimuth and pitch angle features into a 4-channel image, and use the 4-channel image as the input of the trained convolutional neural network. The output of the trained convolutional neural network is the gesture recognition result.
[0039] The beneficial effects of the present invention are:
[0040] The present invention proposes a gesture recognition method and device based on millimeter wave radar, which adopts weighted averaging method to fuse data features of gestures, adopts image coding method to convert the time series of gesture features into image form, and then uses convolutional neural network to classify gesture behavior data. The gesture recognition method proposed in the present invention does not need to retain a large amount of distance Doppler angle spectrum data or point cloud data as gesture feature data. The present invention extracts gesture data features through weighted averaging method, and only needs to retain single data for each feature dimension of each data frame. At the same time, image coding is applied to convert time series features into images, so that feature data is adapted to convolutional neural network, which not only ensures good classification accuracy, but also solves the problem of excessive consumption of storage and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:
[0042] Figure 1 Shown is a flow chart of a gesture recognition method based on millimeter wave radar according to the present invention;
[0043] Figure 2 The figure shows a timing sequence diagram of gesture features obtained when a gesture recognition method based on millimeter wave radar is applied in the present invention;
[0044] Figure 3 FIG. 1 is a schematic diagram of an image encoding process of a gesture recognition method based on millimeter wave radar according to the present invention;
[0045] Figure 4 The figure shows a schematic diagram of the structure of a convolutional neural network used in a preferred embodiment of a gesture recognition method based on millimeter-wave radar according to the present invention. DETAILED DESCRIPTION
[0046] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.
[0047] Example 1, reference Figure 1 The present invention proposes a gesture recognition method based on millimeter wave radar, comprising the following steps:
[0048] Acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, the size of which is the number of virtual channels*the number of chirps per frame*the number of sampling points per chirp;
[0049] For each frame of radar data, first perform range dimension FFT processing and then perform Doppler dimension FFT processing to obtain multi-channel range-Doppler data;
[0050] Preprocessing the multi-channel range-Doppler data to obtain processed data;
[0051] The processed data are modulo-added according to the channel dimension to obtain a distance-Doppler map, and the distance feature and the Doppler feature of the gesture feature data are obtained based on the distance-Doppler map;
[0052] Estimating the azimuth and pitch angles of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtaining azimuth features and pitch angle features of the gesture feature data based on the azimuth data, the pitch angle data and the range-Doppler map;
[0053] Accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data;
[0054] The time series of each feature in the gesture feature time series is converted into an image in a time series image coding manner to obtain a two-dimensional image of the distance, Doppler, azimuth and pitch angle features;
[0055] The two-dimensional images of distance, Doppler, azimuth and pitch angle features are combined into a 4-channel image, and the 4-channel image is used as the input of the trained convolutional neural network. The output of the trained convolutional neural network is the gesture recognition result.
[0056] In this embodiment 1, the weighted average method is used to fuse the data features of the gesture, the image coding method is used to convert the time series of the gesture features into an image form, and then the convolutional neural network is used to classify the gesture behavior data. The gesture recognition method proposed in the present invention does not need to retain a large amount of distance Doppler angle spectrum data or point cloud data as gesture feature data. The present invention extracts gesture data features through the weighted average method, and only a single data is required for each feature dimension of each data frame. At the same time, image coding is applied to convert the time series features into images, so that the feature data is adapted to the convolutional neural network, which not only ensures a good classification accuracy, but also solves the problem of excessive consumption of storage and computing resources.
[0057] In one example of the application, the gesture feature data of each frame is accumulated in the time domain, which means that the feature data of a single frame is expanded into feature data of multiple frames, and the feature data of multiple consecutive frames are arranged in time order to obtain the following: Figure 2 The gesture feature time series of multiple consecutive frames is shown;
[0058] As a preferred embodiment of the present invention, specifically, preprocessing the multi-channel range-Doppler data to obtain processed data includes:
[0059] According to the preset distance range of the user's hand, the multi-channel range-Doppler data is intercepted in the distance dimension to obtain processed data.
[0060] In this preferred embodiment, in order to ensure the accuracy of gesture data and avoid interference from irrelevant signals, the radar data is intercepted in the distance dimension according to a preset distance range of the user's hand, without retaining data for the entire distance.
[0061] As a preferred embodiment of the present invention, specifically, obtaining the distance feature and Doppler feature of the gesture feature data based on the distance-Doppler map includes:
[0062] The range-Doppler map is denoted as X = [x ij ] Nrng×Ndop , when X = [x ij ] Nrng×Ndop In the example, the amplitude value is used as the weight to calculate the weighted average of distance and Doppler. The obtained data is the distance feature RngAvg and Doppler feature DopAvg of the gesture feature data. The calculation formula is as follows:
[0063]
[0064] The range Doppler map is recorded as matrix X, where x ij is the element in the matrix, i and j are the element subscripts, representing the range coordinate and Doppler coordinate respectively, Nrng is the distance dimension length of the matrix, and Ndop is the Doppler dimension length of the matrix.
[0065] As a preferred embodiment of the present invention, specifically, obtaining the azimuth angle feature and the pitch angle feature of the gesture feature data based on the azimuth angle data, the pitch angle data and the range-Doppler map includes:
[0066] The azimuth data is recorded as A = [a ij ] Nrng×Ndop , the pitch angle data is recorded as E = [e ij ] Nrng×Ndop , using the corresponding data in the range-Doppler map as the weight, calculate the weighted average of the azimuth and elevation angles, and use them as the azimuth feature AziAvg and the elevation feature EleAvg of the gesture feature data. The calculation formula is as follows:
[0067]
[0068]
[0069] As a preferred embodiment of the present invention, Figure 3 As shown, based on the Gram angle field, the time series of each feature in the gesture feature time series is converted into an image in the manner of time series image coding. Specifically,
[0070] Normalize the sequence to the maximum value, use the timestamp as the radius, and use the arc cosine of the normalized scaled value as the angle to generate polar coordinates;
[0071] The Gram matrix is calculated as follows:
[0072]
[0073] Among them, φ1~φ n is the angle of the sequence in polar coordinates, then the matrix G corresponds to a two-dimensional image.
[0074] As a preferred embodiment of the present invention, specifically, the trained convolutional neural network performs three consecutive convolution operations on the 4-channel image, the convolution adopts a 3×3 convolution kernel, and moves with a step size of 1, wherein a pooling MaxPool is added after the first convolution operation; subsequently, GlobalAvgPool is used to reduce the data to one dimension, and through two fully connected layer operations and SoftMax, the probabilities of the five action categories are output. Figure 4 Taking the convolutional neural network shown as an example, it can realize 5 classifications of 4 specified gestures and other categories of actions.
[0075] Specifically, a complete simulation process includes:
[0076] (1) Collecting the radar raw ADC data of the set gesture;
[0077] (2) Performing FFT processing on the ADC data in the range dimension and Doppler dimension;
[0078] (3) intercepting the processed data within the range of 0.1m to 1m in the distance dimension;
[0079] (4) extracting four gesture features according to the feature extraction method;
[0080] (5) For each gesture feature, the gesture features of 16 consecutive frames are merged into a gesture feature sequence of length 16 ( Figure 2 );
[0081] (6) For each gesture feature sequence, the image encoding is performed to obtain a 16x16 two-dimensional feature image ( Figure 3 ), combine the four feature images into a 4-channel image;
[0082] (7) Input the 4-channel image into the trained convolutional neural network and output the gesture classification result ( Figure 4 )
[0083] The present invention also proposes a gesture recognition device based on millimeter wave radar, comprising the following:
[0084] A data acquisition module is used to acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, and the size is the number of virtual channels * the number of chirps per frame * the number of sampling points per chirp;
[0085] The data processing module is used to perform range-dimensional FFT processing and then Doppler-dimensional FFT processing on each frame of radar data to obtain multi-channel range-Doppler data;
[0086] A preprocessing module, used for preprocessing the multi-channel range-Doppler data to obtain processed data;
[0087] A first feature extraction module, configured to take the modulus of the processed data and add them according to the channel dimension to obtain a distance-Doppler map, and obtain a distance feature and a Doppler feature of the gesture feature data based on the distance-Doppler map;
[0088] A second feature extraction module is used to estimate the azimuth and pitch angle of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtain azimuth features and pitch angle features of the gesture feature data based on the azimuth data, pitch angle data and range-Doppler map;
[0089] A gesture feature time series calculation module is used to accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data;
[0090] An image conversion module is used to convert the time sequence of each feature in the gesture feature time sequence into an image in a time sequence image coding manner to obtain a two-dimensional image of distance, Doppler, azimuth and pitch angle features;
[0091] The gesture recognition module is used to combine the two-dimensional images of distance, Doppler, azimuth and pitch angle features into a 4-channel image, and use the 4-channel image as the input of the trained convolutional neural network. The output of the trained convolutional neural network is the gesture recognition result.
[0092] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.
[0093] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0094] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.
[0095] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.
Claims
1. A gesture recognition method based on millimeter wave radar, characterized in that: These include: Acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, the size of which is the number of virtual channels*the number of chirps per frame*the number of sampling points per chirp; For each frame of radar data, first perform range dimension FFT processing and then perform Doppler dimension FFT processing to obtain multi-channel range-Doppler data; Preprocessing the multi-channel range-Doppler data to obtain processed data; The processed data are modulo-added according to the channel dimension to obtain a distance-Doppler map, and the distance feature and the Doppler feature of the gesture feature data are obtained based on the distance-Doppler map; Estimating the azimuth and pitch angles of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtaining azimuth features and pitch angle features of the gesture feature data based on the azimuth data, the pitch angle data and the range-Doppler map; Accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data; The time series of each feature in the time series of gesture features is converted into an image in a time series image coding manner to obtain a two-dimensional image of distance, Doppler, azimuth and pitch angle features; The two-dimensional images of distance, Doppler, azimuth and pitch angle features are combined into a 4-channel image, and the 4-channel image is used as the input of the trained convolutional neural network, and the output of the trained convolutional neural network is the gesture recognition result.
2. The gesture recognition method based on millimeter wave radar according to claim 1, characterized in that: Specifically, preprocessing the multi-channel range-Doppler data to obtain processed data includes: According to the preset distance range of the user's hand, the multi-channel range-Doppler data is intercepted in the distance dimension to obtain processed data.
3. The gesture recognition method based on millimeter wave radar according to claim 1, characterized in that: Specifically, the distance feature and Doppler feature of the gesture feature data are obtained based on the distance-Doppler map, including: The range-Doppler map is denoted as X = [x ij ] Nrng×Ndop , when X = [x ij ] Nrng×Ndop In the example, the amplitude value is used as the weight to calculate the weighted average of distance and Doppler. The obtained data is the distance feature RngAvg and Doppler feature DopAvg of the gesture feature data. The calculation formula is as follows:
4. The gesture recognition method based on millimeter wave radar according to claim 3, characterized in that: Specifically, the azimuth angle feature and the pitch angle feature of the gesture feature data are obtained based on the azimuth angle data, the pitch angle data and the range-Doppler map, including: The azimuth data is recorded as A = [a ij ] Nrng×Ndop , the pitch angle data is recorded as E = [e ij ] Nrng×Ndop , using the corresponding data in the range-Doppler map as the weight, calculate the weighted average of the azimuth and elevation angles, and use them as the azimuth feature AziAvg and the elevation feature EleAvg of the gesture feature data. The calculation formula is as follows:
5. The gesture recognition method based on millimeter wave radar according to claim 1, characterized in that: Based on the Gram's angle field, the time series of each feature in the gesture feature time series is converted into an image in the way of time series image coding. Specifically, Normalize the sequence to the maximum value, use the timestamp as the radius, and use the arc cosine of the normalized scaled value as the angle to generate polar coordinates; The Gram matrix is calculated as follows: Among them, φ1~φ n is the angle of the sequence in polar coordinates, then the matrix G corresponds to a two-dimensional image.
6. The gesture recognition method based on millimeter wave radar according to claim 1, characterized in that: Specifically, the trained convolutional neural network performs three consecutive convolution operations on the 4-channel image, using a 3×3 convolution kernel and moving with a step size of 1, and adding a pooling MaxPool after the first convolution operation; subsequently, GlobalAvgPool is used to reduce the data to one dimension, and through two fully connected layer operations and SoftMax, the probabilities of 5 action categories are output.
7. A gesture recognition device based on millimeter wave radar, characterized in that: These include: A data acquisition module is used to acquire gesture action data based on millimeter wave radar, wherein the gesture action data includes multiple frames of radar data, each frame of radar data is represented as a data cube, and the size is the number of virtual channels * the number of chirps per frame * the number of sampling points per chirp; The data processing module is used to perform range-dimensional FFT processing and then Doppler-dimensional FFT processing on each frame of radar data to obtain multi-channel range-Doppler data; A preprocessing module, used for preprocessing the multi-channel range-Doppler data to obtain processed data; A first feature extraction module, configured to take the modulus of the processed data and add them according to the channel dimension to obtain a distance-Doppler map, and obtain a distance feature and a Doppler feature of the gesture feature data based on the distance-Doppler map; A second feature extraction module is used to estimate the azimuth and pitch angle of the processed data in the channel dimension to obtain azimuth data and pitch angle data, and obtain azimuth features and pitch angle features of the gesture feature data based on the azimuth data, pitch angle data and range-Doppler map; A gesture feature time series calculation module is used to accumulate the gesture feature data corresponding to each frame of radar data in the time domain to obtain a gesture feature time series sequence of multiple consecutive frames of radar data; An image conversion module is used to convert the time sequence of each feature in the gesture feature time sequence into an image in a time sequence image coding manner to obtain a two-dimensional image of distance, Doppler, azimuth and pitch angle features; The gesture recognition module is used to combine the two-dimensional images of distance, Doppler, azimuth and pitch angle features into a 4-channel image, and use the 4-channel image as the input of the trained convolutional neural network. The output of the trained convolutional neural network is the gesture recognition result.
