Gesture recognition method and system based on millimeter wave radar

Abstract

The invention discloses a gesture recognition method and system based on a millimeter wave radar, and the method comprises the steps: collecting a gesture echo signal based on the millimeter wave radar, and carrying out the static clutter suppression of the collected gesture echo signal; respectively generating a corresponding time-frequency domain image, a time-distance domain image and a time-angle domain image from the gesture echo signal subjected to static clutter suppression processing through short-time Fourier transform, fast Fourier transform and an MVDR beam forming method in sequence; and performing gesture recognition model training on the obtained image of the time-frequency domain, the image of the time-distance domain and the image of the time-angle domain to obtain a gesture recognition result. According to the gesture recognition method provided by the invention, the accuracy of gesture recognition can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of radar target recognition technology, and specifically relates to a gesture recognition method and system based on millimeter wave radar Background Art

[0002] Gesture recognition technology is a popular technology in the field of human-computer interaction, and is widely used in smart homes, smart driving, virtual reality, etc. In the medical field, doctors can control medical equipment through gestures, thereby achieving contactless medical operations; in the automotive field, drivers and passengers can issue instructions to the car center console through gestures; in the smart home field, people can use gestures to control common appliances such as air conditioners and televisions; in the AR / VR field, players can use gestures to control objects in the game to enhance the sense of immersion. Due to its advantages of non-contact, strong perception of micro-movement targets, ability to work all day and all weather, unaffected by light, and no privacy leakage, the gesture recognition method based on millimeter-wave radar has become an important human-computer interaction method.

[0003] In the prior art, features are generally extracted from a single image. In the case of different users and different orientations, gesture features are easily confused, and the correct recognition rate still needs to be improved.

[0004] Therefore, how to provide a gesture recognition method and improve the accuracy of gesture recognition is a technical problem that needs to be solved at present. Summary of the invention

[0005] The purpose of the present invention is to provide a gesture recognition method and system based on millimeter wave radar to solve the problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a gesture recognition method based on millimeter wave radar, comprising:

[0008] Collect gesture echo signals based on millimeter-wave radar, and perform static clutter suppression on the collected gesture echo signals;

[0009] The gesture echo signal after static clutter suppression is processed by short-time Fourier transform, fast Fourier transform and MVDR beamforming method to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively;

[0010] The obtained time-frequency domain image, time-distance domain image and time-angle domain image are used for gesture recognition model training to obtain gesture recognition results.

[0011] Optionally, static clutter suppression processing is performed on the collected gesture echo signal, including:

[0012] The phases of multiple gesture echo signals are averaged by a mean cancellation algorithm to obtain a mean pulse, which is used to perform a cancellation operation to suppress static clutter.

[0013] Optionally, the gesture echo signal after static clutter suppression is processed by fast Fourier transform to generate a corresponding time-distance domain image, including:

[0014] According to the millimeter wave radar ranging principle, the radial distance of the target gesture from the millimeter wave radar is obtained by using the time delay between the transmission signal and the gesture echo signal;

[0015] Based on the radial distance of the target gesture from the millimeter wave radar, the gesture echo signal is processed by fast Fourier transform to obtain a distance spectrum of the target gesture, and based on the distance spectrum of the target gesture, an image in the time-distance domain is obtained.

[0016] Optionally, the gesture echo signal after static clutter suppression is processed and then generated into an image in the time-angle domain by using an MVDR beamforming method, including:

[0017] Acquire a gesture echo signal received by a linear antenna array;

[0018] The gesture echo signal is processed by using the MVDR beamforming algorithm to obtain the angle information of the target gesture, and a time-angle domain image is obtained according to the angle information of the target gesture.

[0019] Optionally, before performing gesture recognition model training on the obtained time-frequency domain image, time-distance domain image, and time-angle domain image to obtain a gesture recognition result, the method further includes:

[0020] The image in the time-frequency domain, the image in the time-distance domain, and the image in the time-angle domain are respectively normalized to adjust the pixel value of each image to a preset range.

[0021] Optionally, the obtained time-frequency domain image, time-distance domain image, and time-angle domain image are subjected to gesture recognition model training to obtain a gesture recognition result, including:

[0022] The normalized time-frequency domain image, time-distance domain image, and time-angle domain image are input into the corresponding three two-dimensional convolutional neural networks in parallel to obtain the convolution features of each image;

[0023] splicing each image output by the two-dimensional convolutional neural network to form a spliced ​​image;

[0024] Based on the stitched image, the convolution features of each image are fused to obtain a gesture feature vector of the stitched image;

[0025] The gesture feature vector of the spliced ​​image is input into a classifier to train a gesture recognition model, and based on the trained gesture recognition model, a gesture recognition result is obtained, wherein the gesture recognition result is a gesture action that is the same as the original gesture action, wherein the gesture action corresponding to the spliced ​​image is used as an output item of the gesture recognition model, and the spliced ​​image is imported into the trained gesture recognition model.

[0026] Optionally, each two-dimensional convolutional neural network includes a two-dimensional convolutional layer and a two-dimensional pooling layer, and the two-dimensional convolutional layer and the two-dimensional pooling layer are arranged alternately. The two-dimensional convolutional layer extracts features from information in the image, and the two-dimensional pooling layer compresses the feature information to reduce network forward operation consumption.

[0027] The present application also provides a gesture recognition system based on millimeter wave radar, which is characterized by comprising:

[0028] A signal acquisition module, used for collecting gesture echo signals based on the millimeter wave radar, and performing static clutter suppression processing on the collected gesture echo signals;

[0029] An image processing module is used to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively by using short-time Fourier transform, fast Fourier transform and MVDR beamforming method to process the gesture echo signal after static clutter suppression.

[0030] The gesture recognition module is used to perform gesture recognition model training on the obtained time-frequency domain image, time-distance domain image and time-angle domain image to obtain gesture recognition results.

[0031] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0032] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0033] Beneficial effects:

[0034] By using gesture echo signals collected by multiple millimeter-wave radars, images in the time-frequency domain, images in the time-distance domain, and images in the time-angle domain are generated, and the same gesture is expressed from different spatial perspectives to improve the completeness of gesture feature expression. The multi-channel parallel input method is used to extract and fuse various feature information in the image and perform gesture recognition, effectively solving the problems of low accuracy of easily confused gesture recognition and insufficient use of feature information in gesture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The schematic embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0036] Figure 1 A flowchart of a gesture recognition method based on millimeter-wave radar provided in an embodiment of the present application;

[0037] Figure 2 A mean cancellation principle diagram provided in an embodiment of the present application;

[0038] FIG3(a) and FIG3(b) are images in the time-frequency domain, images in the time-distance domain, and images in the time-angle domain under different gestures provided by an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structure of a gesture recognition system based on millimeter wave radar provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0042] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0043] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0044] Embodiment 1:

[0045] like Figure 1 The figure is a flow chart of a gesture recognition method based on millimeter wave radar proposed in an embodiment of the present invention, including:

[0046] S1. Collect gesture echo signals based on the millimeter wave radar, and perform static clutter suppression processing on the collected gesture echo signals.

[0047] Specifically, the millimeter-wave radar includes a transmitting antenna and a receiving antenna. The millimeter-wave radar is real-time and can quickly and accurately obtain dynamic information of gestures, providing a more flexible and instant user interaction experience. In addition, the millimeter-wave radar is more adaptable in complex environments and does not require large pulse power, which helps to miniaturize the equipment and reduce power consumption. The millimeter-wave radar collects gesture echo signals and performs static clutter suppression on the collected gesture echo signals.

[0048] In order to perform static clutter suppression processing on the gesture echo signal, in a preferred embodiment of the present solution, the collected gesture echo signal is subjected to static clutter suppression processing, including:

[0049] The phases of multiple gesture echo signals are averaged by a mean cancellation algorithm to obtain a mean pulse, which is used to perform a cancellation operation to suppress static clutter.

[0050] Specifically, in actual radar detection scenarios, affected by surrounding stationary objects, the radar echo signal contains zero-frequency echoes from stationary objects and low-frequency echoes generated by the human body, which interfere with the effective extraction of target gesture reflection signals. Therefore, in order to overcome the interference caused by stationary objects and low-frequency objects, appropriate signal processing methods need to be used.

[0051] This application adopts the mean cancellation algorithm to suppress static clutter. The mean cancellation algorithm suppresses static clutter by performing a mean operation on the phases of multiple gesture echo signals. The core of this algorithm is that the distance from the stationary target to the radar antenna remains unchanged, and the time delay of the stationary target on each beam of received pulses also remains unchanged. The reference received pulse is obtained by averaging all received pulses, and then the target echo signal is obtained by subtracting the reference received pulse from each beam of received pulses.

[0052] Assume that the stationary target echo signal at different times is represented by n1 in the complex plane, and the moving target echo is m1, m2, m3. Figure 2 It can be seen that since the phase of the moving target echo changes with time, the modulus of its summed vector is relatively small. Assuming that the moving target is random, the sum mΣ of the moving target echo vector will approach zero as the sampling time increases. In contrast, the gesture echo signal of the stationary target remains basically unchanged during multiple sampling moments. Therefore, by averaging multiple gesture echo signals, a reference pulse can be obtained, which can be used for cancellation operations to effectively suppress static clutter.

[0053] S2. The gesture echo signal after static clutter suppression is processed by short-time Fourier transform, fast Fourier transform and MVDR beamforming method to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively.

[0054] Specifically, the essence of short-time Fourier transform is Fourier transform with window function. Fourier transform can only reflect the frequency domain characteristics of the signal and cannot analyze the signal in the time domain, but the data processed by short-time Fourier transform can represent the change of signal frequency in the time dimension. It assumes that the non-stationary signal is stationary in the short time of the analysis window. By moving the window function W on the time axis, the signal is analyzed segment by segment to obtain the frequency domain image at different times.

[0055] In order to obtain a time-distance domain image, in a preferred embodiment of the present solution, the gesture echo signal after static clutter suppression is processed by fast Fourier transform to generate a corresponding time-distance domain image, including:

[0056] According to the millimeter wave radar ranging principle, the radial distance of the target gesture from the millimeter wave radar is obtained by using the time delay between the transmission signal and the gesture echo signal;

[0057] Based on the radial distance of the target gesture from the millimeter wave radar, the gesture echo signal is processed by fast Fourier transform to obtain a distance spectrum of the target gesture, and based on the distance spectrum of the target gesture, an image in the time-distance domain is obtained.

[0058] Specifically, according to the millimeter wave radar ranging principle, the radial distance of the target from the radar can be obtained by using the time delay τ between the transmission signal and the gesture echo signal. Based on the radial distance of the target gesture from the millimeter wave radar, the gesture echo signal is processed by fast Fourier transform to obtain the distance spectrum of the target gesture.

[0059] The range spectrum of the target gesture is obtained by processing the gesture echo signal through fast Fourier transform:

[0060]

[0061] In the formula, R m Represents the distance spectrum of the target gesture, R m (k,m) represents the amplitude of the mth chirp signal at point k, W i represents the window function, N ADC Represents the number of sampling points, S(N ADC -i,m) represents the Nth chirp signal of the mth chirp ADC -i sampling points represent imaginary units.

[0062] Millimeter-wave radar continuously transmits a pulse signal whose frequency changes linearly with time, which is called a linear frequency modulated pulse signal or chirp signal.

[0063] If the target gesture is far away from the millimeter wave radar, the echo signal energy will be low, resulting in unclear difference between the feature information and the background. In order to enhance the target feature information, the multi-channel data accumulation method can be used. The specific implementation method is as follows:

[0064]

[0065] Where: Representative Hadamard, R mc Represents the distance spectrum obtained by the Cth receiving channel, and finally the time-distance domain image is obtained.

[0066] In order to obtain a time-angle domain image, in a preferred embodiment of the present solution, the gesture echo signal after static clutter suppression is processed to generate a time-angle domain image through an MVDR beamforming method, including:

[0067] Acquire a gesture echo signal received by a linear antenna array;

[0068] The gesture echo signal is processed by using the MVDR beamforming algorithm to obtain the angle information of the target gesture, and a time-angle domain image is obtained according to the angle information of the target gesture.

[0069] Specifically, the minimum variance distortionless response (MVDR) beamforming method is used to generate images in the time-angle domain. The phase difference in the vertical direction of the echoes from different receiving antennas can be obtained by using the distance difference between different antennas and the target object. The angle information of the target can be estimated by this phase difference. This paper uses the MVDR beamforming algorithm to process the gesture echo signal to obtain the angle information of the target gesture, and obtains the time-angle domain image based on the angle information of the target gesture.

[0070] The expression of the angle information of the target gesture obtained by processing the gesture echo signal using the MVDR beamforming algorithm is:

[0071] A (θ,m) =a (θ) R xm a′ (θ) ;

[0072] In the formula, A (θ,m) represents the echo signal energy at the θ position in the mth chirp signal, A represents the angle of the target gesture, and R xm Represents the autocorrelation matrix of the distance spectrum of the mth chirp signal, a (θ) is the orientation vector. (θ) It is obtained by the wavelength of the millimeter-wave radar and the antenna array spacing.

[0073] As shown in Figure 3(a) and Figure 3(b), a set of images in the time-frequency domain, the time-distance domain, and the time-angle domain are displayed. The features of different gestures in the same domain may be very similar. Therefore, if a single type of feature is used for recognition, the recognition accuracy will be affected in an unpredictable way. In order to distinguish easily confused gestures, this paper adopts a multi-domain representation fusion method, using complementary features in different domains to more fully express the gesture. The gesture features of three different domains are fused to make the feature expression of each type of gesture more complete. This method can greatly improve the recognition accuracy of easily confused gestures.

[0074] S3. Perform gesture recognition model training on the obtained time-frequency domain image, time-distance domain image and time-angle domain image to obtain gesture recognition results.

[0075] In order to obtain gesture recognition results, in a preferred embodiment of the present solution, the obtained time-frequency domain image, time-distance domain image and time-angle domain image are subjected to gesture recognition model training to obtain gesture recognition results, including:

[0076] The normalized time-frequency domain image, time-distance domain image and time-angle domain image are input into the corresponding three two-dimensional convolutional neural networks in parallel to obtain the convolution features of each image, wherein each two-dimensional convolutional neural network includes a two-dimensional convolutional layer and a two-dimensional pooling layer, the two-dimensional convolutional layer and the two-dimensional pooling layer are arranged alternately, the two-dimensional convolutional layer extracts features from the information in the image, and the two-dimensional pooling layer compresses the feature information to reduce the network forward operation consumption; each image output by the two-dimensional convolutional neural network is spliced ​​to form a spliced ​​image, and the spliced ​​image is a spliced ​​image of the time-frequency domain. The image in the time-distance domain, the image in the time-angle domain and the image in the time-angle domain are spliced ​​into one image through a two-dimensional convolutional neural network, and based on the spliced ​​image, the convolution features of each image are fused to obtain a gesture feature vector of the spliced ​​image; the gesture feature vector of the spliced ​​image is input into a classifier to train a gesture recognition model, and based on the gesture recognition model after the training, a gesture recognition result is obtained, and the gesture recognition result is a gesture action that is the same as the original gesture action, wherein the gesture action corresponding to the spliced ​​image is used as an output item of the gesture recognition model, and the spliced ​​image is imported into the gesture recognition model after the training.

[0077] Specifically, the same gesture echo signal is used to realize the multi-dimensional feature expression of the target motion state, and representation images of different domains are obtained, including the frequency, distance and horizontal angle of the target. These three features are highly complementary. If only one convolutional neural network is used to extract features from the radar multi-domain representation images (images in the time-frequency domain, images in the time-distance domain, and images in the time-angle domain), the spatial resolution will be reduced. Therefore, the present application uses three parallel two-dimensional convolutional neural networks. This design allows the network to effectively extract and fuse spatial texture features in multi-domain representation images while maintaining a low amount of computation. The extracted spatial texture features are used to obtain better recognition effects in subsequent gesture recognition.

[0078] The three 2D convolutional neural networks have the same composition structure. Each 2D convolutional neural network includes 5 2D convolutional layers, 2 2D pooling layers, and 1 adaptive average pooling layer. The network adopts a structure in which convolutional layers and pooling layers are arranged alternately. Among them, the convolutional layer fully extracts the effective information in the images in the time-frequency domain, the time-distance domain, and the time-angle domain, and the pooling layer compresses the effective information to reduce the network forward operation consumption. The images in the time-frequency domain, the time-distance domain, and the time-angle domain are inputted into the corresponding three two-dimensional convolutional neural networks in parallel to obtain the convolutional features of each image, and each image outputted by the two-dimensional convolutional neural network is spliced ​​to form a spliced ​​image; based on the spliced ​​image, the convolutional features of each image are fused to obtain the gesture feature vector of the spliced ​​image; the gesture feature vector of the spliced ​​image is inputted into the classifier to train the gesture recognition model, and based on the gesture recognition model after training, the gesture recognition result is obtained, and the gesture recognition result is the same gesture action as the original gesture action, wherein the gesture action corresponding to the spliced ​​image is used as the output item of the gesture recognition model, and the spliced ​​image is imported into the gesture recognition model after training. The classifier has three layers, including an input layer, an intermediate layer, and an output layer. A deeper fully connected layer can improve the complexity and nonlinear expression ability of the gesture recognition model, which is conducive to improving the learning ability of the gesture recognition model.

[0079] As an embodiment of the present application, before training the obtained time-frequency domain image, time-distance domain image, and time-angle domain image for gesture recognition model, the method further includes:

[0080] The image in the time-frequency domain, the image in the time-distance domain, and the image in the time-angle domain are respectively normalized to adjust the pixel value of each image to a preset range.

[0081] Specifically, the images in the time-frequency domain, the time-distance domain, and the time-angle domain are subjected to data preprocessing. First, the three images are uniformly and adaptively reshaped into a 224×224×3 tensor, and the tensor is normalized, that is, the pixel values ​​of the representation images are adjusted to real numbers in the range of [0,1]. The normalized frequency, distance, and horizontal angle representation images are used as the input of three two-dimensional convolutional neural networks to extract the unique feature information of each type of representation image.

[0082] The millimeter wave-based gesture recognition method of the present application first uses short-time Fourier transform, fast Fourier transform and MVDR beamforming methods to sequentially generate time-frequency domain, time-distance domain and time-angle domain images. For the three images, three parallel two-dimensional convolutional neural networks are designed to extract convolution features from the three images respectively and fuse them, and finally send them to the classifier for gesture recognition model training to perform gesture recognition.

[0083] Figure 4 A structural diagram of a gesture recognition system based on millimeter wave radar provided by the present invention includes:

[0084] The signal acquisition module 10 is used to collect gesture echo signals based on the millimeter wave radar and perform static clutter suppression processing on the collected gesture echo signals;

[0085] An image processing module 20 is used to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively by using short-time Fourier transform, fast Fourier transform and MVDR beamforming method to process the gesture echo signal after static clutter suppression.

[0086] The gesture recognition module 30 is used to perform gesture recognition model training on the obtained time-frequency domain image, time-distance domain image and time-angle domain image to obtain a gesture recognition result.

[0087] Figure 5 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 4 As shown, the electronic device 60 includes: a processor 601 (processor), a memory 602 (memory) and a bus 603;

[0088] The processor 601 and the memory 602 communicate with each other via the bus 603;

[0089] The processor 601 is used to call the program instructions in the memory 602 to execute the methods provided by the above-mentioned method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative labor.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A gesture recognition method based on millimeter wave radar, characterized in that: include: Collect gesture echo signals based on millimeter-wave radar, and perform static clutter suppression on the collected gesture echo signals; The gesture echo signal after static clutter suppression is processed by short-time Fourier transform, fast Fourier transform and MVDR beamforming method to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively; The obtained time-frequency domain image, time-distance domain image and time-angle domain image are used for gesture recognition model training to obtain gesture recognition results.

2. The method according to claim 1, characterized in that Perform static clutter suppression processing on the collected gesture echo signal, including: The phases of multiple gesture echo signals are averaged by a mean cancellation algorithm to obtain a mean pulse, which is used to perform a cancellation operation to suppress static clutter.

3. The method according to claim 1, characterized in that The gesture echo signal after static clutter suppression is processed by fast Fourier transform to generate the corresponding time-distance domain image, including: According to the millimeter wave radar ranging principle, the radial distance of the target gesture from the millimeter wave radar is obtained by using the time delay between the transmission signal and the gesture echo signal; Based on the radial distance of the target gesture from the millimeter wave radar, the gesture echo signal is processed by fast Fourier transform to obtain a distance spectrum of the target gesture, and based on the distance spectrum of the target gesture, an image in the time-distance domain is obtained.

4. The method according to claim 1, characterized in that The gesture echo signal after static clutter suppression is processed and the image in the time-angle domain is generated by the MVDR beamforming method, including: Acquire a gesture echo signal received by a linear antenna array; The gesture echo signal is processed by using the MVDR beamforming algorithm to obtain the angle information of the target gesture, and a time-angle domain image is obtained according to the angle information of the target gesture.

5. The method according to claim 1, characterized in that Before training the obtained time-frequency domain image, time-distance domain image and time-angle domain image for gesture recognition model to obtain gesture recognition results, the method further includes: The image in the time-frequency domain, the image in the time-distance domain, and the image in the time-angle domain are respectively normalized to adjust the pixel value of each image to a preset range.

6. The method according to claim 5, characterized in that The obtained time-frequency domain image, time-distance domain image and time-angle domain image are subjected to gesture recognition model training to obtain gesture recognition results, including: The normalized time-frequency domain image, time-distance domain image, and time-angle domain image are input into the corresponding three two-dimensional convolutional neural networks in parallel to obtain the convolution features of each image; splicing each image output by the two-dimensional convolutional neural network to form a spliced ​​image; Based on the stitched image, the convolution features of each image are fused to obtain a gesture feature vector of the stitched image; The gesture feature vector of the spliced ​​image is input into a classifier to train a gesture recognition model, and based on the trained gesture recognition model, a gesture recognition result is obtained, wherein the gesture recognition result is a gesture action that is the same as the original gesture action, wherein the gesture action corresponding to the spliced ​​image is used as an output item of the gesture recognition model, and the spliced ​​image is imported into the trained gesture recognition model.

7. The method according to claim 5, characterized in that Each two-dimensional convolutional neural network includes a two-dimensional convolutional layer and a two-dimensional pooling layer, wherein the two-dimensional convolutional layer and the two-dimensional pooling layer are arranged alternately, the two-dimensional convolutional layer extracts features from information in the image, and the two-dimensional pooling layer compresses the feature information to reduce network forward operation consumption.

8. A gesture recognition system based on millimeter wave radar, characterized in that: include: A signal acquisition module, used for collecting gesture echo signals based on the millimeter wave radar, and performing static clutter suppression processing on the collected gesture echo signals; An image processing module is used to generate corresponding time-frequency domain images, time-distance domain images and time-angle domain images respectively by using short-time Fourier transform, fast Fourier transform and MVDR beamforming method to process the gesture echo signal after static clutter suppression. The gesture recognition module is used to perform gesture recognition model training on the obtained time-frequency domain image, time-distance domain image and time-angle domain image to obtain gesture recognition results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the gesture recognition method based on millimeter wave radar are implemented as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the gesture recognition method based on millimeter-wave radar as described in any one of claims 1 to 7 are implemented.

Images