A millimeter wave radar multi-person gesture recognition method

Abstract

This invention discloses a method for multi-person gesture recognition using millimeter-wave radar. First, gesture actions are designed based on potential application scenarios, and radar system parameters are determined. Then, gesture data is acquired and preprocessed, including filtering and gesture signal extraction. Next, the number of gestures is estimated using a blind source number estimation method, and multiple gesture data are separated using a blind source separation method. Finally, a classification model is designed, inputting gesture signal data or manually extracted features to obtain the gesture recognition result. This method solves the problem that existing millimeter-wave radar sensor gesture recognition technology can only recognize single gesture actions and cannot recognize multiple gesture actions existing simultaneously within the detection range. It achieves multi-person gesture recognition, enhances the practicality of radar gesture recognition technology, and can complete the task of recognizing multiple gestures existing simultaneously in multi-person scenarios.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and radar technology, and specifically relates to a method for multi-person gesture recognition using millimeter-wave radar. Background Technology

[0002] Gesture recognition is an exploration of the next generation of human-computer interaction, aiming to communicate with machines in a more natural and convenient way. Gesture recognition technology has potential applications in fields such as smart cockpits, smart homes, and consumer electronics.

[0003] The implementation of gesture recognition is closely related to the development of sensor technology. Different sensor technologies will employ different methods to perceive gestures. Radar-based gesture recognition solutions have many advantages, such as being unaffected by lighting conditions, strong privacy, smooth operation, and the ability to capture dynamic gestures. These advantages make them stand out from other gesture recognition technologies, such as vision-based and wearable device-based solutions. Furthermore, their low cost, low power consumption, small size, and strong penetration capabilities allow radar-based gesture recognition hardware modules to be easily integrated into other devices, which is more conducive to productization.

[0004] While radar gesture recognition technology continues to achieve higher precision and accuracy, we have noticed that most existing gesture recognition technologies limit the existence of only a single gesture within the radar detection range. How to improve the freedom of gesture operation and realize multiple gesture recognition tasks in real-world scenarios with multiple people has become an urgent problem to be solved.

[0005] The paper "Blind separation of Doppler human gesture signals based on continuous-wave radar sensors IEEE Transactions on Instrumentation and Measurement 2019, 68(7): 2659-2661" uses independent principal component analysis (ICA) to separate the breathing and gesture signals of a single person, verifying that blind source separation technology can achieve the separation of multi-source limb signals. However, this method does not consider the separation of gesture signals from multiple people. The paper "Hand gesture recognition using FMCW radar in multi-person scenario IEEE Topical Conference on Wireless Sensors and SensorNetworks (WiSNeT) 2021: 50-52" focuses on the gesture recognition problem in multi-person scenarios, but only analyzes the feature distribution in multi-person scenarios and does not propose a practical and effective solution for recognizing multiple gestures in multi-person scenarios. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a millimeter-wave radar multi-person gesture recognition method. It addresses the shortcomings of existing radar gesture recognition technologies in multi-person gesture scenarios by resolving issues such as the unknown number of gestures and overlapping features, which are unresolved by existing technologies. This enables multi-person gesture recognition and enhances the practicality of radar gesture recognition technology.

[0007] To facilitate the description of the present invention, the following terms will first be explained:

[0008] Term 1: Multi-person gestures;

[0009] Multi-person gestures refer to the presence of multiple people within the detection range of a radar sensor, with one or more people performing gesture actions. These gestures include, but are not limited to, gestures from a gesture action library.

[0010] Term 2: Blind source number estimation;

[0011] Blind source number estimation refers to estimating the number of sources in a mixed signal. This information is often used as a condition for subsequent blind source separation to assist in the implementation of blind source separation algorithms.

[0012] Term 3: Blind source separation;

[0013] Blind source separation refers to separating mixed signals by observing them and using the statistical independence properties between signals, with limited prior knowledge, to extract useful source signals.

[0014] The technical solution adopted in this invention is: a multi-person gesture recognition method using millimeter-wave radar, the specific steps of which are as follows:

[0015] S1. Design gesture actions based on potential application scenarios and determine the parameters of the millimeter-wave radar system;

[0016] S2. The radar sensor transmits detection signals and receives gesture echo data, and performs filtering processing on the echo data;

[0017] S3. Use the blind source number estimation method to estimate the number of gestures;

[0018] S4. Use blind source separation method to separate multiple gesture data;

[0019] S5. Design a classification model, input gesture signal data or manually extracted features, and obtain gesture recognition results.

[0020] Furthermore, step S1 is specifically as follows:

[0021] S11. Design gestures and actions;

[0022] Considering two typical multi-person gesture recognition application scenarios, namely smart cockpit and smart home, the gesture actions are designed as follows:

[0023] (1) Raise: Within the radar detection range, raise your palm naturally (away from the radar direction) with a displacement of more than 10cm;

[0024] (2) Press down: Within the radar detection range, press down naturally with your palm (closer to the radar direction), with a displacement of more than 10cm;

[0025] (3) Left swing: Within the radar detection range, the palm and forearm naturally swing from right to left, with a displacement of more than 10cm;

[0026] (4) Right swing: Within the radar detection range, the palm and forearm naturally swing from left to right, with a displacement of more than 10cm;

[0027] (5) Double tap: Within the radar detection range, keep your arm still and tap up and down twice with your palm or fingers;

[0028] (6) Thumb flicking: Within the radar detection range, the palm is still and the four fingers are bent, and the thumb flicks up and down along the index finger.

[0029] S12. Determine the parameters of the millimeter-wave radar system;

[0030] Using FMCW millimeter-wave radar for radar gesture recognition, the parameters set include: number of transmit and receive antennas, radar carrier frequency f... c Signal sweep bandwidth B, signal time width T c Pulse repetition frequency (PRF), number of chirp signal sampling points (n) chirp .

[0031] Furthermore, step S2 is specifically as follows:

[0032] S21. Radar signal transmission and reception;

[0033] The radar transmitter emits a sawtooth wave linear frequency modulated signal s T (t):

[0034]

[0035] Where t is a time variable, the signal is reflected by the gesture and received by the receiving antenna after a time delay τ. The FMCW radar receiver uses a mixer to obtain s by differentiating the frequency of the transmitted and received signals. IF (t), represented as:

[0036]

[0037] Replace B / T with K c Meanwhile, the time delay τ equals Where R represents the relative distance between the target and the radar, v r Let represent the radial velocity of the motion and radar, and 'c' represent the speed of light. After approximation, the intermediate frequency signal is represented as:

[0038]

[0039] The intermediate frequency signals received by all receiving channels pass through filters and ADC converters in sequence.

[0040] S22. Moving target display filtering preprocessing;

[0041] Using an exponentially weighted average MTI filtering method:

[0042]

[0043]

[0044] Where s(n) represents the nth received intermediate frequency echo signal, Let represent the filtered signal, and α represent the smoothing factor. From the above formula, we can obtain the non-recursive formula for background clutter:

[0045]

[0046] Among them, the smaller the smoothing factor α of the moving target display filter, the stronger its clutter suppression capability.

[0047] Furthermore, step S3 is specifically as follows:

[0048] S31. Perform a unitary transformation on the covariance matrix of the received signal;

[0049] Let the number of receiving sensors be N, then for an N×N dimensional received signal covariance matrix R x Let the element at (i, j) be represented as a. ij Then define the center c of the Gaelic circle. i =a ii The radius of the Gaelic circle is:

[0050]

[0051] To make the differences between the Geil circles corresponding to the source and noise greater, the covariance matrix R of the received data is first adjusted. X Perform a unitary transformation.

[0052] S32. The number of information sources is determined by the decision threshold;

[0053] After unitary transformation, the radius of the Gell circle is calculated. Assuming there are M unrelated sources, the decision threshold GDE(k) for estimating the number of blind sources based on the Gell circle criterion is set as follows:

[0054]

[0055] Where k ranges from 1, 2, ..., N-1; D(L) represents a correction factor ranging from 0 to 1; r k This represents the radius of the k-th Gell-Hill circle.

[0056] The calculation of GDE(k) starts from k=1 and stops when GDE(k) takes a negative value for the first time. The estimated number of source signals is k-1.

[0057] Furthermore, in step S4, a feature matrix joint approximate diagonalization algorithm is used for blind source separation, as detailed below:

[0058] S41. Establish a signal model;

[0059] The radar array receiving signal model is: X(t)=AS(t), where S(t) consists of M source signals, X(t) consists of N receiving sensor channel signals, and A represents the N×M dimensional confusion coefficient matrix.

[0060] S(t)=[s1(t),s2(t),…,s M (t)] T

[0061] X(t)=[x1(t),x2(t),…,x N (t)] T

[0062] Where T represents the transpose of the matrix.

[0063] S42. Preprocessing received data;

[0064] The observed data X(t) is centered and whitened to remove correlation and redundancy between source signals, resulting in preprocessed data Z(t):

[0065] Z(t) = WX(t) = WAS(t) = US(t)

[0066] Where W represents the whitening matrix and U represents the confusion matrix after whitening.

[0067] S43. Construct a fourth-order cumulant matrix Q using the whitened data. z (M);

[0068] S44, regarding the fourth-order cumulant matrix Q z (M) Perform eigenvalue decomposition to obtain the estimate V of matrix U;

[0069] S45. Reconstruct the signals s1(t), s2(t), ... s using the above formula. M (t), yielding the blind source separation result Y = V H WX, H represents conjugate transpose.

[0070] Furthermore, step S5 is specifically as follows:

[0071] S51. Determine the model input and extract gesture motion features;

[0072] For any gesture signal from the multi-person gestures obtained from the above steps, end-to-end recognition is performed using the raw data, or one or more features of the gesture are extracted for recognition. The features of the gesture mainly include distance-Doppler features, distance-time features, Doppler-time features, angle-time features, etc.

[0073] Distance information extraction: The intermediate frequency signal frequency f is obtained by performing a fast Fourier transform on a single pulse. IF The distance information of the gesture within the current single pulse time is obtained, and the distance information of each pulse is integrated to obtain the overall distance change map of a single gesture. The formula is:

[0074]

[0075] Doppler information extraction: After obtaining distance information from a single pulse, a Fourier transform is performed again in the slow time dimension to obtain the Doppler frequency f.d To represent the speed information of gestures; short-time Fourier transform or wavelet transform are used to obtain the speed change graph of dynamic gestures:

[0076]

[0077] Where λ represents wavelength.

[0078] Angle information extraction: When using multiple receiving antennas to receive gesture echoes, the angle information of the gesture echoes is obtained by utilizing the phase information between different receiving antennas.

[0079] The MUSIC algorithm is used to estimate the dynamic angle change of the gesture. mu (θ).

[0080] S52. Design a classification model to obtain gesture recognition results;

[0081] Based on the model input determined by the above steps, a radar gesture recognition classification model is designed to obtain the gesture recognition result. The classification model design includes models based on SVM, KNN, random forest, hidden Markov, RNN, LSTM, CNN, and Transformer.

[0082] The beneficial effects of this invention are as follows: The method of this invention first designs gesture actions based on potential application scenarios and determines radar system parameters. Then, it performs gesture data acquisition and signal preprocessing, including filtering and gesture signal extraction. Next, it estimates the number of gestures using a blind source number estimation method and separates multiple gesture data using a blind source separation method. Finally, it designs a classification model, inputting gesture signal data or manually extracted features to obtain gesture recognition results. This invention solves the problem that existing millimeter-wave radar sensor gesture recognition technology can only recognize single gesture actions and cannot recognize multiple gesture actions existing simultaneously within the detection range. It achieves multi-person gesture recognition, enhances the practicality of radar gesture recognition technology, and can complete the task of recognizing multiple gestures existing simultaneously in multi-person scenarios. Attached Figure Description

[0083] Figure 1 This is a flowchart of a millimeter-wave radar multi-person gesture recognition method according to the present invention.

[0084] Figure 2 This is a typical application scenario diagram of multi-person gestures in an embodiment of the present invention.

[0085] Figure 3 This is a schematic diagram of the gesture action library in an embodiment of the present invention.

[0086] Figure 4 This is a diagram showing the estimation result of the number of gestures of a single person in an embodiment of the present invention.

[0087] Figure 5 This is a diagram showing the estimation result of the number of gestures between two people in an embodiment of the present invention.

[0088] Figure 6 These are images showing the results of extracting features from four types of single-person gestures in embodiments of the present invention.

[0089] Figure 7 This is a comparison diagram of feature extraction results before and after two-person gesture separation in three embodiments of the present invention. Detailed Implementation

[0090] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0091] In Example 1, as Figure 1 The flowchart of a multi-person gesture recognition method using millimeter-wave radar according to the present invention is shown below, and the specific steps are as follows:

[0092] S1. Design gesture actions based on potential application scenarios and determine the parameters of the millimeter-wave radar system;

[0093] S2. The radar sensor transmits detection signals and receives gesture echo data, and performs filtering processing on the echo data;

[0094] S3. Use the blind source number estimation method to estimate the number of gestures;

[0095] S4. Use blind source separation method to separate multiple gesture data;

[0096] S5. Design a classification model, input gesture signal data or manually extracted features, and obtain gesture recognition results.

[0097] In this embodiment, step S1 is specifically as follows:

[0098] S11. Design gestures and actions;

[0099] Multi-person gesture recognition refers to the presence of multiple people within the detection range of a radar sensor, with one or more people performing gesture actions. Multi-person gestures are random combinations of gestures from a gesture action library. This embodiment considers two typical application scenarios for multi-person gesture recognition: smart cockpits and smart homes, such as... Figure 2 As shown, (a) is a smart cockpit scenario, and (b) is a smart home scenario.

[0100] In this embodiment, the installation position of the millimeter-wave radar sensor is designed to ensure that there is no mutual obstruction between multiple hand gestures.

[0101] When applied to smart cockpits, this invention can place millimeter-wave radar sensors between the driver and passenger, such as in the center console, armrest box, or the ceiling above the armrest box.

[0102] When applied to smart homes, this invention can integrate millimeter-wave radar sensors into home appliances such as televisions and stereos.

[0103] like Figure 3 As shown, the designed gesture actions are as follows:

[0104] (1) Raise: Within the radar detection range, raise your palm naturally (away from the radar direction) with a displacement of more than 10cm;

[0105] (2) Press down: Within the radar detection range, press down naturally with your palm (closer to the radar direction), with a displacement of more than 10cm;

[0106] (3) Left swing: Within the radar detection range, the palm and forearm naturally swing from right to left, with a displacement of more than 10cm;

[0107] (4) Right swing: Within the radar detection range, the palm and forearm naturally swing from left to right, with a displacement of more than 10cm;

[0108] (5) Double tap: Within the radar detection range, keep your arm still and tap up and down twice with your palm or fingers;

[0109] (6) Thumb flicking: Within the radar detection range, the palm is still and the four fingers are bent, and the thumb flicks up and down along the index finger.

[0110] S12. Determine the parameters of the millimeter-wave radar system;

[0111] Current radar gesture recognition technology relies on different radar sensors such as linear frequency modulated continuous wave (FMCW) radar, continuous wave (CW) radar, and pulse radar. In this embodiment, FMCW millimeter-wave radar is used to complete radar gesture recognition. The parameters set include: the number of transmit and receive antennas, and the radar carrier frequency f. c Signal sweep bandwidth B, signal time width T c Pulse repetition frequency (PRF), number of chirp signal sampling points (n) chirp .

[0112] Furthermore, step S2 is specifically as follows:

[0113] S21. Radar signal transmission and reception;

[0114] The radar transmitter emits a sawtooth wave linear frequency modulated signal s T (t):

[0115]

[0116] Where t is a time variable, the signal is reflected by the gesture and received by the receiving antenna after a time delay τ. The FMCW radar receiver uses a mixer to obtain s by differentiating the frequency of the transmitted and received signals. IF(t), represented as:

[0117]

[0118] Replace B / T with K c Meanwhile, the time delay τ equals Where R represents the relative distance between the target and the radar, v r Let represent the radial velocity of the motion and radar, and 'c' represent the speed of light. After approximation, the intermediate frequency signal is represented as:

[0119]

[0120] The intermediate frequency signals received by all receiving channels pass through filters and ADC converters in sequence.

[0121] S22. Moving target display filtering preprocessing;

[0122] For interfering targets within radar range, stationary targets and low-speed moving targets are the primary considerations, which can be filtered out using a Moving Target Indication Filter (MTI Filter). Object movement generates a Doppler frequency shift, and the faster the speed, the greater the Doppler frequency shift. Stationary targets and low-speed targets will generate echo accumulation at zero-frequency and low-frequency locations. MTI Filters often use primary or secondary clutter cancellers.

[0123] This embodiment uses an exponentially weighted average MTI filtering method:

[0124]

[0125]

[0126] Where s(n) represents the nth received intermediate frequency echo signal, Let represent the filtered signal, and α represent the smoothing factor. From the above formula, we can obtain the non-recursive formula for background clutter:

[0127]

[0128] In this moving target display filter, the smaller the smoothing factor α, the stronger the clutter suppression capability. In this embodiment, it is set to 0.3.

[0129] Furthermore, step S3 is specifically as follows:

[0130] In multi-person gesture scenarios, multiple moving targets may exist simultaneously within the radar detection range. These moving targets can refer to multiple simultaneous gestures or include random, highly interfering actions. To address this issue, this embodiment employs a blind source number estimation method based on the Gell circle criterion for gesture number estimation. This method does not require knowledge of the noise model and does not demand ideal Gaussian white noise, making it more practical.

[0131] S31. Perform a unitary transformation on the covariance matrix of the received data;

[0132] Let the number of receiving sensors be N, then for an N×N dimensional received signal covariance matrix R x Let the element at (i, j) be represented as a. ij Then define the center c of the Gaelic circle. i =a ii The radius of the Gaelic circle is:

[0133]

[0134] To make the differences between the Geil circles corresponding to the source and noise greater, the covariance matrix R of the received data is first adjusted. X Perform a unitary transformation.

[0135] S32. The number of information sources is determined by the decision threshold;

[0136] After unitary transformation, the radius of the Gell circle is calculated. Assuming there are M unrelated sources, the decision threshold GDE(k) for estimating the number of blind sources based on the Gell circle criterion is set as follows:

[0137]

[0138] Where k ranges from 1, 2, ..., N-1; D(L) represents a correction factor ranging from 0 to 1; r k Let D(L) represent the radius of the k-th Gell circle. The radius of the noise Gell circle decreases as the number of samples L increases, therefore D(L) is a non-increasing function of the number of samples L.

[0139] The calculation of GDE(k) starts from k=1 and stops when GDE(k) takes a negative value for the first time, estimating the number of source signals as k-1. This shows that increasing the number of transceivers in a millimeter-wave radar can effectively improve the upper limit of the number estimation.

[0140] Furthermore, in step S4, a feature matrix joint approximate diagonalization algorithm is used for blind source separation, as detailed below:

[0141] Blind source separation refers to separating mixed received signals by utilizing their independent statistical properties, with limited prior knowledge, to extract useful echo signals. This invention employs the Joint Approximate Diagonalization of Eigen Matrices (JADE) algorithm for blind source separation.

[0142] S41. Establish a signal model;

[0143] The radar array receiving signal model is: X(t)=AS(t), where S(t) consists of M source signals, X(t) consists of N receiving sensor channel signals, and A represents the N×M dimensional confusion coefficient matrix.

[0144] S(t)=[s1(t),s2(t),…,s M (t)] T

[0145] X(t)=[x1(t),x2(t),…,x N (t)] T

[0146] Where T represents the transpose of the matrix.

[0147] S42. Preprocessing received data;

[0148] The observed data X(t) is centered and whitened to remove correlation and redundancy between source signals, resulting in preprocessed data Z(t):

[0149] Z(t) = WX(t) = WAS(t) = US(t)

[0150] Where W represents the whitening matrix and U represents the confusion matrix after whitening.

[0151] S43. Construct a fourth-order cumulant matrix Q using the whitened data. z (M);

[0152] S44, regarding the fourth-order cumulant matrix Q z (M) Perform eigenvalue decomposition to obtain the estimate V of matrix U;

[0153] S45. Reconstruct the signals s1(t), s2(t), ... s using the above formula. M (t), yielding the blind source separation result Y = V H WX, H represents conjugate transpose.

[0154] Furthermore, step S5 is specifically as follows:

[0155] S51. Determine the model input and extract gesture motion features;

[0156] For any gesture signal from the multi-person gestures obtained from the above steps, end-to-end recognition is performed using the raw data, or one or more features of the gesture are extracted for recognition. The features of the gesture mainly include distance-Doppler features, distance-time features, Doppler-time features, angle-time features, etc.

[0157] Distance information extraction: Distance information is contained in the frequency of each echo pulse. By performing a fast-time Fourier transform on a single pulse, the intermediate frequency signal frequency f is obtained. IF The distance information of the gesture within the current single pulse time is obtained, and the distance information of each pulse is integrated to obtain the overall distance change map of a single gesture. The formula is:

[0158]

[0159] Doppler information extraction: After obtaining distance information from a single pulse, a Fourier transform is performed again in the slow time dimension to obtain the Doppler frequency f. d To represent the speed information of the gesture; when performing Fourier transform in the slow time dimension, it is necessary to ensure that the target is at the same distance gate in order to obtain accurate Doppler information. Short-time Fourier transform or wavelet transform is used to obtain the speed change map of the dynamic gesture:

[0160]

[0161] Where λ represents wavelength.

[0162] Angle information extraction: When using multiple receiving antennas to receive gesture echoes, the angle information of the gesture echoes is obtained by utilizing the phase information between different receiving antennas.

[0163] The MUSIC (Multiple Signal Classification) algorithm is used to estimate the dynamic angle change of the gesture, P. mu (θ).

[0164] S52. Design a classification model to obtain gesture recognition results;

[0165] Based on the model input determined by the above steps, a radar gesture recognition classification model is designed to obtain the gesture recognition result. The classification model design is currently mainly divided into two categories: machine learning methods and deep learning methods, including those based on SVM, KNN, random forest, hidden Markov, RNN, LSTM, CNN, and Transformer.

[0166] The present invention also provides Embodiment 2, which uses a 60GHz band large bandwidth, one-transmit four-receive FMCW millimeter wave radar with a bandwidth of 6.8GHz. Other parameters are shown in Table 1.

[0167] Table 1

[0168] parameter numerical values Send and receive mechanism 1 send 4 receive Bandwidth B 6.8GH Frequency sweep range 57.1-63.9GHz Distance resolution ΔR 2.2cm <![CDATA[Signal time width T c > 55μs <![CDATA[Frame rate R FRAME > 90Hz Pulse Repetition Frequency (PRF) 1440Hz Chirp sampling points 64 Receiver Gain (RX Gain) 60dB

[0169] Four volunteers (two men and two women) were recruited to conduct the operation within a 0.1m-0.5m field of view above the radar. Figure 3 The four single-person gestures shown are (1) raising, (2) pressing down, (5) double-clicking, and (6) thumb flicking. There are also three combined double-person gestures made up of these four single-person gestures: raising-pressing down, raising-double-clicking, and double-clicking-thumb flicking. The gesture dataset contains a total of 7 categories × 100 data samples.

[0170] The obtained single-person and two-person gesture data are processed as described in Example 1. First, filtering is performed, and then the number of gestures is estimated for the seven types of data. The results are as follows. Figure 4 , Figure 5 As shown in Table 2, the accuracy of gesture number estimation is highly correlated with the correction factor. Considering all factors, the optimal range for the correction factor is 0.45-0.60.

[0171] Table 2

[0172]

[0173] Based on achieving high-accuracy gesture number estimation, the effectiveness of gesture separation is verified.

[0174] like Figure 6 As shown, (a), (b), (c), and (d) are respectively examples of this embodiment. Figure 3 The results of feature extraction for the four single-person gestures shown are (1) raising, (2) pressing down, (5) double-clicking, and (6) thumb flicking, which are distance, micro-Doppler, and angle feature extraction respectively.

[0175] like Figure 7 As shown, (a), (b), and (c) are comparison images of the feature extraction results after separating the three double-person gestures (① raise-② lower), ① double-tap-② raise, and ① thumb flick-② double-tap) using the four single-person gestures in this embodiment according to the gesture separation method. Distance, micro-Doppler, and angle features were extracted before and after the double-person gestures were separated. From top to bottom, the features are: signal features before separation, gesture ① features after separation, and gesture ② features after separation.

[0176] As can be seen, using the method of the present invention, the number of gestures can be accurately estimated based on the gesture data of multiple people, and the single-person gesture data can be separated from it. The obtained features are consistent with the features directly extracted from the single-person gesture.

[0177] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for multi-person gesture recognition using millimeter-wave radar, the specific steps of which are as follows: S1. Design gesture actions based on potential application scenarios and determine the parameters of the millimeter-wave radar system; Using FMCW millimeter-wave radar to perform radar gesture recognition, the parameters set include: Number of transmit and receive antennas, radar carrier frequency Signal sweep bandwidth Signal bandwidth Pulse repetition frequency Number of chirp signal sampling points ; S2. The radar sensor transmits detection signals and receives gesture echo data, and performs filtering processing on the echo data; S3. Estimate the number of gestures using a blind source number estimation method; step S3 is as follows: S31. Perform a unitary transformation on the covariance matrix of the received signal; Let the number of receiving sensors be... For 3D received signal covariance matrix ,set up The element is represented as Then define the center of the Gaelic circle. The radius of the Gaelic circle is: ； To make the differences between the Geil circles corresponding to the source and noise greater, the covariance matrix of the received data is first adjusted. Perform unitary transform; S32, determine the number of information sources by the decision threshold; Find the radius of the Gell circle after a unitary transformation, assuming the existing... For each unrelated source, a judgment threshold is set based on the Gell circle criterion for estimating the number of blind sources. for: ； in, The value range is 1, 2, ..., N-1; This represents a correction factor with a value between 0 and 1; Indicates the first The radius of a Gaelic circle; from Start calculation ,when Stop when the first negative value is obtained, and estimate the number of source signals as follows: ; S4. Use blind source separation method to separate multiple gesture data; In step S4, a feature matrix joint approximate diagonalization algorithm is used for blind source separation, as detailed below: S41. Establish a signal model; The radar array signal receiving model is as follows: ,in, Depend on Composed of individual source signals, Depend on It consists of signals received from sensor channels. express 3D confusion coefficient matrix; ； ； Where T represents the transpose of the matrix; S42. Preprocessing received data; Observe the received data Centralization and whitening processes are performed to remove correlations and redundancy between source signals, resulting in preprocessed data. : ； in, Represents the whitening matrix. This represents the confusion matrix after whitening. S43. Construct a fourth-order cumulant matrix using the whitened data. ; S44, For the fourth-order cumulant matrix Perform eigenvalue decomposition to obtain the matrix. Estimate ; S45. Reconstruct the signal using the above formula. The results of blind source separation were obtained. H represents conjugate transpose; S5. Design a classification model, input gesture signal data or manually extracted features, and obtain gesture recognition results; The gesture signal data or manually extracted features are obtained from any gesture signal among the multiple gestures obtained in step S4.

2. The millimeter-wave radar multi-person gesture recognition method according to claim 1, characterized in that, The specific steps of S1 are as follows: S11. Design gestures and actions; Considering two typical multi-person gesture recognition application scenarios, namely smart cockpit and smart home, the gesture actions are designed as follows: (1) Raise: Within the radar detection range, the palm naturally rises, with a displacement of more than 10cm; (2) Pressing down: Within the radar detection range, press down naturally with your palm, with a displacement of more than 10cm; (3) Left swing: Within the radar detection range, the palm and forearm naturally swing from right to left, with a displacement of more than 10cm; (4) Right swing: Within the radar detection range, the palm and forearm naturally swing from left to right, with a displacement of more than 10cm; (5) Double tap: Within the radar detection range, with the arm still, tap up and down twice with the palm or fingers; (6) Thumb flicking: Within the radar detection range, the palm is still and the four fingers are bent, and the thumb flicks up and down along the index finger; S12. Determine the parameters of the millimeter-wave radar system.

3. The millimeter-wave radar multi-person gesture recognition method according to claim 1, characterized in that, Step S2 is as follows: S21. Radar signal transmission and reception; The radar transmitter transmits a sawtooth wave linear frequency modulated signal. : ； in, As a time variable, the signal, after being reflected by the gesture, undergoes a time delay. The FMCW radar receiver uses a mixer to obtain the frequency difference between the transmitted and received signals, which are received by the receiving antenna. , represented as: ； use replace Simultaneous delay equal ,in, Indicates the relative distance between the target and the radar. Represents the radial velocity of motion and radar. Representing the speed of light, after approximation, the intermediate frequency signal is represented as: ； The intermediate frequency signals received by all receiving channels pass through filters and ADC converters in sequence; S22. Moving target display filtering preprocessing; Using an exponentially weighted average MTI filtering method: ； ； in, Indicates the received number The intermediate frequency echo signal This represents the filtered signal. Let represent the smoothing factor. From the above equation, we can obtain the non-recursive formula for background clutter: ； Among them, the moving target display filter, smoothing factor The smaller the value, the stronger the clutter suppression capability.

4. The millimeter-wave radar multi-person gesture recognition method according to claim 1, characterized in that, Step S5 is as follows: S51. Determine the model input and extract gesture motion features; For any gesture signal from the multi-person gestures obtained in step S4, end-to-end recognition is performed using the raw data, or one or more features of the gesture are extracted for recognition. The features of the gesture mainly include distance-Doppler features, distance-time features, Doppler-time features, angle-time features, etc. Distance information extraction: The intermediate frequency signal frequency is obtained by performing a fast Fourier transform on a single pulse. The distance information of the gesture within the current single pulse time is obtained, and the distance information of each pulse is integrated to obtain the overall distance change map of a single gesture. The formula is: ； in, Represents the speed of light; Doppler information extraction: After obtaining distance information from a single pulse, a Fourier transform is performed again in the slow time dimension to obtain the Doppler frequency. To represent the speed information of gestures; short-time Fourier transform or wavelet transform are used to obtain the speed change graph of dynamic gestures: ； in, Indicates wavelength; Angle information extraction: When using multiple receiving antennas to receive gesture echoes, the angle information of the gesture echoes is obtained by utilizing the phase information between different receiving antennas. The MUSIC algorithm is used to estimate the dynamic angle change of the gesture. ; S52. Design a classification model to obtain gesture recognition results; Based on the model input determined in step S51, a radar gesture recognition classification model is designed to obtain the gesture recognition result. The classification model design includes one of the following: SVM, KNN, Random Forest, Hidden Markov, RNN, LSTM, CNN, and Transformer.

Images