Radar gesture recognition method and apparatus, electronic device, and medium

Abstract

The application provides a radar gesture recognition method and device, electronic equipment and medium, comprising: acquiring a target radar signal collected by a radar; performing standardization processing on the target radar signal to obtain standardized data; performing dimension reduction processing on the standardized data to obtain principal component data; performing dimension increase processing on the principal component data to obtain detection data; acquiring a weight value and an intercept value corresponding to the detection data, the weight value and the intercept value being used to represent the relevance between the detection data and a gesture; based on the detection data and the weight value and the intercept value corresponding to the detection data, a gesture category probability of the target radar signal is calculated to determine a gesture category corresponding to the target radar signal. Through the radar signal collected by the radar, the angle information is not required, and the hardware requirement and design cost of the radar are greatly reduced. Through the dimension reduction processing on the radar signal, the dimension of the feature information of the radar signal is greatly reduced, so that the gesture recognition technology can be implemented on a more lightweight device.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically to radar gesture recognition methods, devices, electronic equipment, and media. Background Technology

[0002] Gesture recognition based on millimeter-wave radar utilizes electromagnetic waves to acquire the motion characteristics of human dynamic gestures, enabling automatic gesture recognition. As a non-contact recognition method, it can achieve simple and efficient information exchange. High accuracy is key to gesture recognition research. Existing gesture detection schemes use radar to provide information such as the target's azimuth, elevation, velocity, and distance to train algorithm models, but this leads to problems such as excessive radar cost, larger size, and higher power consumption.

[0003] In related technologies, increasing the number of virtual channels to obtain the elevation angle information of the target under test requires multiple antennas to receive echo signals in order to obtain the azimuth angle information of the target under test, and the radar needs to have more memory and higher computing power to process multi-dimensional information. All of these have drawbacks such as high cost and high complexity. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the defect of excessively high radar cost required for high accuracy recognition in the prior art, thereby providing a radar gesture recognition method, device, electronic device and medium.

[0005] In conjunction with the first aspect, the present invention provides a radar gesture recognition method, the method comprising:

[0006] Acquire target radar signals collected by radar;

[0007] The target radar signal is standardized to obtain standardized data;

[0008] The standardized data is then subjected to dimensionality reduction processing to obtain principal component data;

[0009] The principal component data is then subjected to dimensionality upscaling to obtain the detection data;

[0010] Obtain the weight value and intercept value corresponding to the detection data, and the weight value and intercept value are used to characterize the correlation between the detection data and the gesture;

[0011] Based on the detection data and the corresponding weight and intercept values, the probability of the gesture category of the target radar signal is calculated, and the gesture category corresponding to the target radar signal is determined.

[0012] In this approach, radar signals acquired via radar do not require angle information, significantly reducing the hardware requirements and design costs for the radar. By performing dimensionality reduction processing on the radar signals to obtain the principal component data, the dimensionality of the radar signal's feature information is greatly reduced, lowering the demand for radar computing power and enabling gesture recognition technology to be implemented on more lightweight devices. Dimensionality enhancement processing improves the fitting ability of gesture recognition, thereby significantly improving the accuracy of gesture recognition and providing reliable support for gesture recognition in more complex scenarios.

[0013] In conjunction with the first aspect, in the first embodiment of the first aspect, before performing standardization, dimensionality reduction, and dimensionality increase processing on the target radar signal to obtain detection data, the method further includes: screening the target radar signal.

[0014] In conjunction with the first embodiment of the first aspect, in the second embodiment of the first aspect, the filtering of the target radar signal includes:

[0015] The target radar signal is frequency domain transformed to obtain the range Doppler image;

[0016] Based on the range Doppler image, the radar point cloud data of the target with the highest echo energy value is selected.

[0017] In conjunction with the first aspect, in the third embodiment of the first aspect, the dimensionality reduction processing of the standardized data to obtain principal component data includes:

[0018] The standardized data is subjected to zero-mean processing to obtain the first input matrix;

[0019] Based on the first input matrix, calculate the transpose matrix of the first input matrix, and based on the first input matrix and the transpose matrix, calculate the covariance matrix;

[0020] Based on the covariance matrix, the eigenvalues ​​and eigenvectors of the covariance matrix are calculated.

[0021] Based on the eigenvalues ​​and eigenvectors, the principal component data is calculated.

[0022] In conjunction with the first aspect, in the fourth embodiment of the first aspect, the step of performing dimensionality upscaling on the principal component data to obtain detection data includes:

[0023] The principal component data is subjected to polynomial feature upscaling to obtain the detection data.

[0024] In conjunction with the first aspect, in the fifth embodiment of the first aspect, obtaining the weight value and intercept value corresponding to the detection data includes:

[0025] Acquire training radar signals collected by radar, the training radar signals including several frames of radar signals containing a complete gesture;

[0026] Gesture annotations are performed on the training radar signals to obtain a gesture annotation dataset;

[0027] The gesture annotation dataset is preprocessed to obtain the training set;

[0028] Based on the training set, the initial gesture recognition model is trained to obtain the target gesture recognition model;

[0029] The detection data is input into the target gesture model to obtain the weight value and intercept value corresponding to the detection data.

[0030] In conjunction with the fifth embodiment of the first aspect, in the sixth embodiment of the first aspect, training the initial gesture recognition model based on the training set to obtain the target gesture recognition model includes:

[0031] The training set is input into the initial gesture recognition model to calculate the forward propagation value;

[0032] Based on the error between the forward propagation value and the corresponding gesture annotation in the training set, the initial gesture recognition model is trained to obtain the target gesture recognition model.

[0033] In a second aspect, the present invention also provides a radar gesture recognition device, the device comprising:

[0034] The first acquisition unit is used to acquire the target radar signal collected by the radar.

[0035] A standardization unit is used to standardize the target radar signal to obtain standardized data;

[0036] The dimensionality reduction unit is used to perform dimensionality reduction processing on the standardized data to obtain principal component data;

[0037] The dimension-upgrading unit is used to perform dimension-upgrading processing on the principal component data to obtain detection data;

[0038] The second acquisition unit is used to acquire the weight value and intercept value corresponding to the detection data, wherein the weight value and intercept value are used to characterize the correlation between the detection data and the gesture;

[0039] The category calculation unit is used to calculate the gesture category probability of the target radar signal based on the detection data and the weight value and intercept value corresponding to the detection data, and to determine the gesture category corresponding to the target radar signal.

[0040] In conjunction with the second aspect, in the first embodiment of the second aspect, the apparatus further includes:

[0041] A filtering unit is used to filter the target radar signal.

[0042] In conjunction with the first embodiment of the second aspect, in the second embodiment of the second aspect, the screening unit includes:

[0043] The frequency domain transformation unit is used to perform frequency domain transformation on the target radar signal to obtain a range Doppler image.

[0044] The filtering subunit is used to filter out the target radar point cloud data with the largest echo energy value based on the range Doppler image.

[0045] In conjunction with the second aspect, in the third embodiment of the second aspect, the dimensionality reduction unit includes:

[0046] A zero-mean processing unit is used to perform zero-mean processing on the standardized data to obtain a first input matrix;

[0047] The first calculation unit is used to calculate the transpose matrix of the first input matrix based on the first input matrix, and to calculate the covariance matrix based on the first input matrix and the transpose matrix.

[0048] The second calculation unit is used to calculate the eigenvalues ​​and eigenvectors of the covariance matrix based on the covariance matrix.

[0049] The third calculation unit is used to calculate the principal component data based on the eigenvalues ​​and eigenvectors.

[0050] In conjunction with the second aspect, in the fourth embodiment of the second aspect, the dimensionality-upgrading unit includes:

[0051] The dimension-upgrading subunit is used to perform polynomial feature dimension upgrading on the principal component data to obtain detection data.

[0052] In conjunction with the second aspect, in the fifth embodiment of the second aspect, the second acquisition unit includes:

[0053] The second acquisition subunit is used to acquire training radar signals collected by the radar, wherein the training radar signals include several frames of radar signals containing a complete gesture.

[0054] An annotation unit is used to annotate the training radar signals with gestures to obtain a gesture annotation dataset.

[0055] The preprocessing unit is used to preprocess the gesture annotation dataset to obtain the training set;

[0056] The training unit is used to train the initial gesture recognition model based on the training set to obtain the target gesture recognition model.

[0057] The input unit is used to input the detection data into the target gesture model to obtain the weight value and intercept value corresponding to the detection data.

[0058] In conjunction with the fifth embodiment of the second aspect, in the sixth embodiment of the second aspect, the training unit includes:

[0059] The fourth calculation unit is used to input the training set into the initial gesture recognition model and calculate the forward propagation value;

[0060] The training subunit is used to train the initial gesture recognition model based on the error between the forward propagation value and the corresponding gesture annotation in the training set, so as to obtain the target gesture recognition model.

[0061] According to a third aspect, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the radar gesture recognition method of any one of the first aspects and its optional embodiments.

[0062] According to a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing the computer to perform the radar gesture recognition method of any one of the first aspects and its optional embodiments. Attached Figure Description

[0063] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0064] Figure 1 This is a flowchart of a radar gesture recognition method according to an exemplary embodiment.

[0065] Figure 2 This is a flowchart of a training method for a gesture recognition model according to an exemplary embodiment.

[0066] Figure 3 This is a schematic diagram of a radar gesture data acquisition module according to an exemplary embodiment.

[0067] Figure 4This is a schematic diagram of serial port data according to an exemplary embodiment.

[0068] Figure 5 This is a schematic diagram of the structure of a preprocessing module according to an exemplary embodiment.

[0069] Figure 6 This is a structural block diagram of a radar gesture recognition device according to an exemplary embodiment.

[0070] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0071] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] In related technologies, increasing the number of virtual channels to obtain the elevation angle information of the target under test requires multiple antennas to receive echo signals in order to obtain the azimuth angle information of the target under test, and the radar needs to have more memory and higher computing power to process multi-dimensional information. All of these have drawbacks such as high cost and high complexity.

[0073] To address the aforementioned issues, this invention provides a radar gesture recognition method applied to radar systems. In the following embodiments, the radar is used as the executing entity. The radar in this embodiment is suitable for scenarios where the radar identifies the specific type of gesture. The radar gesture recognition method provided by this invention utilizes radar signals acquired by the radar, eliminating the need for angle information and significantly reducing the hardware requirements and design costs for the radar. By performing dimensionality reduction processing on the radar signals to obtain the principal component data, the dimensionality of the radar signal's feature information is greatly reduced, lowering the computational power required for the radar and enabling gesture recognition technology to be implemented on more lightweight devices. Dimensionality enhancement processing improves the fitting ability of gesture recognition, thereby significantly improving the accuracy of gesture recognition and providing reliable support for gesture recognition in more complex scenarios.

[0074] Figure 1 This is a flowchart of a radar gesture recognition method according to an exemplary embodiment. Figure 1 As shown, the radar gesture recognition method includes the following steps S101 to S106.

[0075] In step S101, the target radar signal acquired by the radar is obtained.

[0076] In this embodiment of the invention, the target radar signal is a number of radar signals containing one or more complete gestures.

[0077] In step S102, the target radar signal is standardized to obtain standardized data.

[0078] In this embodiment of the invention, to facilitate accurate selection of target radar signals suitable for gesture recognition, the target radar signals need to be screened before standardization processing to obtain target radar signals for gesture recognition. This screening includes: performing frequency domain transformation on the target radar signals to obtain a range Doppler image; and based on the range Doppler image, selecting the target radar point cloud data with the highest echo energy value.

[0079] In one example, filtering the target radar signal may include: performing a 2DFFT transform on the target radar signal to obtain a range-Doppler image. Based on a custom velocity threshold and amplitude threshold, the target radar signal is filtered to obtain N target points. If there are multiple target points in a single frame of radar signal, the top N target points with the largest echo energy values ​​in the range-Doppler image are selected as target radar point cloud data. N target radar point cloud data are obtained for each frame of radar signal. Each radar point cloud data contains: energy value, range, and velocity data. Each frame of radar signal has N*3 dimensional features. The number of data frames processed each time is 15 frames, and the frame vector is 15*(N*3). Among them, the number of radar signal frames containing a complete gesture is 15, and the number of radar signal frames is not limited in this invention.

[0080] In one example, the normalization formula is:

[0081]

[0082] Where, x * For standardized data, x represents the target radar point cloud data, μ represents the mean of the target radar point cloud data, and σ represents the variance of the target radar point cloud data. The standardized data after processing is x. * It follows a normal distribution, i.e., μ = 0, σ = 1.

[0083] In step S103, the standardized data is subjected to dimensionality reduction processing to obtain principal component data.

[0084] In this embodiment of the invention, by performing dimensionality reduction processing on radar signals, principal component data of the radar signals are obtained, which significantly reduces the dimensionality of the feature information of the radar signals, lowers the demand for radar computing power, and enables gesture recognition technology to be implemented on more lightweight devices. Dimensionality reduction processing of standardized data to obtain principal component data includes: performing zero-mean processing on the standardized data to obtain a first input matrix; calculating the transpose matrix of the first input matrix based on the first input matrix; calculating the covariance matrix based on the first input matrix and the transpose matrix; calculating the eigenvalues ​​and eigenvectors of the covariance matrix based on the covariance matrix; and calculating the principal component data based on the eigenvalues ​​and eigenvectors.

[0085] In one example, dimensionality reduction can be achieved through principal component analysis (PCA). A linear transformation maps the original data to a new coordinate system, maximizing the variance at the first mapped coordinate. This first coordinate represents the first principal component, and so on, until the other principal components are obtained. Therefore, the proportion of each principal component's variance to the total variance is adjusted; a larger proportion indicates a more important principal component. Specifically, this involves zero-mean normalization of the standardized (N*3)*15 matrix. The mean of each standardized data point is calculated, and this mean is subtracted from each value in the matrix (i.e., the mean of that row) to obtain the first input matrix. The covariance matrix of the first input matrix is ​​then calculated using the following formula:

[0086]

[0087] Where c is the covariance matrix, x is the first input matrix, and m is the number of columns in the first input matrix. T This is the transpose of the first input matrix, which is obtained by swapping the row and column data of the first input matrix.

[0088] The eigenvalues ​​and eigenvectors of the covariance matrix are obtained using the methods for finding eigenvalues ​​and eigenvectors in linear algebra. The eigenvectors are then arranged in descending order of their corresponding eigenvalues, row by row, from top to bottom, to form a second matrix. The first K rows of this second matrix are then used to form matrix P, where the value of K depends on the desired dimensionality reduction. After obtaining matrix P, the final principal component data is calculated using the following formula:

[0089] y = Px

[0090] Here, y is a matrix composed of principal component data, and x is the first input matrix. After processing by the principal component analysis dimensionality reduction algorithm, the principal component data in the resulting matrix are orthogonal, which can effectively eliminate the mutual influence between target radar signal data. While increasing the nonlinearity of the radar signal data, it also reduces the dimensionality of the target radar signal data, making it easier to fit the radar signal data.

[0091] In step S104, the principal component data is subjected to dimensionality upscaling to obtain the detection data.

[0092] In this embodiment of the invention, the dimensionality increase processing can be performed using polynomial feature dimensionality increase. By using polynomial dimensionality increase to further process the principal component data, the data can be better fitted, improving the fitting ability of gesture recognition and thus significantly improving the accuracy of gesture recognition, providing reliable support for gesture recognition in more complex scenarios.

[0093] In one example, assuming the input data is a 1x2 matrix [a,b], using polynomial 2D dimension upscaling, this matrix will become [a,b,a²,b²,a*b], resulting in a 1x5 matrix. If the input data is a 1x3 matrix like [a,b,c], it will be expanded to [a,b,c,a²,a*b,a*c,b²,b*c,c²]. For multiple rows of data, each row will be expanded in the same way.

[0094] In step S105, the weight value and intercept value corresponding to the detection data are obtained.

[0095] In this embodiment of the invention, the weight value and the intercept value are used to characterize the correlation between the detection data and the gesture. They are calculated by a pre-trained gesture recognition model and are pre-stored in the radar, which can avoid training in the radar and thus reduce the requirements for radar computing power.

[0096] In step S106, based on the detection data and the corresponding weight and intercept values, the probability of the gesture category of the target radar signal is calculated, and the gesture category corresponding to the target radar signal is determined.

[0097] In one example, the process of determining the gesture category corresponding to the target radar signal may include: using a Sofrmax regression classifier, converting the weight parameters corresponding to the detection data into a matrix of 54 columns and 6 rows; calculating the probability of each category by multiplying the detection data by its corresponding weights, then adding the intercept value to obtain the data value corresponding to each category; comparing this data value with the sum of the data values ​​for each category to calculate the probability value of each category relative to the total data for all categories; and finally, determining the final category based on the maximum category probability, with the category number corresponding to the maximum probability being the final category.

[0098] Through the above embodiments, radar signals acquired by radar do not require angle information, significantly reducing the hardware requirements and design costs for radar. By performing dimensionality reduction processing on the radar signals to obtain the principal component data, the dimensionality of the radar signal's feature information is greatly reduced, lowering the demand for radar computing power and enabling gesture recognition technology to be implemented on more lightweight devices. By performing dimensionality enhancement processing, the fitting ability of gesture recognition is improved, thereby significantly improving the accuracy of gesture recognition and providing reliable support for gesture recognition in more complex scenarios.

[0099] The following examples will specifically illustrate the process of training a gesture recognition model to obtain the weights and intercepts corresponding to the detection data.

[0100] Figure 2 This is a flowchart illustrating a training method for a gesture recognition model based on an exemplary embodiment. Figure 2 As shown, the training method for the gesture recognition model includes the following steps.

[0101] In step S201, the training radar signal collected by the radar is acquired.

[0102] In this embodiment of the invention, the training radar signal includes several frames of radar signal containing a complete gesture. The training radar signal is used to train the gesture recognition model to obtain the target gesture recognition model.

[0103] In one example, Figure 3 This is a schematic diagram of a radar gesture data acquisition module according to an exemplary embodiment. The radar gesture data acquisition module consists of a millimeter-wave radar 11, a 1T1R radio frequency antenna 111, a chip 112 for processing radio frequency signals, a microcontroller 113, and a USB-to-serial converter connected to a data acquisition host computer 14. The millimeter-wave radar 11 and the data acquisition host computer 14 are connected via serial communication.

[0104] The process of acquiring training radar signals may include: 1. Debugging hardware equipment: Before acquiring data, the corresponding hardware equipment needs to be debugged to ensure that the host computer 14 and the radar 11 can communicate with each other.

[0105] 2. Determine the data acquisition conditions: Data acquisition needs to be carried out separately according to different gesture data. Multiple people need to cooperate during acquisition. One person controls the host computer software, while the others take turns performing the corresponding gestures in front of radar 11. When acquiring a specific gesture from a particular person, that person needs to face radar 11 at the required angle, distance, and speed. At this point, they must wait for instructions from the host computer operator before starting.

[0106] 3. Start Data Acquisition: After the host computer operator is ready, wait for the data acquisition personnel to give the start command. After the data acquisition personnel give the start command, the host computer operator clicks the start button on the host computer 14, and the host computer 14 sends the radar operation data to the radar 11 via serial communication.

[0107] 4. Data Acquisition: Radar 11 will send radar signals. If a person is detected within the radar signal range, the radar signal will be reflected back to Radar 11 at the location of the person. After analyzing the reflected and transmitted data internally, Radar 11 will transmit the corresponding data to the host computer 14 via serial communication. Upon receiving the data sent by Radar 11, the host computer 14 will automatically create a file and save the data reflected by Radar 11 in the file.

[0108] 5. Data Acquisition Completion: On the data acquisition end, before the gesture is completed, the host computer operator is notified via voice that the gesture is about to end. At this time, the host computer operator will press the end button, causing the host computer 14 to send an end signal to the radar 11, and the radar 11 will stop working. The data saved at this time will be the radar data of the complete gesture.

[0109] 6. Collect data under different conditions: During the data collection process, the data collector needs to make corresponding hand gestures facing the radar 11 at different angles, distances, and directions. Since different people have different habits when making the same gesture, it is also necessary to collect the hand gestures of different people.

[0110] 7. Data Processing: After data collection is completed, the data is packaged and then proceeds to the data annotation step. Figure 4 This is a schematic diagram of serial port data according to an exemplary embodiment. A frame of serial port data consists of a start bit, data length, radar point cloud data, parity bit, and stop bit. The radar point cloud data consists of N points, each containing three features: velocity, distance, and echo energy.

[0111] In step S202, gesture annotations are performed on the training radar signals to obtain a gesture annotation dataset.

[0112] In this embodiment of the invention, since different people have different habits when making gestures, in order to ensure that the trained target gesture recognition model has versatility, it is necessary to annotate the different gestures made by different people and use the gesture annotation dataset for training.

[0113] In one example, the data annotation process may include: classifying the data according to different gestures, and then further classifying the data for each gesture according to different data collectors. Hundreds to thousands of data points for each gesture were collected from each person each time, depending on the needs. The number of data lines for each gesture was determined by the duration of the action. Typically, a single collected gesture's txt file is about 3kb in size, containing approximately 150 lines of data. Each line contains the distance, energy, and velocity values ​​for N consecutive radar point clouds.

[0114] According to the test, a total of 15 lines of data are collected to generate one valid data point. Using a queue, a fixed window of 15 lines slides downwards in increments of 1, with each 15 lines constituting a valid data point. Each data point generated in this way is considered valid if no more than 20% of its values ​​are zero. One line is popped out at a time. The generated valid data is written to a new file. Then, the valid data generated by different actions are named according to the action and the name of the person performing the action. After processing all the valid data, each action is labeled according to its different names. A sequence number is manually assigned to each action, and the corresponding valid data file for that action is located. Each line of data in the file for that gesture action is labeled with the sequence number of that action.

[0115] In step S203, the gesture annotation dataset is preprocessed to obtain the training set.

[0116] In this embodiment of the invention, the preprocessing process includes standardization, dimensionality reduction and dimensionality increase. For specific methods, please refer to steps S102 to S104, which will not be repeated here.

[0117] In one example, Figure 5This is a schematic diagram of a preprocessing module proposed according to an exemplary embodiment. The radar data preprocessing module 12 includes a data conversion unit 121 and a principal component analysis unit 123. The data conversion unit groups 15 frames of data into a first-in-first-out queue and uses data standardization 122 to standardize the mean and standard deviation of each frame. Through this processing, the frame data conforms to a normal distribution (i.e., μ = 0, σ = 1). The principal component analysis unit uses a dimensionality reduction method to convert high-dimensional data into low-dimensional data, reducing the 15*27 dimensionality of the data. The multinomial feature dimensionality enhancement unit 124 converts the linear relationship between the target point cloud data and the gesture result into a non-linear relationship and increases the data dimension. The gesture detection module 13 uses a softmax regression classifier to perform gesture classification and detection on the 15x54 matrix data output by the data preprocessing module.

[0118] In step S204, the initial gesture recognition model is trained based on the training set to obtain the target gesture recognition model.

[0119] In this embodiment of the invention, training an initial gesture recognition model based on a training set to obtain a target gesture recognition model includes: inputting the training set into the initial gesture recognition model and calculating the forward propagation value; and training the initial gesture recognition model based on the error between the forward propagation value and the corresponding gesture annotation in the training set to obtain the target gesture recognition model.

[0120] In one example, the training process may include: inputting the training set into an initial gesture recognition model, and performing forward propagation based on the input data, as shown in the following formula:

[0121]

[0122] Where x is the training set, w T Let be the initial model data value, b be the initial intercept value, and x1 be the calculated value obtained by multiplying the training set x with the initial model data and then adding it to the initial intercept value. The calculated x1 is then processed using softmax to obtain the probability value A for each class. 0 a represents the probability value of the first category. n This represents the probability value of the nth category.

[0123] The formula for calculating softmax is:

[0124]

[0125] Among them, e i ∑ represents the calculated numerical value of the i-th category. j e jThis represents the total numerical value calculated for all categories. After forward propagation, the forward propagation value A is used to calculate the error between the forward propagation value and the corresponding gesture annotation in the training set. The calculation formula is as follows:

[0126]

[0127] Where m is the total number of categories, y i Let y be the true values ​​for the i categories. For example, if the true category is 1, then y = ... 1 =1, y i =0, where i is any value other than 1, ranging from the total number of categories, and J is the error between the forward propagation value and the corresponding gesture annotation in the training set. Backpropagation is used to adjust the initial model values; the backpropagation formula is shown below:

[0128]

[0129] Where w is the initial weight value, and w1 is the updated weight value. This indicates that the partial derivative of the obtained error value with respect to w is taken, b is the initial weight value, and b1 is the updated weight value. To obtain the partial derivative of the obtained error value with respect to b, specifically, It is obtained through chain rule differentiation, and the chain rule differentiation formula is as follows:

[0130]

[0131] The calculation formula is:

[0132] In step S205, the detection data is input into the target gesture model to obtain the weight value and intercept value corresponding to the detection data.

[0133] In one example, the model training module uses logistic regression for multi-class classification. This invention adopts a one-to-one (OVO) strategy, which pairs the N gesture categories labeled in the training data to obtain N(N-1) / 2 classifiers. The training hyperparameters are: 20,000 iterations, lbfgs for the iteration optimization loss function, 1e-4 for the stopping error standard, L2 regularization with a regularization coefficient of 2.5, and the weight and intercept parameters are output and saved after the iteration count ends or the error standard is reached.

[0134] In one implementation scenario, the classification and detection module uses a Sofrmax regression classifier to determine the gesture category corresponding to the target radar signal. The steps are as follows: 1. Process the weight parameter dimension. The model's weight parameter is the number of gesture categories * 54 (54 is the final processed data obtained after principal component analysis dimensionality reduction and then multinomial dimensionality increase) in rows and columns. The weight data is converted into a matrix of 54 rows and columns representing the number of categories. 2. Calculate the probability of each category. The input gesture data is multiplied by the weight data, and the intercept value is added to obtain the data value corresponding to each category. This data value is then compared with the sum of the data values ​​of each category to calculate the probability value of each category relative to the total data of all categories. This probability value is the probability value of each category. 3. The final category is obtained based on the highest confidence rate. The highest confidence rate is the maximum probability value of the obtained category, and the category number corresponding to the maximum probability value is the calculated final category.

[0135] Through the above embodiments, by controlling the number of target point clouds output in each frame, the computational power requirements of the chip can be reduced. After preprocessing and principal component analysis of the detection data, the weight and intercept parameters obtained by the model algorithm only occupy a small amount of space, reducing the chip flash capacity requirements and lowering chip costs. By employing multi-class logistic regression and softmax regression classifiers, using L2 regularization to prevent overfitting, and using data dimensionality reduction based on principal component analysis to reduce the impact of noise and interference, highly robust gesture recognition capabilities are achieved. Stable gesture detection can be performed in complex environments, including changes in lighting and background interference. By using low-power logistic regression and softmax regression classifiers, gesture recognition can be achieved with low-cost radar and low-power chips.

[0136] Based on the same inventive concept, the present invention also provides a radar gesture recognition device.

[0137] Figure 6 This is a structural block diagram of a radar gesture recognition device according to an exemplary embodiment. Figure 6 As shown, the radar gesture recognition device includes a first acquisition unit 601, a standardization unit 602, a dimensionality reduction unit 603, a dimensionality increase unit 604, a second acquisition unit 605, and a category calculation unit 606.

[0138] The first acquisition unit 601 is used to acquire the target radar signal collected by the radar.

[0139] The standardization unit 602 is used to standardize the target radar signal to obtain standardized data;

[0140] Dimensionality reduction unit 603 is used to reduce the dimensionality of standardized data to obtain principal component data;

[0141] The dimension-upgrading unit 604 is used to perform dimension-upgrading processing on the principal component data to obtain the detection data;

[0142] The second acquisition unit 605 is used to acquire the weight value and intercept value corresponding to the detection data. The weight value and intercept value are used to characterize the correlation between the detection data and the gesture.

[0143] The category calculation unit 606 is used to calculate the probability of the gesture category of the target radar signal based on the detection data and the corresponding weight and intercept values, and to determine the gesture category corresponding to the target radar signal.

[0144] In one embodiment, the radar gesture recognition device provided by the present invention further includes a filtering unit for filtering target radar signals.

[0145] In another embodiment, the filtering unit includes: a frequency domain transformation unit for performing frequency domain transformation on the target radar signal to obtain a range Doppler map; and a filtering subunit for filtering the target radar point cloud data with the largest echo energy value based on the range Doppler map.

[0146] In another embodiment, the dimensionality reduction unit 603 includes: a zero-mean processing unit for performing zero-mean processing on standardized data to obtain a first input matrix; a first calculation unit for calculating the transpose matrix of the first input matrix based on the first input matrix, and calculating the covariance matrix based on the first input matrix and the transpose matrix; a second calculation unit for calculating the eigenvalues ​​and eigenvectors of the covariance matrix based on the covariance matrix; and a third calculation unit for calculating the principal component data based on the eigenvalues ​​and eigenvectors.

[0147] In another embodiment, the dimensionality-upgrading unit 604 includes: a dimensionality-upgrading subunit, used to perform polynomial feature dimensionality-upgrading on the principal component data to obtain detection data.

[0148] In another embodiment, the second acquisition unit 605 includes: a second acquisition subunit, used to acquire training radar signals collected by the radar, the training radar signals including several frames of radar signals containing a complete gesture; an annotation unit, used to annotate the training radar signals with gestures to obtain a gesture annotation dataset; a preprocessing unit, used to preprocess the gesture annotation dataset to obtain a training set; a training unit, used to train an initial gesture recognition model based on the training set to obtain a target gesture recognition model; and an input unit, used to input the detection data into the target gesture model to obtain the weight value and intercept value corresponding to the detection data.

[0149] In another embodiment, the training unit includes: a fourth calculation unit, used to input the training set into the initial gesture recognition model and calculate the forward propagation value; and a training subunit, used to train the initial gesture recognition model based on the error between the forward propagation value and the corresponding gesture annotation in the training set to obtain the target gesture recognition model.

[0150] The specific limitations and beneficial effects of the aforementioned radar gesture recognition device can be found in the limitations of the radar gesture recognition method described above, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the electronic device, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0151] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment. For example... Figure 7 As shown, the device includes one or more processors 710 and a memory 720, the memory 720 including persistent memory, volatile memory, and a hard disk. Figure 7 Taking a processor 710 as an example, the device may also include an input device 730 and an output device 740.

[0152] The processor 710, memory 720, input device 730, and output device 740 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0153] Processor 710 can be a Central Processing Unit (CPU). Processor 710 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0154] The memory 720, as a non-transitory computer-readable storage medium, includes persistent memory, volatile memory, and a hard disk. It can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the radar gesture recognition method in this embodiment. The processor 710 executes various server functions and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 720, thereby implementing any of the aforementioned radar gesture recognition methods.

[0155] The memory 720 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data that is needed and required. Furthermore, the memory 720 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 720 may optionally include memory remotely located relative to the processor 710, and these remote memories can be connected to the data processing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0156] The input device 730 can receive input digital or character information, and generate key signal inputs related to user settings and function control. The output device 740 may include a display device such as a display screen.

[0157] One or more modules are stored in memory 720, and when executed by one or more processors 710, they perform actions such as... Figures 1-5 The method shown.

[0158] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in [reference 1]. Figures 1-5 The relevant descriptions in the illustrated embodiments.

[0159] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the authentication method in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0160] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A radar gesture recognition method, characterized in that, The method includes: Acquire target radar signals collected by radar; The target radar signal is filtered; The target radar signal is standardized to obtain standardized data; The standardized data is then subjected to dimensionality reduction processing to obtain principal component data; The principal component data is then subjected to dimensionality upscaling to obtain the detection data; Obtain the weight value and intercept value corresponding to the detection data, and the weight value and intercept value are used to characterize the correlation between the detection data and the gesture; Based on the detection data and the corresponding weight and intercept values, the probability of the gesture category of the target radar signal is calculated, and the gesture category corresponding to the target radar signal is determined. The step of performing dimensionality upscaling on the principal component data to obtain detection data includes: The principal component data is subjected to polynomial feature upscaling to obtain the detection data; The target radar signal is filtered, including: The target radar signal is subjected to frequency domain transformation to obtain the range Doppler image; Based on the range Doppler image, the radar point cloud data of the target with the highest echo energy value is selected; Based on custom velocity and amplitude thresholds, N target points are obtained by filtering the target radar signal. When there are multiple target points in a single frame of radar signal, the top N target points with the highest echo energy values ​​in the range Doppler image are selected as target radar point cloud data. N target radar point cloud data are obtained for each frame of radar signal. Each radar point cloud data contains: energy value, range, and velocity data. Each frame of radar signal is N 3D features, processing 15 data frames at a time, with a frame vector of 15. (N) 3); The step of obtaining the weight value and intercept value corresponding to the detection data includes: Acquire training radar signals collected by radar, the training radar signals including several frames of radar signals containing a complete gesture; Gesture annotations are performed on the training radar signals to obtain a gesture annotation dataset; The gesture annotation dataset is preprocessed to obtain the training set; Based on the training set, the initial gesture recognition model is trained to obtain the target gesture recognition model; The detection data is input into the target gesture recognition model to obtain the weight value and intercept value corresponding to the detection data; The training process includes: inputting the training set into the initial gesture recognition model, and performing forward propagation based on the input data. The forward propagation formula is shown below: in, x For the training set, Here, b represents the initial model data value, and b represents the initial intercept value. x1 For training set x After multiplying with the initial model value, the calculated value is obtained by adding it to the initial intercept value. The calculated value is then... x1 go through softmax The probability value A for each category is calculated. This represents the probability value of the first category. Indicates the first n The probability values ​​of each category; softmax The calculation formula is: in, Represents the calculated first... i The numerical value of each category This represents the calculation of the total numerical value of all categories. After forward propagation, the forward propagation value A is obtained. The error between the forward propagation value and the corresponding gesture annotation in the training set is then calculated. The calculation formula is as follows: Where m is the total number of categories. for i The true value of each category i The range is any value other than 1, up to the total number of categories. J is the error between the forward propagation value and the corresponding gesture annotation in the training set. The initial model values ​​are adjusted through backpropagation. The backpropagation formula is shown below: in, To initialize the weight values, For the updated weight values, This indicates that the obtained error value is paired with... Find the partial derivative. To initialize the weight values, For the updated weight values, To pair the obtained error values Find the partial derivative. It is obtained through chain rule differentiation, and the chain rule differentiation formula is as follows: The calculation formula is: .

2. The method according to claim 1, characterized in that, The dimensionality reduction process performed on the standardized data to obtain principal component data includes: The standardized data is subjected to zero-mean processing to obtain the first input matrix; Based on the first input matrix, the transpose matrix of the first input matrix is ​​calculated, and the covariance matrix is ​​calculated based on the first input matrix and the transpose matrix. Based on the covariance matrix, the eigenvalues ​​and eigenvectors of the covariance matrix are calculated. Based on the eigenvalues ​​and eigenvectors, the principal component data is calculated.

3. The method according to claim 1, characterized in that, The step of training the initial gesture recognition model based on the training set to obtain the target gesture recognition model includes: The training set is input into the initial gesture recognition model to calculate the forward propagation value; Based on the error between the forward propagation value and the corresponding gesture annotation in the training set, the initial gesture recognition model is trained to obtain the target gesture recognition model.

4. A radar gesture recognition device, characterized in that, The device includes: The first acquisition unit is used to acquire the target radar signal collected by the radar. The filtering unit is used to filter target radar signals; A standardization unit is used to standardize the target radar signal to obtain standardized data; The dimensionality reduction unit is used to perform dimensionality reduction processing on the standardized data to obtain principal component data; The dimension-upgrading unit is used to perform dimension-upgrading processing on the principal component data to obtain detection data; The second acquisition unit is used to acquire the weight value and intercept value corresponding to the detection data, wherein the weight value and intercept value are used to characterize the correlation between the detection data and the gesture; The category calculation unit is used to calculate the gesture category probability of the target radar signal based on the detection data and the weight value and intercept value corresponding to the detection data, and to determine the gesture category corresponding to the target radar signal. The dimension-upgrading unit includes: a dimension-upgrading subunit, used to perform multinomial feature dimension-upgrading on the principal component data to obtain the detection data; The filtering unit includes: a frequency domain transformation unit, used to perform frequency domain transformation on the target radar signal to obtain a range Doppler map; and a filtering subunit, used to filter the target radar point cloud data with the largest echo energy value based on the range Doppler map. Based on custom velocity and amplitude thresholds, N target points are obtained by filtering the target radar signal. When there are multiple target points in a single frame of radar signal, the top N target points with the highest echo energy values ​​in the range Doppler image are selected as target radar point cloud data. N target radar point cloud data are obtained for each frame of radar signal. Each radar point cloud data contains: energy value, range, and velocity data. Each frame of radar signal is N 3D features, processing 15 data frames at a time, with a frame vector of 15. (N) 3); The second acquisition unit includes: a second acquisition subunit for acquiring training radar signals collected by the radar, the training radar signals including several frames of radar signals containing a complete gesture; an annotation unit for annotating the training radar signals to obtain a gesture annotation dataset; a preprocessing unit for preprocessing the gesture annotation dataset to obtain a training set; a training unit for training an initial gesture recognition model based on the training set to obtain a target gesture recognition model; and an input unit for inputting detection data into the target gesture recognition model to obtain the weight values ​​and intercept values ​​corresponding to the detection data. The training process includes: inputting the training set into the initial gesture recognition model, and performing forward propagation based on the input data. The forward propagation formula is shown below: in, x For the training set, Here, b represents the initial model data value, and b represents the initial intercept value. x1 For training set x After multiplying with the initial model value, the calculated value is obtained by adding it to the initial intercept value. The calculated value is then... x1 go through softmax The probability value A for each category is calculated. This represents the probability value of the first category. Indicates the first n The probability values ​​of each category; softmax The calculation formula is: in, Represents the calculated first... i The numerical value of each category This represents the calculation of the total numerical value of all categories. After forward propagation, the forward propagation value A is obtained. The error between the forward propagation value and the corresponding gesture annotation in the training set is then calculated. The calculation formula is as follows: Where m is the total number of categories. for i The true value of each category i The range is any value other than 1, up to the total number of categories. J is the error between the forward propagation value and the corresponding gesture annotation in the training set. The initial model values ​​are adjusted through backpropagation. The backpropagation formula is shown below: in, To initialize the weight values, For the updated weight values, This indicates that the obtained error value is paired with... Find the partial derivative. To initialize the weight values, For the updated weight values, To pair the obtained error values Find the partial derivative. It is obtained through chain rule differentiation, and the chain rule differentiation formula is as follows: The calculation formula is: .

5. An electronic device, characterized in that, The method includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the radar gesture recognition method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the radar gesture recognition method according to any one of claims 1-3.

Images