A digital action recognition method based on one-transmitting and two-receiving millimeter wave radar

Abstract

The application discloses a digital action recognition method based on a one-transmitting and two-receiving millimeter wave radar, which comprises the following steps: collecting radar original data of a digital action, processing each frame of the radar original data to obtain a target gravity center, calculating information statistics of multiple frames of the target gravity center to obtain a target feature, inputting the target feature into a machine learning model as a basis for classification and recognition of the digital action, and inputting statistical information for classification and recognition of the digital action. The application has the advantages of simple implementation, small calculation resource consumption, and the ability to recognize the digital action under the condition that the hardware of the millimeter wave radar is a one-transmitting and two-receiving antenna, and is not limited by light in the environment, and can accurately recognize the action.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar. Background Technology

[0002] Existing methods for recognizing hand gestures as numbers can be divided into two categories: visual and radar. Visual methods primarily acquire images of hand gestures through optical devices. By comparing the number of fingers, skin color, and surrounding environment in the image, a neural network model is used to extract features representing different numbers in the image, thus achieving a static recognition. Currently, there are few radar applications that can achieve digit action recognition, but research tends to use recognition methods based on Range-Doppler Map (RD) data and deep learning. By distinguishing different RD maps of actions, digit action recognition can be achieved.

[0003] For vision-based recognition, the technology is highly susceptible to interference from ambient light and is prone to exposing personal information; deep learning-based implementation requires high computing power and is difficult to implement in a SOC (System on Chip) environment. Summary of the Invention

[0004] The purpose of this invention is to provide a digital motion recognition method based on a one-transmitter-two-receiver millimeter-wave radar, which has low computational resource overhead and is not limited by ambient light, and can more accurately identify images.

[0005] To achieve the above objectives, the present invention provides a digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar, comprising:

[0006] Collect raw radar data of digital motion, process each frame of raw radar data, and obtain the target's center of gravity;

[0007] Calculate the information statistics of the target centroid across multiple frames to obtain the target features;

[0008] The target features are input into a machine learning model as a basis for classifying and recognizing digital actions;

[0009] Input statistical information to classify and recognize digital actions.

[0010] Furthermore, the classification and recognition of digital actions includes the recognition of digits 0-3.

[0011] Furthermore, the step of processing each frame of raw radar data to obtain the target centroid includes:

[0012] Generate a single-frame radar data matrix in units of data frames;

[0013] Perform a two-dimensional fast Fourier transform on the radar data matrix to obtain range-Doppler image data;

[0014] Peak extraction is performed on the distance-Doppler map data to obtain multiple target points;

[0015] The target centroid is obtained from multiple target points.

[0016] Furthermore, after performing a two-dimensional fast Fourier transform on the radar data matrix to obtain range-Doppler image data, before extracting the peak values ​​from the range-Doppler image data, the method further includes: removing static targets from the range-Doppler image data based on the radar Doppler image velocity measurement principle.

[0017] Furthermore, obtaining the target centroid based on multiple target points includes:

[0018] The target centroid is obtained by performing a weighted calculation on multiple target points with respect to amplitude.

[0019] Furthermore, the step of calculating the statistical information of the target centroid across multiple frames to obtain target features includes: calculating the cumulative changes in the distance, velocity, and phase of the target centroid across multiple frames as the target features.

[0020] Furthermore, the calculation of the cumulative changes in distance, velocity, and phase of the target centroid across multiple frames includes: sampling the changes using a fixed-length sliding window; determining that the data segment is valid when the distance, velocity, and phase of multiple consecutive frames are not zero and appear in the middle of the sliding window; treating frames in the segment where all changes are zero as invalid frames and clearing them; and arranging the remaining data in chronological order for feature data extraction and calculation.

[0021] Furthermore, the feature data extraction and calculation includes:

[0022] (a) The proportion of frames with positive velocity values ​​and the proportion of frames with negative velocity values ​​in the valid data are used to determine the number 1.

[0023] (b) In the valid data, the point closest to the radar in the vertical direction is used as the dividing point, and the velocity sum of the second half of the valid frames is used to determine the digit 0.

[0024] (c) In the valid data, the phase sign of the data in the last few frames is used to distinguish the numbers 2 and 3.

[0025] Furthermore, the target features are input into a machine learning model to classify and recognize digital actions, including:

[0026] Machine learning models classify and recognize digital actions based on whether the input statistical information matches the target features.

[0027] Furthermore, machine learning models distinguish the characteristics of different digital actions based on the differences in statistical information of different digital actions in the training data;

[0028] By manually analyzing statistical information, we can determine the differences between various digital action features to accommodate the characteristics of different people's actions; and identify whether the digital actions in the statistical information and the target features are the same feature.

[0029] Compared to existing technologies, the digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar of this invention collects raw radar data of digital actions, processes each frame of raw radar data to obtain the target centroid; calculates the information statistics of the target centroid across multiple frames to obtain target features; inputs the target features into a machine learning model as the basis for classifying and recognizing digital actions; and inputs statistical information to classify and recognize digital actions. This achieves digital action recognition. This technology has low computational resource requirements, is not limited by ambient light, and can more accurately identify images. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a digital action recognition method based on a single-transmitter, dual-receiver millimeter-wave radar in one embodiment of the present invention.

[0031] Figure 2 This is a prediction result of the SVM model recognizing the digit 1 in one embodiment of the present invention;

[0032] Figure 3 This is a prediction result of the SVM model recognizing the digit 0 in one embodiment of the present invention;

[0033] Figure 4 This is a diagram showing the prediction effect of an SVM model recognizing the digits 2 and 3 in one embodiment of the present invention;

[0034] Figure 5 This is a training result diagram of an SVM model recognizing the digit 1 in one embodiment of the present invention;

[0035] Figure 6 This is a training result diagram of an SVM model recognizing the digit 0 in one embodiment of the present invention;

[0036] Figure 7 This is a training result diagram of an SVM model recognizing the digits 2 and 3 in one embodiment of the present invention;

[0037] Figure 8 This is a design effect diagram of handwritten digital actions in one embodiment of the present invention. Detailed Implementation

[0038] The following will describe in more detail a digital action recognition method based on a one-transmitter, two-receiver millimeter-wave radar according to the present invention, with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0039] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description and claims. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0040] This invention provides a digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar, with reference to... Figure 1 As shown, it includes:

[0041] Collect raw radar data of digital motion, process each frame of raw radar data, and obtain the target's center of gravity;

[0042] Calculate the information statistics of the target centroid across multiple frames to obtain the target features;

[0043] The target features are input into a machine learning model as a basis for classifying and recognizing digital actions;

[0044] Input statistical information to classify and recognize digital actions.

[0045] In this embodiment, the machine learning model used is a support vector machine (SVM). In addition, considering human writing habits, it is generally more comfortable to write with the action of making movements parallel to the front of the human body. Since most of the radar data is radial data relative to the radar, in order to enable the radar to obtain as much effective data as possible, the present invention sets the user's hand parallel to the radar hardware when building the hardware. The writing action is a radial movement relative to the radar, so more effective data is obtained by recognizing digital actions.

[0046] Specifically, the classification and recognition of digital actions includes the recognition of digits 0-3.

[0047] Specifically, in this embodiment, the step of processing each frame of raw radar data to obtain the target centroid includes: defining a frame period to divide continuous raw radar data into discrete data sets in units of frames; arranging a frame of raw data according to range, velocity, and antenna channel to obtain a three-dimensional array, which is a single-frame radar data matrix generated in units of data frames; performing two-dimensional fast Fourier transforms on the radar data matrix in the range and velocity dimensions to obtain range-Doppler image data; obtaining multiple target points by setting an energy amplitude threshold and extracting peak values ​​from the range-Doppler image data in terms of amplitude and velocity values; and obtaining the target centroid based on the multiple target points.

[0048] The process of performing a two-dimensional fast Fourier transform on the radar data matrix to obtain range-Doppler image data, and before extracting peak values ​​from the range-Doppler image data, further includes: removing static targets from the range-Doppler image data based on the radar Doppler image velocity measurement principle.

[0049] Specifically, obtaining the target centroid based on multiple target points includes: performing a weighted calculation on the multiple target points with respect to amplitude to obtain the target centroid; the target centroid can be regarded as an abstract representation of the digital action at this moment.

[0050] In this embodiment, the step of calculating the information statistics of the target centroid over multiple frames to obtain target features includes: calculating the cumulative changes in the distance, velocity, and phase of the target centroid over multiple frames as the target features.

[0051] The calculation of the cumulative changes in distance, velocity, and phase of the target's center of gravity across multiple frames includes: sampling the changes using a fixed-length sliding window; determining the validity of a data segment when the distance, velocity, and phase of multiple consecutive frames are non-zero and appear in the middle of the sliding window; treating frames in the segment where all changes are zero as invalid frames and removing them; and arranging the remaining data in chronological order for feature data extraction and calculation. The SVM model determines features that can distinguish different digital actions based on the differences in statistical information of different digital actions in the training data. These features are determined by manually analyzing statistical information and further normalized to adapt to the differences in actions of different people; therefore, the features used to distinguish different digital actions are not necessarily the same feature.

[0052] In this embodiment, the machine learning model distinguishes the features of different digital actions based on the differences in statistical information of different digital actions in the training data; it determines the differences in the features of different digital actions by analyzing statistical information manually, in order to adapt to the characteristics of different people's actions; and it identifies whether the digital actions in the statistical information and the target features are the same feature.

[0053] The process of inputting the target features into a machine learning model for the classification and recognition of digital actions includes: the machine learning model classifying and recognizing digital actions based on whether the input statistical information matches the target features.

[0054] In this embodiment, multiple frames of information for a complete action are cached, with 50 frames considered as a processing unit. Data with a target center-of-gravity velocity of 0 in the processing unit are discarded, and the remaining data are treated as continuous data for statistical calculation. This statistical information is then input into a machine learning model (such as an SVM model) for digital action recognition. The SVM model contains various features of the digital action and determines the type of digital action based on whether the input statistical information matches the features.

[0055] The SVM model determines features that can distinguish different digital actions based on the statistical differences in the training data. These features are determined through human analysis of statistical information and further normalized to adapt to variations in individual actions; therefore, the features used to distinguish different digital actions are not necessarily the same. In this embodiment, the target features extracted from the collected motion trajectory data are input into the SVM model, which calculates the relevant feature thresholds that can distinguish the motion data. This step is called training the SVM model based on motion trajectory data. The final SVM model's training and testing performance on the training dataset is as follows: Figures 2-7 As shown in the diagram, A represents the positive class, B represents the negative class, C represents the incorrect prediction result, D represents the support vector, and the dividing line in the diagram represents the hyperplane of the SVM. The criteria for judging digital actions are calculated by training the SVM model on the data, and the SVM model is a general-purpose model.

[0056] In this embodiment, the effective statistical information that needs to be calculated and used to determine different digital action characteristics includes:

[0057] (a) The proportion of frames with positive velocity values ​​and the proportion of frames with negative velocity values ​​in the valid data are used to determine the value 1. For example... Figure 2 and Figure 5 As shown, 1 is the positive class, and 0, 2, and 3 are the negative classes. The ratio of positive to negative speeds can effectively distinguish 1 from other numbers.

[0058] (b) In the valid data, the point closest to the radar in the vertical direction is used as the dividing point, and the velocity sum of the latter half of the valid frames is used to determine the digit 0. For example... Figure 3 and Figure 6As shown, 0 is the positive class, and 2 and 3 are the negative classes. The sum of positive velocities after the lowest point is used as a feature to distinguish 0 from other numbers.

[0059] (c) In the valid data, the phase sign of the last few frames is used to distinguish the digits 2 and 3. For example... Figure 4 and Figure 7 As shown, data that meets condition (a) is judged as the number 1, data that meets condition (b) is judged as the number 0, and data that does not meet (a) and (b) are distinguished as the number 2 or the number 3 by condition (c), where 2 is the positive class and 3 is the negative class.

[0060] Furthermore, since the features used to distinguish different digit actions in the model are relatively fixed, and there is more than one action to gesture the same digit, in this embodiment, we use the action for specifying the digit action, such as... Figure 8 As indicated by the arrow.

[0061] This invention enables the recognition of four digital actions from 0 to 3 using a millimeter-wave radar with one transmitting and two receiving antennas. Furthermore, it requires less computational resources and is unaffected by ambient light, making it simpler than existing image-based recognition technologies.

[0062] In summary, this invention enables the identification of digital actions using a millimeter-wave radar with one transmitting and two receiving antennas, and is not limited by ambient light, thus allowing for more accurate action identification. Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A digital action recognition method based on a one-transmitter, two-receiver millimeter-wave radar, characterized in that, include: Collect raw radar data of digital motion, process each frame of raw radar data, and obtain the target's center of gravity; Calculate the information statistics of the target centroid across multiple frames to obtain the target features; Among them, the cumulative changes in the distance, velocity, and phase of the target's centroid over multiple frames are used as the target features; The calculation of the cumulative changes in distance, velocity, and phase of the target's centroid across multiple frames includes: sampling the changes using a fixed-length sliding window; determining that the data segment is valid when the distance, velocity, and phase of multiple consecutive frames are not zero and appear in the middle of the sliding window; treating frames in the segment where all changes are zero as invalid frames and clearing them; and arranging the remaining data in chronological order for feature data extraction and calculation. The target features are input into a machine learning model as a basis for classifying and recognizing digital actions; Input statistical information to classify and recognize digital actions; The process of obtaining the target features includes extracting and calculating feature data, which includes: (a) The proportion of frames with positive velocity values ​​and the proportion of frames with negative velocity values ​​in the valid data are used to determine the number 1. (b) In the valid data, the point closest to the radar in the vertical direction is used as the dividing point, and the velocity sum of the second half of the valid frames is used to determine the digit 0. (c) In the valid data, the phase sign of the data in the last few frames is used to distinguish the numbers 2 and 3.

2. The digital action recognition method based on a one-transmitter, two-receiver millimeter-wave radar as described in claim 1, characterized in that, The classification and recognition of digital actions includes the recognition of digits 0-3.

3. The digital action recognition method based on a one-transmitter, two-receiver millimeter-wave radar as described in claim 1, characterized in that, The step of processing each frame of raw radar data to obtain the target centroid includes: Generate a single-frame radar data matrix in units of data frames; Perform a two-dimensional fast Fourier transform on the radar data matrix to obtain range-Doppler image data; Peak extraction is performed on the distance-Doppler map data to obtain multiple target points; The target centroid is obtained from multiple target points.

4. The digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar as described in claim 3, characterized in that, After performing a two-dimensional fast Fourier transform on the radar data matrix to obtain range-Doppler image data, before extracting peak values ​​from the range-Doppler image data, the method further includes: removing static targets from the range-Doppler image data based on the radar Doppler image velocity measurement principle.

5. The digital action recognition method based on a one-transmitter, two-receiver millimeter-wave radar as described in claim 3, characterized in that, Obtaining the target centroid from multiple target points includes: The target centroid is obtained by performing a weighted calculation on multiple target points with respect to amplitude.

6. The digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar as described in claim 1, characterized in that, The target features are input into a machine learning model to classify and recognize digital actions, including: Machine learning models classify and recognize digital actions based on whether the input statistical information matches the target features.

7. The digital action recognition method based on a one-transmitter-two-receiver millimeter-wave radar as described in claim 6, characterized in that, Machine learning models distinguish the features of different digital actions based on the differences in statistical information of different digital actions in the training data; By manually analyzing statistical information, we can determine the differences between various digital action features to accommodate the characteristics of different people's actions; and identify whether the digital actions in the statistical information and the target features are the same feature.

Images