Radar gesture recognition method, apparatus, and electronic device

Abstract

The application provides a radar gesture recognition method, device and electronic equipment, comprising: acquiring a target radar signal collected by a radar, and preprocessing point cloud data based on the target radar signal; extracting a point cloud feature queue from the point cloud data; calculating a plurality of gesture feature queues based on the point cloud feature queue; performing normalization processing on each gesture feature value queue, and constructing a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard; calculating a probability of the existence of a gesture in the target radar signal based on a weight value and an offset value corresponding to each element in the gesture feature value matrix; and determining a gesture recognition result of the target radar signal based on the probability. By storing the parameter values required for gesture calculation trained on a computer in the radar, gesture feature training on the radar is avoided, a high resolution is not required to realize gesture feature recognition, the requirements for the computing power and the memory of the radar are reduced, and the gesture recognition function of the radar is realized at a low cost.

Description

Technical Field

[0001] This invention relates to the field of information technology, and more specifically to radar gesture recognition methods, devices, and electronic equipment. Background Technology

[0002] Gesture recognition refers to understanding a user's interaction intent by recognizing natural finger, wrist, and arm movements. It offers a wider interaction space, greater flexibility, and a better interactive experience. Radar gesture recognition, as a non-contact recognition method, enables simple and efficient information exchange and has wide applications in healthcare, automotive, and entertainment. Therefore, high accuracy is crucial for radar gesture recognition. Most existing radar gesture recognition systems rely on high resolution, generating a point cloud with a large number of points, extracting feature values ​​from the point cloud, and then using machine learning for gesture recognition.

[0003] In related technologies, higher resolution places high demands on radar bandwidth, further increasing chip costs. Meanwhile, using a microcontroller for machine learning based on point clouds with a large number of points requires significant computing power and memory, further increasing product costs. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the high cost of products in the prior art, thereby providing a radar gesture recognition method, device and electronic device.

[0005] In conjunction with the first aspect, the present invention provides a radar gesture recognition method, the method comprising:

[0006] Acquire the target radar signal collected by the radar, and preprocess the target radar signal to obtain point cloud data;

[0007] Extract a point cloud feature queue from the point cloud data. The point cloud feature queue includes several point cloud feature groups consisting of distance weighted values, maximum absolute velocity values, and maximum target reflection energy.

[0008] Based on the point cloud feature queue, multiple gesture feature queues are calculated, and the gesture feature queue includes multiple gesture feature values;

[0009] Each gesture feature value queue is normalized, and a gesture feature value matrix is ​​constructed based on the normalized gesture feature value queue according to a preset standard.

[0010] Based on the weight and offset value corresponding to each element in the gesture feature value matrix, the probability of a gesture in the target radar signal is calculated. Under a preset standard, the weight and offset value corresponding to each element in the gesture feature value matrix are used to characterize the correlation between the element and the gesture.

[0011] Based on the probability, the gesture recognition result of the target radar signal is determined.

[0012] In this approach, point cloud data is generated from the radar-acquired signals, and point cloud features are extracted. This initial screening of the radar-acquired signal data reduces the amount of data transmitted by the radar, thereby lowering the memory requirements of the radar. By storing the parameters required for gesture calculation, trained on a computer, into the radar, the radar can directly use these parameters to perform gesture recognition on target signals. This avoids training gesture features on the radar itself, eliminates the need for high resolution to recognize gesture features, reduces the radar's computing power and memory requirements, and achieves radar gesture recognition functionality at a lower cost.

[0013] In conjunction with the first aspect, in a first embodiment of the first aspect, the step of extracting a point cloud feature queue from the point cloud data includes:

[0014] Based on the point cloud data, the sum of the target reflection energy of each point cloud in the point cloud data is calculated;

[0015] When the sum of the reflected energy of the target is greater than zero, the distance weighted value, the maximum absolute value of velocity, and the maximum value of the reflected energy of the target are calculated for each point cloud in the point cloud data.

[0016] When the sum of the reflected energy of the target is equal to zero, the distance weighted value, the maximum absolute value of velocity, and the maximum value of the reflected energy of the target in each point cloud are all recorded as zero.

[0017] The point cloud feature queue is formed based on the distance weighted value, the maximum absolute velocity value, and the maximum target reflection energy of each point cloud in the point cloud data.

[0018] In conjunction with the first aspect, in the second embodiment of the first aspect, the step of calculating multiple gesture feature queues based on the point cloud feature queue includes:

[0019] Based on the gesture feature value extraction function, gesture feature values ​​are extracted for each point cloud feature group in the point cloud feature queue to obtain multiple gesture feature values.

[0020] Multiple gesture feature queues are obtained by classifying the various gesture feature values.

[0021] In conjunction with the second embodiment of the first aspect, in the third embodiment of the first aspect, the step of extracting gesture feature values ​​from the point cloud feature queue to obtain gesture feature values ​​corresponding to the point cloud feature queue includes:

[0022] Construct a temporary array based on the maximum absolute value of the speed;

[0023] The difference between the maximum and minimum values ​​of the distance weighted value is calculated.

[0024] Based on the temporary array, the difference between the maximum and minimum distance weighted values, and the maximum target reflection energy, the gesture feature value corresponding to the point cloud feature queue is obtained.

[0025] In conjunction with the first aspect, in the fourth embodiment of the first aspect, the normalization processing of each gesture feature value queue, and the construction of a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard, includes:

[0026] The mean and variance of each gesture feature value queue are obtained, and the mean and variance of each gesture feature value queue are obtained by training on the host computer.

[0027] Based on the mean and variance of each gesture feature value queue, the gesture feature value queues are normalized to obtain multiple normalized gesture feature value queues.

[0028] The multiple normalized gesture feature value queues are expanded in dimension according to a preset standard to obtain the gesture feature value matrix.

[0029] In conjunction with the first aspect, in the fifth embodiment of the first aspect, the weights and offsets are obtained in the following manner:

[0030] The host computer trains the sample radar signals with gesture tags to obtain a preset standard training matrix. The gesture tags include: presence of gesture and absence of gesture.

[0031] Based on the training matrix and gesture labels, the weight and offset value corresponding to each element in the training matrix are calculated.

[0032] The weight and offset value corresponding to each element in the training matrix are determined as the weight and offset value of the corresponding element in the gesture feature value matrix.

[0033] In conjunction with the first aspect, in the sixth embodiment of the first aspect, determining the gesture recognition result of the target radar signal based on the probability includes:

[0034] Determine whether the probability is greater than a probability threshold;

[0035] When the probability is greater than the probability threshold, it is determined that a gesture exists in the target radar signal;

[0036] When the probability is not greater than the probability threshold, it is determined that there is no hand gesture in the target radar signal.

[0037] In conjunction with the first aspect, in the seventh embodiment of the first aspect, determining the gesture recognition result of the target radar signal based on the probability further includes:

[0038] Determine whether the probability corresponding to multiple consecutive target radar signals is greater than the probability threshold;

[0039] Count the number of targets in a series of consecutive radar signals whose probability is greater than a probability threshold;

[0040] When the number is greater than the counting threshold, it is determined that there is a gesture in the target radar signal;

[0041] When the number is not greater than the counting threshold, it is determined that there is no hand gesture in the target radar signal.

[0042] In a second aspect, the present invention also provides a radar gesture recognition device, the device comprising:

[0043] The point cloud acquisition unit is used to acquire the target radar signal collected by the radar, and preprocess the target radar signal to obtain point cloud data.

[0044] The first extraction unit is used to extract a point cloud feature queue from the point cloud data. The point cloud feature queue includes: a number of point cloud feature groups composed of distance weighted values, the maximum absolute value of velocity, and the maximum value of target reflection energy.

[0045] The second extraction unit is used to calculate multiple gesture feature queues based on the point cloud feature queue, wherein the gesture feature queue includes multiple gesture feature values;

[0046] The normalization processing unit is used to normalize each gesture feature value queue and construct a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard.

[0047] The calculation unit is used to calculate the probability that a gesture exists in the target radar signal based on the weight and offset value corresponding to each element in the gesture feature value matrix. Under a preset standard, the weight and offset value corresponding to each element in the gesture feature value matrix are used to characterize the correlation between the element and the gesture.

[0048] A determining unit is used to determine the gesture recognition result of the target radar signal based on the probability.

[0049] According to a third aspect, embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the radar gesture recognition method of any one of the first aspects and its optional embodiments. Attached Figure Description

[0050] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart of a radar gesture recognition method proposed according to an exemplary embodiment.

[0052] Figure 2 This is a schematic diagram of a radar hardware connection according to an exemplary embodiment.

[0053] Figure 3 This is a radar signal preprocessing procedure proposed according to an exemplary embodiment.

[0054] Figure 4 This is a flowchart of a point cloud feature extraction method proposed according to an exemplary embodiment.

[0055] Figure 5 This is a point cloud feature queue data structure proposed according to an exemplary embodiment.

[0056] Figure 6 This is a flowchart of a gesture feature extraction method proposed according to an exemplary embodiment.

[0057] Figure 7 This is a flowchart of the main program for gesture recognition on a radar, according to an exemplary embodiment.

[0058] Figure 8 This is a flowchart illustrating sampling and transmission on a radar microcontroller according to an exemplary embodiment.

[0059] Figure 9 It is a point cloud data format proposed according to an exemplary embodiment.

[0060] Figure 10 This is a format for storing files during sampling, according to an exemplary embodiment.

[0061] Figure 11 This is a flowchart of a data training method on a host computer according to an exemplary embodiment.

[0062] Figure 12 This is a flowchart of a radar point cloud feature extraction method proposed according to an exemplary embodiment.

[0063] Figure 13This is a flowchart of a gesture feature extraction method proposed according to an exemplary embodiment.

[0064] Figure 14 This is a structural block diagram of a radar gesture recognition device according to an exemplary embodiment.

[0065] Figure 15 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0066] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] In related technologies, radar gesture recognition mostly relies on high resolution to form a point cloud with a large number of points. Then, the feature values ​​of the point cloud are extracted for machine learning to determine the type of gesture. High resolution places high demands on the bandwidth of the radar, further increasing chip costs. At the same time, when using a microcontroller to perform machine learning based on a large number of points, more computing power and memory are required, further increasing the cost of the product.

[0068] To address the aforementioned problems, this invention provides a radar gesture recognition method applied to radar. In the following method embodiments, the radar is used as the executing entity. The radar in this embodiment is suitable for identifying the presence of gestures in usage scenarios. The radar gesture recognition method provided by this invention generates point cloud data from the radar-acquired signals and extracts point cloud features, performing preliminary screening of the radar-acquired signal data. This reduces the amount of data transmitted by the radar, thereby lowering the memory requirements of the radar. By storing the parameters required for gesture calculation obtained through training on a computer into the radar, the radar can directly use gesture feature values ​​to perform gesture recognition on target signals, avoiding gesture feature training on the radar itself. This eliminates the need for high resolution to achieve gesture feature recognition, reducing the radar's computing power and memory requirements, and implementing radar gesture recognition functionality at a lower cost.

[0069] Figure 1 This is a flowchart of a radar gesture recognition method according to an exemplary embodiment. Figure 1 As shown, the radar gesture recognition method includes the following steps S101 to S106.

[0070] In step S101, the target radar signal collected by the radar is acquired, and point cloud data is obtained by preprocessing based on the target radar signal.

[0071] In this embodiment of the invention, point cloud data for gesture recognition is obtained by performing preliminary processing on the signals collected by radar.

[0072] In one example, the radar preprocessing process can be performed in the radar's radio frequency and signal processing unit. Figure 2 This is a schematic diagram of a radar hardware connection according to an exemplary embodiment. For example... Figure 2 As shown, the radio frequency and signal processing unit 11 in radar 1 includes a radio frequency unit 111 and a signal processing unit 112. The microcontroller 12 is connected to the radio frequency and signal processing unit 11 through I2C and SPI interfaces. Radar 1 is connected to the host computer 2 through a serial port.

[0073] Figure 3 This is a radar signal preprocessing procedure proposed according to an exemplary embodiment. For example... Figure 3 As shown, the radar signal preprocessing process can include: receiving the microcontroller configuration and configuring radio frequency and signal processing; reading the analog-to-digital conversion (AD) value for each chirp signal; windowing the AD value of each chirp signal and performing a one-dimensional Fourier transform (1DFFT); after completing the 1DFFT of one frame of chirps signals, static clutter cancellation is performed on the first N points (N is configured by the microcontroller) of the 1DFFT of multiple chirps signals by column, followed by windowing, and then a two-dimensional Fourier transform (2DFFT) is performed. The 2DFFT results in a two-dimensional matrix with the number of rows equal to the number of chirp signal frames and the number of columns equal to N. A constant false alarm rate (CFAR) is performed on this two-dimensional matrix, and the nine targets with the highest reflected energy are uploaded to the microcontroller. If there are fewer than nine targets, they are padded with zeros to make nine targets. After the message upload is completed, the processing of the next frame begins, and the next cycle starts.

[0074] In step S102, a point cloud feature queue is extracted from the point cloud data; based on the point cloud feature queue, multiple gesture feature queues are calculated.

[0075] In this embodiment of the invention, the point cloud feature queue includes several point cloud feature groups composed of distance-weighted values, maximum absolute velocity values, and maximum target reflection energy. Therefore, extracting the point cloud feature queue from the point cloud data includes: calculating the sum of target reflection energy for each point cloud in the point cloud data based on the point cloud data; when the sum of target reflection energy is greater than zero, calculating the distance-weighted value, maximum absolute velocity value, and maximum target reflection energy for each point cloud in the point cloud data; when the sum of target reflection energy is equal to zero, recording the distance-weighted value, maximum absolute velocity value, and maximum target reflection energy for each point cloud in the point cloud data as zero; and forming the point cloud feature queue based on the distance-weighted value, maximum absolute velocity value, and maximum target reflection energy for each point cloud in the point cloud data.

[0076] In one example, the feature extraction function of radar point cloud in the microcontroller can be called by the main program of the microcontroller program to extract the feature queue of point cloud. Figure 4 This is a flowchart of a point cloud feature extraction method proposed according to an exemplary embodiment. For example... Figure 4 As shown, the point cloud extraction process can include: the radar point cloud feature extraction function obtains the input point cloud cache; the target reflection energy of all point clouds is summed; when the sum of target reflection energy > 0, the range weighted value is obtained as: target point range index * target point reflection energy sum / sum of all point target reflection energy; the maximum absolute velocity value is obtained as: max(abs(target point velocity index of point cloud - radar frame chirps / 2)); the maximum target reflection energy value is obtained as: max(target point energy of point cloud); when the sum of target reflection energy is 0, all three feature values ​​are set to 0. The three feature values ​​of target point range weighted value, target reflection energy, and maximum absolute velocity value are returned to the microcontroller main program. At this time, when the length of the point cloud feature queue is < 15, the point cloud feature values ​​are directly appended to the queue and the queue length is incremented by 1; when the length of the point cloud feature queue is greater than or equal to 15, the first element of the point cloud feature queue is dequeued, and the subsequent point cloud feature values ​​are appended to the queue. Figure 5 This is a point cloud feature queue data structure proposed according to an exemplary embodiment.

[0077] In step S103, multiple gesture feature queues are calculated based on the point cloud feature queues.

[0078] In this embodiment of the invention, the gesture feature queue includes multiple gesture feature values. Based on the point cloud feature queue, multiple gesture feature queues are calculated, including: extracting gesture feature values ​​from each point cloud feature group in the point cloud feature queue using a gesture feature value extraction function to obtain multiple gesture feature values; and classifying each gesture feature value to obtain multiple gesture feature queues. Specifically, extracting gesture feature values ​​from the point cloud feature queues to obtain the gesture feature values ​​corresponding to the point cloud feature queues includes: constructing a temporary array based on the maximum absolute velocity value; calculating the difference between the maximum and minimum distance weighted values; and obtaining the gesture feature values ​​corresponding to the point cloud feature queues based on the temporary array, the difference between the maximum and minimum distance weighted values, and the maximum target reflection energy.

[0079] In one example, the gesture feature extraction function in the microcontroller can be called by the main program of the microcontroller program to extract the gesture feature queue. Figure 6 This is a flowchart of a gesture feature extraction method proposed according to an exemplary embodiment. Figure 6As shown, the extraction process of the gesture feature queue can include: the gesture feature extraction function obtains the input point cloud feature queue, firstly storing the maximum absolute velocity value in the queue into a temporary array 1, calculating the maximum distance weighted value = max(distance weighted value column of the input queue), and calculating the minimum distance weighted value = min(distance weighted value column of the input queue). Parameter 1 = Maximum distance weighted value - Minimum distance weighted value; Parameter 2 = max(temporary array 1), setting the maximum value position of the column with the maximum absolute velocity value of temporary array 1 to 0; Parameter 3 = max(temporary array 1), setting the maximum value position of the column with the maximum absolute velocity value of temporary array 1 to 0; Parameter 4 = max(temporary array 1), setting the maximum value position of the column with the maximum absolute velocity value of temporary array 1 to 0; Parameter 5 = average(temporary array 1) calculates the mean; Parameter 6 = average(distance weighted value of input queue) calculates the mean; Parameter 7 = var(temporary array 1) calculates the variance; Parameter 8 = var(distance weighted value of input queue) calculates the variance; Parameter 9 = max(maximum reflected energy of target in input queue), and finally, these 9 parameters are returned to the main program of the microcontroller that calls the program.

[0080] In step S104, each gesture feature value queue is normalized, and a gesture feature value matrix is ​​constructed based on the normalized gesture feature value queue according to a preset standard.

[0081] In this embodiment of the invention, the normalization process may include: obtaining the mean and variance corresponding to each gesture feature value queue, wherein the mean and variance corresponding to each gesture feature value queue are obtained by training on a host computer; performing normalization processing on each gesture feature value queue based on the mean and variance corresponding to each gesture feature value queue to obtain multiple normalized gesture feature value queues; and expanding the dimensions of the multiple normalized gesture feature value queues according to a preset standard to obtain a gesture feature value matrix.

[0082] In one example, taking a gesture feature value count of 9 and a gesture feature value queue length of 15 as an example, the normalization process may include: subtracting the mean of each of the 9 gesture feature values ​​and then dividing by the variance of each of the 9 gesture feature values ​​to normalize them; then expanding the dimensionality of the normalized 9 gesture feature values ​​to generate 54 gesture feature values. The mean and variance of the gesture feature values ​​are obtained through training on a host computer.

[0083] In step S105, the probability of a gesture being present in the target radar signal is calculated based on the weight and offset value corresponding to each element in the gesture feature value matrix.

[0084] In this embodiment of the invention, the weights and offsets are obtained as follows: A training matrix with preset standards is obtained by training sample radar signals with gesture tags on a host computer. The gesture tags include: presence of gesture and absence of gesture. Based on the training matrix and gesture tags, the weights and offsets corresponding to each element in the training matrix are calculated. The weights and offsets corresponding to each element in the training matrix are then determined as the weights and offsets of the corresponding elements in the gesture feature value matrix. In one example, a data X can be generated by multiplying the 54 gesture feature values ​​obtained in the above embodiment by their respective weights, summing them, and adding the offset value b. The weights corresponding to the 54 gesture feature values ​​are obtained through training on a host computer and written into the microcontroller program. The probability of the gesture is then calculated by substituting X into a probability calculation function.

[0085] In step S106, the gesture recognition result of the target radar signal is determined based on probability.

[0086] In this embodiment of the invention, the process of determining the gesture recognition result of the target radar signal may include: determining whether the probability is greater than a probability threshold; when the probability is greater than the probability threshold, determining that there is a gesture in the target radar signal; when the probability is not greater than the probability threshold, determining that there is no gesture in the target radar signal.

[0087] In one implementation scenario, determining the gesture recognition result of the target radar signal based on probability also includes: determining whether the probability corresponding to multiple consecutive target radar signals is greater than a probability threshold; counting the number of consecutive target radar signals whose corresponding probability is greater than the probability threshold; determining that there is a gesture in the target radar signal when the number is greater than the counting threshold; and determining that there is no gesture in the target radar signal when the number is not greater than the counting threshold.

[0088] In one example, the process of determining the gesture recognition result of the target radar signal based on probability may include: if the probability > 0.6, incrementing the gesture counter by 1; otherwise, setting the gesture counter to 0; if the gesture counter > 5, sending a gesture message to the host computer; otherwise, sending a no-gesture message. After sending the message, the process continues to receive point cloud messages from the radio frequency and signal processing units, entering the next loop. Figure 7 This is a flowchart of the main program for gesture recognition on a radar, according to an exemplary embodiment.

[0089] Through the above embodiments, by generating point cloud data from the radar-acquired signals and extracting point cloud features, the radar-acquired signal data can be preliminarily screened, reducing the amount of data transmitted by the radar and thus lowering the memory requirements of the radar. By storing the parameters required for gesture calculation obtained through training on a computer into the radar, the radar can directly use the parameters required for gesture calculation to perform gesture recognition on target signals, avoiding gesture feature training on the radar itself. This eliminates the need for high resolution to achieve gesture feature recognition, reducing the radar's computing power and memory requirements, and enabling radar gesture recognition functionality at a lower cost.

[0090] The following examples will specifically illustrate the process of performing data training on a host computer to obtain parameter values ​​for radar gesture recognition.

[0091] The host computer receives point cloud data generated by the radar, uses the point cloud data as sample data, and trains the sample data by calling the host computer's machine learning training program. After the training is completed, the host computer obtains the parameters required for gesture calculation and writes them into the microcontroller program.

[0092] In one example, the sample collection process may involve two people working together: one performs the actions, and the other operates the sampling software. When the action begins, the "Start Sampling" button is pressed, causing the host computer to begin receiving radar messages. After receiving 15 messages (15 being the number of messages within one gesture, which can be adjusted), the sampled data is written to files (positive samples to a positive sample file, and negative samples to a negative sample file), resulting in both positive and negative sample data. The amount of positive and negative sample data collected should be similar; a significant difference would cause the results to shift towards the sample with the larger number of samples collected. Figure 8 This is a flowchart illustrating sampling and transmission on a radar microcontroller according to an exemplary embodiment. After configuring the radio frequency (RF) and signal processing chips through the microcontroller program, the microcontroller receives point cloud data packets preprocessed by the RF and signal processing units, packages them, and forwards them to the host computer. During forwarding, the microcontroller only modifies the packet header, checksum, and packet trailer; the data and length remain unchanged. Figure 9 It is a point cloud data format proposed according to an exemplary embodiment. Figure 10 This is a sampling-time storage file format proposed according to an exemplary embodiment. In this format, a velocity index, a distance index, and a target reflection energy value are each represented by a point; nine points constitute a point cloud; and fifteen point clouds constitute one sampling.

[0093] In one example, after sampling is completed, data training needs to be performed on the host computer. Figure 11 This is a flowchart illustrating a data training method on a host computer according to an exemplary embodiment. For example... Figure 11As shown, the data training method on the host computer includes: opening the positive sample file and reading it with a data length of one sampling (9*15 targets). The point cloud features of the sample data obtained from one sampling are extracted using a radar point cloud feature extraction function to obtain a point cloud feature queue. Figure 12 This is a flowchart of a radar point cloud feature extraction method according to an exemplary embodiment. The radar point cloud feature extraction process may include: calculating the distance-weighted value, the maximum absolute velocity value, and the maximum energy value of target reflection of the sample data point cloud; calculating 15 point cloud feature values ​​and storing them in a matrix; and returning the results to the machine learning main program on the host computer.

[0094] Gesture features of the radar point cloud feature queue are extracted using a gesture feature extraction function. Nine gesture features are selected and stored in the training sample matrix 1. Figure 12 This is a flowchart of a gesture feature extraction method according to an exemplary embodiment. The gesture feature extraction process may include: calculating 9 gesture feature values ​​based on the point cloud feature values ​​of 15 input point clouds, and returning them to the host computer machine learning main program in a matrix manner.

[0095] After processing the positive sample data in the positive sample file, open the negative sample file and process the negative sample data in the same way as the positive sample file. After reading and processing both positive and negative sample data, calculate the mean and variance of the nine gesture feature parameters column-wise from the training sample 1 matrix, and then normalize the nine gesture feature parameters column-wise from the training sample 1 matrix. To better extract the features of the samples and make them more discriminative, thus facilitating classification or regression, expand the dimension of the nine gesture feature parameters of the training sample 1 matrix row-wise, generating a 54-column matrix, which is then stored in the training sample 2 matrix.

[0096] A training result matrix is ​​constructed, which indicates whether the current training samples contain gestures. Positions containing gestures are set to 1, and positions not containing gestures are set to 0. The number of positive samples is calculated from the positive sample file, and the number of negative samples is calculated from the negative sample file. Therefore, in the training result matrix, the data for the first positive sample count is set to 1, and the data for the second negative sample count is set to 0. The training sample 2 matrix and the training result matrix are input into a logistic regression training algorithm for training. The training result contains the parameters required for gesture calculation, including the weights of 54 gesture feature parameters and an offset value b. The training sample 1 matrix is ​​output, and the mean and variance of the 9 gesture feature parameters, the weights of the 54 gesture feature parameters after dimensionality expansion, and the offset value b are calculated column-wise. The mean, variance, weights, and offset value b of the gesture feature parameters are then written into the microcontroller program.

[0097] Based on the same inventive concept, the present invention also provides a radar gesture recognition device.

[0098] Figure 14 This is a structural block diagram of a radar gesture recognition device according to an exemplary embodiment. Figure 14 As shown, the radar gesture recognition device includes a point cloud acquisition unit 1401, a first extraction unit 1402, a second extraction unit 1403, a normalization processing unit 1404, a calculation unit 1405, and a determination unit 1406.

[0099] The point cloud acquisition unit 1401 is used to acquire the target radar signal collected by the radar, and to obtain point cloud data based on the target radar signal through preprocessing.

[0100] The first extraction unit 1402 is used to extract a point cloud feature queue from the point cloud data. The point cloud feature queue includes several point cloud feature groups composed of distance weighted values, the maximum absolute value of velocity, and the maximum value of target reflection energy.

[0101] The second extraction unit 1403 is used to calculate multiple gesture feature queues based on the point cloud feature queues. The gesture feature queues include multiple gesture feature values.

[0102] The normalization processing unit 1404 is used to normalize each gesture feature value queue and construct a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard.

[0103] The calculation unit 1405 is used to calculate the probability that a gesture exists in the target radar signal based on the weight and offset value corresponding to each element in the gesture feature value matrix. Under a preset standard, the weight and offset value corresponding to each element in the gesture feature value matrix are used to characterize the correlation between the element and the gesture.

[0104] The determination unit 1406 is used to determine the gesture recognition result of the target radar signal based on probability.

[0105] The specific limitations and beneficial effects of the aforementioned radar gesture recognition device can be found in the limitations of the radar gesture recognition method described above, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0106] Figure 15 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment. For example... Figure 15As shown, the device includes one or more processors 1510 and a memory 1520, the memory 1520 including persistent memory and volatile memory. Figure 15 Taking a processor 1510 as an example, the device may also include an input device 1530 and an output device 1540.

[0107] The processor 1510, memory 1520, input device 1530, and output device 1540 can be connected via a bus or other means. Figure 15 Taking the example of a connection between China and Israel via a bus.

[0108] Processor 1510 can be a Central Processing Unit (CPU). Processor 1510 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0109] The memory 1520, as a non-transitory computer-readable storage medium, includes persistent memory and volatile memory, and can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the radar gesture recognition method in this embodiment. The processor 1510 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1520, thereby implementing any of the aforementioned radar gesture recognition methods.

[0110] The memory 1520 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data that is needed and required. Furthermore, the memory 1520 may include high-speed random access memory and may also include non-transitory memory, such as at least one flash memory device or other non-transitory solid-state storage devices. In some embodiments, the memory 1520 may optionally include memory remotely located relative to the processor 1510, and these remote memories can be connected to the data processing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0111] Input device 1530 can receive input digital or character information, and generate key signal inputs related to user settings and function control. Output device 1540 may include display devices such as a display screen.

[0112] One or more modules are stored in memory 1520, and when executed by one or more processors 1510, they perform actions such as... Figures 1-13 The method shown.

[0113] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in [reference 1]. Figures 1-13 The relevant descriptions in the illustrated embodiments.

[0114] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A radar gesture recognition method, characterized by, The method comprises: acquiring a target radar signal collected by a radar, and preprocessing point cloud data based on the target radar signal; extracting a point cloud feature queue from the point cloud data, the point cloud feature queue comprising a plurality of point cloud feature groups constituted by a distance weighting value, a maximum value of absolute value of velocity, and a maximum value of target reflection energy; calculating a plurality of gesture feature queues based on the point cloud feature queue, the gesture feature queue comprising a plurality of gesture feature values; normalizing each gesture feature value queue, and constructing a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard; calculating a probability of existence of a gesture in the target radar signal based on a weight value and an offset value corresponding to each element in the gesture feature value matrix, the weight value and the offset value corresponding to each element in the gesture feature value matrix being used to represent the relevance between the element and the gesture under a preset standard; determining a gesture recognition result of the target radar signal based on the probability; the calculating of the plurality of gesture feature queues based on the point cloud feature queue comprises: extracting gesture feature values from each point cloud feature group in the point cloud feature queue based on a gesture feature value extraction function to obtain a plurality of gesture feature values; classifying the gesture feature values to obtain a plurality of gesture feature queues; the extracting of gesture feature values from the point cloud feature queue to obtain gesture feature values corresponding to the point cloud feature queue comprises: constructing a temporary array based on the maximum value of absolute value of velocity; calculating a difference between the maximum value and the minimum value of the distance weighting value; obtaining the gesture feature values corresponding to the point cloud feature queue based on the temporary array, the difference between the maximum value and the minimum value of the distance weighting value, and the maximum value of target reflection energy.

2. The method of claim 1, wherein, the extracting of the point cloud feature queue from the point cloud data comprises: calculating a sum of target reflection energy of each point cloud in the point cloud data based on the point cloud data; when the sum of target reflection energy is greater than zero, calculating the distance weighting value, the maximum value of absolute value of velocity, and the maximum value of target reflection energy of each point cloud in the point cloud data; when the sum of target reflection energy is equal to zero, recording the distance weighting value, the maximum value of absolute value of velocity, and the maximum value of target reflection energy of each point cloud in the point cloud data as zero; composing the point cloud feature queue based on the distance weighting value, the maximum value of absolute value of velocity, and the maximum value of target reflection energy of each point cloud in the point cloud data.

3. The method of claim 1, wherein, the normalizing of each gesture feature value queue and the construction of the gesture feature value matrix based on the normalized gesture feature value queue according to the preset standard comprise: obtaining a mean value and a variance corresponding to each gesture feature value queue, the mean value and the variance corresponding to each gesture feature value queue being obtained by training of an upper computer; normalizing each gesture feature value queue based on the mean value and the variance to obtain a plurality of normalized gesture feature value queues; expanding the plurality of normalized gesture feature value queues according to the preset standard to obtain the gesture feature value matrix.

4. The method of claim 1, wherein, the weight value and the offset value are obtained in the following manner: The host computer trains a sample radar signal with a gesture label to obtain a preset standard training matrix, the gesture label including: presence of a gesture and absence of a gesture; Based on the training matrix and the gesture label, the weight value and the offset value corresponding to each element in the training matrix are calculated; The weight value and the offset value corresponding to each element in the training matrix are determined as the weight value and the offset value of the corresponding element in the gesture feature value matrix.

5. The method of claim 1, wherein, The gesture recognition result of the target radar signal is determined based on the probability, including: Determine whether the probability is greater than a probability threshold value; When the probability is greater than the probability threshold value, it is determined that there is a gesture in the target radar signal; When the probability is not greater than the probability threshold value, it is determined that there is no gesture in the target radar signal.

6. The method of claim 1, wherein, The gesture recognition result of the target radar signal is determined based on the probability, and further includes: Respectively determine whether the probabilities corresponding to the continuous multiple target radar signals are greater than a probability threshold value; Count the number of probabilities greater than the probability threshold value in the continuous multiple target radar signals; When the number is greater than a count threshold value, it is determined that there is a gesture in the target radar signal; When the number is not greater than the count threshold value, it is determined that there is no gesture in the target radar signal.

7. A radar gesture recognition apparatus, characterized by The device includes: A point cloud acquisition unit configured to acquire a target radar signal collected by a radar, and preprocess to obtain point cloud data based on the target radar signal; A first extraction unit configured to extract a point cloud feature queue from the point cloud data, the point cloud feature queue including: a plurality of point cloud feature groups composed of a distance weighting value, a maximum value of an absolute value of a velocity, and a maximum value of a target reflection energy; A second extraction unit configured to calculate a plurality of gesture feature queues based on the point cloud feature queue, the gesture feature queue including: a plurality of gesture feature values; the calculation of the plurality of gesture feature queues based on the point cloud feature queue includes: performing gesture feature value extraction on each point cloud feature group in the point cloud feature queue based on a gesture feature value extraction function to obtain a plurality of gesture feature values; and classifying the gesture feature values to obtain a plurality of gesture feature queues; the gesture feature value extraction on the point cloud feature queue includes: constructing a temporary array based on the maximum value of the absolute value of the velocity; calculating a difference between a maximum value and a minimum value of the distance weighting value; and obtaining the gesture feature value corresponding to the point cloud feature queue based on the temporary array, the difference between the maximum value and the minimum value of the distance weighting value, and the maximum value of the target reflection energy; A normalization processing unit configured to normalize each gesture feature value queue, and construct a gesture feature value matrix based on the normalized gesture feature value queue according to a preset standard; A calculation unit configured to calculate a probability of presence of a gesture in the target radar signal based on the weight value and the offset value corresponding to each element in the gesture feature value matrix, the weight value and the offset value corresponding to each element in the gesture feature value matrix being used to represent the relevance between the element and a gesture under a preset standard; A determination unit configured to determine a gesture recognition result of the target radar signal based on the probability.

8. An electronic device, comprising: The radar gesture recognition method comprises a memory and a processor, the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the radar gesture recognition method according to any one of claims 1-6.

Images