A gesture recognition system and method based on microwave signals

Abstract

The application relates to a gesture recognition system and method based on a microwave signal, a microwave signal transceiving system emits a microwave signal to a to-be-detected object and a scene where the to-be-detected object is located in a detection range, and meanwhile, echo data is collected; a microwave signal preprocessing system processes the echo data to obtain original point cloud data; valid point cloud data is screened out, if a gesture exists, target parameters of the gesture point cloud are extracted, gesture characteristic information is generated; whether the gesture characteristic information exists in a gesture characteristic database is judged, if yes, the gesture characteristic information is called and the gesture characteristic information and gesture category information of the to-be-detected object are displayed through a man-machine interaction interface, if not, the gesture recognition system classifies and identifies the gesture characteristic information by using a classification model to obtain corresponding gesture category information, and the gesture category information is stored in the gesture characteristic database; the application can overcome the influence of environmental interference and other user interference on microwave gesture recognition, and the accuracy of microwave gesture recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a gesture recognition system and method, in particular to a gesture recognition system and method based on microwave signals. BACKGROUND

[0002] With the rapid development of technology and the improvement of people's living standards, the demand for human-computer interaction technology in the fields of smart home, smart cockpit, smart factory, smart medical treatment and the like is increasing. Compared with existing contact-type sensors (such as touch screens), optical sensors (such as cameras), acoustic sensors (such as microphones) and the like, microwave sensors have the advantages of non-contact, non-invasion of privacy, resistance to environmental interference, all-day availability and the like, so that gesture recognition based on microwave sensors will become a human-computer interaction technology with great application prospects in the future.

[0003] The gesture recognition method using lower frequency, non-wideband microwave signals such as Wi-Fi has a very low distance resolution and poor anti-interference ability, and cannot realize gesture recognition with interference suppression and robustness. Lower frequency microwave signals require the use of larger size transmitting and receiving antennas.

[0004] Interference from the environment and other users is a major factor affecting the accuracy and robustness of microwave gesture recognition, so overcoming the influence of environmental interference and interference from other users on microwave gesture recognition is a problem that needs to be solved urgently. SUMMARY

[0005] In view of the above problems, the present application provides a gesture recognition system based on microwave signals, which comprises:

[0006] A microwave signal transceiving system is placed in front of a to-be-detected object to obtain echo data of the to-be-detected object and the scene thereof;

[0007] A microwave signal preprocessing system extracts gesture feature information of the to-be-detected object;

[0008] A gesture feature database stores gesture category information and corresponding gesture feature information;

[0009] A gesture recognition system classifies the gesture feature information of the to-be-detected object and determines corresponding gesture category information;

[0010] A human-computer interaction interface displays gesture feature information and gesture category information.

[0011] Further, the microwave signal transceiving system comprises a microwave transmitting component, a microwave receiving component and an analog-to-digital conversion component.

[0012] The application further provides a gesture recognition method based on microwave signals, which depends on the gesture recognition system based on microwave signals.

[0013] S1: The microwave signal transceiver system transmits microwave signals to the to-be-detected object and the scene where the to-be-detected object is located in the detection range, collects echo data reflected by the to-be-detected object and the scene where the to-be-detected object is located, and converts analog echo data into digital echo data;

[0014] S2: The microwave signal preprocessing system processes the echo data to obtain original point cloud data of the to-be-detected object and the scene where the to-be-detected object is located;

[0015] S3: The effective point cloud data in the original point cloud data is screened, and it is determined whether a gesture exists, if yes, step S4 is performed, and if no, step S1 is returned.

[0016] S4: The target parameters of the gesture point cloud are extracted to generate gesture feature information;

[0017] S5: It is determined whether the gesture feature information exists in the gesture feature database, if yes, step S6 is performed, and if no, step S7 is jumped to;

[0018] S6: The gesture feature information is called and the gesture feature information and gesture category information of the to-be-detected object are displayed through a human-computer interaction interface;

[0019] S7: The gesture recognition system classifies and identifies the gesture feature information by using a classification model to obtain corresponding gesture category information and stores the gesture category information in the gesture feature database.

[0020] Further, step S3 screens the original point cloud data by using an interference suppression algorithm.

[0021] Further, step S3 includes the following steps:

[0022] S31: According to the set detection range radius R epsilon and the minimum point number N minpts , all points in the current frame point cloud data are divided into core points, boundary points and noise points;

[0023] S32: The core points and the boundary points are composed into clustering clusters to find all dense regions;

[0024] S33: The dense regions are regarded as multiple clustering clusters;

[0025] S34: Each clustering cluster of each frame of spatial point cloud is numbered and counted, and it is determined whether the count is greater than a point threshold, if yes, the target gesture object is determined, and if no, step S31 is returned;

[0026] Further, the step S3 comprises the following steps:

[0027] S35: determining all valid points and interference points within the neighborhood radius of the target gesture object space coordinates;

[0028] S36: judging whether the number of valid points is greater than or equal to the update threshold, if yes, updating the position of the target gesture object, if no, proceeding to S37;

[0029] S37: judging whether the number of valid points is less than the minimum number of points, if yes, proceeding to step S38, if no, returning to step S35;

[0030] S38: judging whether the number of invalid points is greater than or equal to the valid point threshold of the target gesture object leaving and the cluster is not 0, if yes, proceeding to 39, if no, returning to step S35;

[0031] S39: not updating the position of the target gesture object, and adding 1 to the frame number parameter corresponding to the disappearance of the target gesture object;

[0032] S310: outputting the current frame number and the continuous leaving frame number;

[0033] Further, the step S4 further comprises a step S41 of performing feature enhancement on the gesture feature information.

[0034] Further, the formula of the feature enhancement is: A1=10log[10xA0]+bias, wherein A1 represents the amplitude of the point cloud data after enhancement, A0 represents the amplitude of the point cloud data before enhancement, and bias represents a bias constant.

[0035] The application provides a gesture recognition system and method based on a microwave signal, a microwave signal transceiver system transmits microwave signals to a to-be-measured object and a scene in a detection range, and simultaneously collects echo data; a microwave signal preprocessing system processes the echo data to obtain original point cloud data; valid point cloud data is screened out, it is judged whether a gesture exists, target parameters of the gesture point cloud are extracted, and gesture feature information is generated; it is further judged whether the gesture feature information exists in a gesture feature database, if yes, the gesture feature information is called and gesture feature information and gesture category information of the to-be-measured object are displayed through a man-machine interaction interface, if not, the gesture recognition system classifies and recognizes the gesture feature information by using a classification model to obtain corresponding gesture category information, and the gesture category information is stored in the gesture feature database, the application can overcome the influence of environmental interference and other user interference on microwave gesture recognition, and improve the accuracy of microwave gesture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a schematic diagram of the gesture recognition system based on the microwave signal;

[0037] Figure 2 is a gesture schematic diagram designed in the embodiment of the present application;

[0038] Figure 3 is a three-dimensional feature map of a gesture in the embodiment of the present application;

[0039] Figure 4 is a convolutional neural network schematic diagram in the embodiment of the present application.

[0040] Explanation of reference signs

[0041] 1 microwave signal transceiver system 2 microwave signal preprocessing system 3 gesture feature database

[0042] 4 gesture recognition system 5 human-computer interaction interface. DETAILED DESCRIPTION

[0043] In order to have a further understanding of the object, structure, characteristics, and functions of the present application, the following detailed description is given in conjunction with the embodiments.

[0044] In view of the above problems, the present application provides a gesture recognition system based on microwave signals, referring to Figure 1 , Figure 1 is a gesture recognition system based on microwave signals schematic diagram, a gesture recognition system based on microwave signals includes a microwave signal transceiver system 1, a microwave signal preprocessing system 2, a gesture feature database 3, a gesture recognition system 4 and a human-computer interaction interface 5, wherein the microwave signal transceiver system 1 is placed in front of the object to be measured, and the echo data of the object to be measured and the scene is obtained; the microwave signal preprocessing system 2 processes the echo data, extracts the gesture feature information of the object to be measured, the gesture feature database 3 stores the gesture category information and the corresponding gesture feature information, the gesture recognition system 4 classifies the gesture feature information of the object to be measured and determines the corresponding gesture category information, and the human-computer interaction interface 5 displays the gesture feature information and the gesture category information.

[0045] The microwave signal transceiver system 1 includes a microwave transmitting component, a microwave receiving component and an analog-to-digital conversion component, the microwave transmitting component transmits microwave signals to the object to be measured and the scene, the microwave receiving component collects echo data emitted by the object to be measured and the scene, and the analog-to-digital conversion component converts analog echo data into digital echo data.

[0046] The present application further provides a gesture recognition method based on microwave signals, which depends on the above-mentioned gesture recognition system based on microwave signals, and the gesture recognition method comprises the following steps:

[0047] S1: The microwave signal transceiver system transmits microwave signals to the to-be-measured object and the scene where the to-be-measured object is located in the detection range, collects echo data reflected by the to-be-measured object and the scene where the to-be-measured object is located, and converts analog echo data into digital echo data;

[0048] S2: The microwave signal preprocessing system processes the echo data to obtain original point cloud data of the to-be-measured object and the scene where the to-be-measured object is located;

[0049] S3: Screening the effective point cloud data in the original point cloud data, judging whether there is a gesture, if yes, then performing step S4, if no, then returning to step S1.

[0050] S4: Extracting the target parameters of the gesture point cloud to generate gesture feature information;

[0051] S5: Judging whether the gesture feature information exists in the gesture feature database, if yes, then performing step S6, if no, then jumping to step S7;

[0052] S6: Calling the gesture feature information and displaying the gesture feature information and gesture category information of the to-be-measured object through a man-machine interaction interface;

[0053] S7: The gesture recognition system classifies and identifies the gesture feature information by using a classification model to obtain corresponding gesture category information and stores the gesture category information in the gesture feature database.

[0054] Further, step S3 screens the original point cloud data by using an interference suppression algorithm.

[0055] Further, step S3 includes the following steps:

[0056] S31: According to the set detection range radius R epsilon and the minimum point number N minpts , all points in the current frame point cloud data are divided into core points, boundary points and noise points;

[0057] S32: The core points and the boundary points are composed into clustering clusters to find all dense regions;

[0058] S33: The dense regions are taken as a plurality of clustering clusters;

[0059] The specific formula is: Wherein, is a clustering cluster, P S is all points in the current frame point cloud data, is the noise points distinguished from the current frame point cloud data.

[0060] S34: Number and count each cluster of each frame of spatial point cloud, determine whether the count is greater than the point threshold value, if yes, determine as the target gesture object, if not, return to step S31;

[0061] Number and count each cluster of each frame of spatial point cloud, denoted as N det , set the point threshold value as N thre , if N det >N thre , determine as the target gesture object, immediately delete the points in the remaining cluster, filter the remaining interference objects, and only keep one cluster, which improves the accuracy of microwave gesture recognition.

[0062] Further, step S3 includes the following steps:

[0063] S35: Determine all effective points and interference points within the neighborhood radius of the spatial coordinates of the target gesture object;

[0064] Define the spatial coordinates (X human , Y human ) of the target gesture object, and the specific formula is:

[0065]

[0066] Wherein, N is the number of all point clouds, x i represents the horizontal coordinate of the i-th point cloud in space, and y i represents the vertical coordinate of the i-th point cloud in space.

[0067] S36: Determine whether the number of effective points is greater than or equal to the update threshold value, if yes, update the position of the target gesture object, if not, proceed to S37;

[0068] S37: Determine whether the number of effective points is less than the minimum point number N minpts , if yes, proceed to step S38, if not, return to step S35;

[0069] S38: Determine whether the number of invalid points is greater than or equal to the effective point threshold value of the target gesture object leaving and the cluster is not 0, if yes, proceed to 39, if not, return to step S35;

[0070] S39: Do not update the position of the target gesture object, and add 1 to the frame number parameter fram out corresponding to the disappearance of the target gesture object;

[0071] S310: Output the current frame number and the continuous leaving frame number;

[0072] Set the frame number threshold value of the target gesture object leaving the current position or the target gesture object being small as fram outherIf fram out >fram outher The current gesture object is considered to leave, and a new target gesture object is switched to.

[0073] Further, the step S4 further comprises a step S41 of performing feature enhancement on the gesture feature information. The formula of the feature enhancement is A1=10log[10×A0]+bias, wherein A1 represents the amplitude of the point cloud data after enhancement, A0 represents the amplitude of the point cloud data before enhancement, and bias represents a bias constant.

[0074] The following examples are specifically described, first referring to Figure 2 Ten types of gestures are designed, including (a) waving hand upward, (b) waving hand downward, (c) waving hand leftward, (d) waving hand rightward, (e) pushing hand forward, (f) pulling hand backward, (h) turning circle clockwise, (i) turning circle counterclockwise, (j) double-clicking hand, and (k) pushing hand forward twice.

[0075] A frequency-modulated continuous wave millimeter wave radar device with preset configuration parameters is used as a microwave signal transceiving system. The millimeter wave radar device transmits microwave signals in real time and receives echo signals reflected in the action space. When the human hand moves in the action space, the millimeter wave radar device captures raw point cloud data containing current human gesture motion information.

[0076] A digital signal processing unit is used as a microwave data preprocessing subsystem. First, the echo data is processed to obtain raw point cloud data containing gesture motion information of the target scene. Then, the effective point cloud data in the raw point cloud data is screened. After data feature extraction, the data is sent to a computer to call a classification model to obtain corresponding gesture category information, including Doppler radial velocity-time-amplitude (DTA) features, azimuth-time-amplitude (ATA) features, and elevation-time-amplitude (ETA) features. Three two-dimensional features of 10 types of gestures are shown in Figure 3 , Figure 3 For the three two-dimensional feature maps of gestures in the embodiment of the present application, if a gesture exists, the target parameters of the gesture point cloud are extracted to generate gesture feature information. If no gesture exists, no further processing is performed.

[0077] For the gesture feature database system and the gesture recognition system, the gesture recognition system trains different gesture feature information of multiple users offline or online, generates a classification model, after obtaining the classification model, classifies and recognizes the new input gesture feature information by using the classification model, and obtains corresponding gesture category information. For each gesture, 60 sample data are collected by different users of different body types, genders and ages at specific positions in the application scene, abnormal samples are excluded by comparing the feature similarity between the same samples, the sample features in the same class are similar and different between the classes are ensured, and the classification model of the feature samples is obtained by training all feature samples of 10 gestures using a convolutional neural network. The system is initialized on the computer, the classification model is loaded, and the convolutional neural network is referred to as Figure 4 , Figure 4 The convolutional neural network is a schematic diagram of the embodiment of the application, the gesture feature data of the input layer passes through the convolutional layer 1, the pooling layer 1, the convolutional layer 2, the pooling layer 2, the convolutional layer 3, the pooling layer 3 and the two fully connected layers, and finally reaches the output layer, to obtain the classification model of the feature samples.

[0078] A display is used as a man-machine interaction interface system, the current gesture feature information and the gesture category information are displayed on the display, and the gesture feature information and the gesture category information of the measured object are updated in real time.

[0079] The application provides a gesture recognition system and method based on a microwave signal, a microwave signal transceiver system transmits microwave signals to a to-be-measured object and a scene in a detection range, and simultaneously collects echo data; a microwave signal preprocessing system processes the echo data to obtain original point cloud data; valid point cloud data is screened out, it is judged whether there is a gesture, target parameters of the gesture point cloud are extracted, gesture feature information is generated; it is further judged whether the gesture feature information exists in a gesture feature database, if yes, the gesture feature information is called and the gesture feature information and gesture category information of the to-be-measured object are displayed through a man-machine interaction interface, if not, the gesture recognition system classifies and recognizes the gesture feature information by using a classification model, corresponding gesture category information is obtained, and the gesture category information is stored in the gesture feature database. The application can overcome the influence of environmental interference and other user interference on microwave gesture recognition, and improve the accuracy of microwave gesture recognition.

[0080] The application has been described by the above-mentioned related embodiments, however, the above-mentioned embodiments are only examples for implementing the application. It must be pointed out that the disclosed embodiments do not limit the scope of the application. On the contrary, changes and modifications made without departing from the spirit and scope of the application are within the scope of the patent protection of the application.

Claims

1. A gesture recognition method based on microwave signals, characterized in that, The method includes the following steps: S1: The microwave signal transceiver system transmits microwave signals to the object under test and the scene within the detection range, while simultaneously acquiring the echo data reflected by the object under test and the scene, and converting the analog echo data into digital echo data. S2: The microwave signal preprocessing system processes the echo data to obtain the original point cloud data of the object under test and the scene in which it is located; S3: Filter the valid point cloud data in the original point cloud data and determine whether there is a gesture. If yes, proceed to step S4; otherwise, return to step S1. S4: Extract the target parameters of the gesture point cloud and generate gesture feature information; S5: Determine whether the gesture feature information exists in the gesture feature database. If yes, proceed to step S6; otherwise, skip to step S7. S6: Call up gesture feature information and display the gesture feature information and gesture category information of the object under test through the human-computer interaction interface; S7: The gesture recognition system uses a classification model to classify and recognize the gesture feature information, obtain the corresponding gesture category information, and store it in the gesture feature database; Step S3 includes the following steps: S31: Based on the set detection range radius R epsilon and the minimum number of points N minpts The current frame point cloud data is divided into core points, boundary points, and noise points. S32: Form clusters from core points and boundary points to find all dense regions; S33: Treat this dense region as multiple clusters; S34: Number and count each cluster of spatial point cloud in each frame, and determine whether the count is greater than the point count threshold. If yes, it is determined to be the target gesture object; otherwise, return to step S31. S35: Determine all valid points and interference points within the neighborhood radius of the target gesture object's spatial coordinates; S36: Determine whether the number of valid points is greater than or equal to the update threshold. If yes, update the position of the target gesture object. If no, proceed to S37. S37: Determine whether the number of valid points is less than the minimum number of points. If yes, proceed to step S38; otherwise, return to step S35. S38: Determine whether the number of invalid points is greater than or equal to the threshold of valid points where the target gesture object leaves and whether the cluster is not 0. If yes, proceed to step 39; otherwise, return to step S35. S39: Do not update the position of the target gesture object, but increment the frame number parameter corresponding to the disappearance of the target gesture object by 1; S310: Outputs the current frame number and the number of consecutive frames left.

2. The gesture recognition method based on microwave signals according to claim 1, characterized in that, Step S3 uses an interference suppression algorithm to filter the original point cloud data.

3. The gesture recognition method based on microwave signals according to claim 1, characterized in that, Step S4 also includes step S41: performing feature enhancement on the gesture feature information.

4. The gesture recognition method based on microwave signals according to claim 3, characterized in that, The formula for feature enhancement is: A1 = 10log[10×A0] + bias, where A1 represents the amplitude of the enhanced point cloud data, A0 represents the amplitude of the point cloud data before enhancement, and bias represents the bias constant.

Images